If you do get another copy of "Harpo Speaks", check out chapter 8 "The Silencing of Patsy Brannigan".
Here is an excerpt:
"...It all sounded great to me except for one thing: Uncle Al didn't write a single line for me. I protested. Uncle Al said I could add wonderful contrast to the act if I played in pantomime. The hell with that. I would ad lib all the lines I wanted to, I said. "Okay, okay," said Uncle Al. "Go right ahead."...
"...Champiign, Illinois. The critic in the Champagne-Urbana paper wrote something like: "The Marx Brother who plays 'Patsy Brannigan' is made up and costumed to a fare-thee-well and he takes off on an Irish immigrant most amusingly in pantomime. Unfortunately the effect is spoiled when he speaks."...
..."When I read the review I knew Uncle Al had been right. I simply couldn't outtalk Groucho or Chico, and it was ridiculous of me to try. It was a cruel blow to my pride nevertheless. When I announced to Minnie that I would never speak another word onstage, she knew I had been hurt, and she looked at me with sorrow and sympathy. But she didn't say, "Forget it-what does he know?" She said nothing. I went silent. I never uttered another word, onstage or in front of a camera, as a Marx Brother."
