I always suspected Harpo made tape recordings in connection with the creation of Harpo Speaks.
It's a shock hearing exactly what Harpo did sound like, but, you know, that wasn't the Harpo we saw on the screen except, perhaps when he played the harp.
I think if he had said the one word in "Night in Casablanca" that would have destroyed everything forever. Chaplin was always afraid that if the tramp spoke that would destroy that character, but the tramp wasn't supposed to be mute, and Chaplin's voice--a British one that seemed just a little too refined to be natural--went perfectly with the persona of his character, the bum who wanted to give everyone the impression he was some sort of displaced English nobleman.
Harpo was always a being beyond words. He was not simply mute because of deafness (his character wasn't deaf). He was a being from another realm, a socially inept angel.
All we have of Harpo in costume speaking is him saying the single word "harp" to Ed Sullivan. He's not in costume in the "Great Ziegfield Premiere" clip. There he's Arthur Marx, not Harpo. The spell is not broken.