Berman was born in New York City. At age fifteen, he was arrested and tried for attempted rape but found not guilty. He later became an accountant, well known for his ability to figure complex mathematical equations and algebraic expressions in a matter of seconds, without the use of paper or pen. A member of the New York nightlife, he met and befriended writer Damon Runyon. Runyon based his recurring character Regret on Berman, portrayed by character actor Lynne Overman in the movie version of Runyon's story, Little Miss Marker.
In the 1930s Berman became the accountant for, and advisor to, gangster Dutch Schultz. In 1935 Berman was having a meeting at the Palace Chophouse tavern in Newark, New Jersey with Schultz and gunmen Abe Landau and Lulu Rosenkrantz when assassins hired by Lucky Luciano burst into the room. Berman was hit by several bullets as well as buckshot from a 12-gauge shotgun, and was the second of the four men to die. A photo of his bullet-riddled body appeared alongside a photo of Schultz undergoing surgery in the next morning's newspaper, under the headline, "Schultz, Five Pals Shot" (in fact, only four others were shot: Berman, Landau, Rosenkrantz, and Marty Krompier, one of Schultz's lieutenants). The accompanying article claimed that Berman was a gunman for Schultz; an angry Damon Runyon quickly submitted an editorial to the newspaper, defending Berman.
In the novel Billy Bathgate, Berman appears as the mentor to the main character.