Rosenberg was born in Brooklyn, educated at City College of New York and received a law degree from St. Lawrence College in 1927. Later, he often said he was "educated on the steps of the New York Public Library." From 1938 to 1942 he was art editor for the American Guide Series produced by the Works Progress Administration. Later he was deputy chief of domestic radio in the Office of War Information and a consult for the Treasury Department and the Advertising Council of America. Later, he was professor of social thought in the art department of the University of Chicago.
Rosenberg is best known for his art criticism. Beginning in the early 1960s he became art Critic for the New Yorker magazine. His books on art theory include The Tradition of the New (1959), The Anxious Object (1964), Art Work and Packages, Art and the Actor and The De-Definition of Art. He also wrote monographs on Willem de Kooning, Saul Steinberg, and Arshile Gorky.
Rosenberg was also the subject of a painting by Elaine de Kooning. Along with Clement Greenberg and Leo Steinberg, he was identified in Tom Wolfe's 1975 book The Painted Word as one of the three "kings of Cultureburg", so named for the enormous degree of influence their criticism exerted over the world of modern art.
Saul Bellow wrote a fictional portrait of Rosenberg in his short story "What Kind of Day Did You Have?".