Black graduated from the University of Texas at Austin in 1935 and later obtained a master's degree in English. He received his LL.B. from Yale Law School in 1943, then served in the Army Air Corps as a teacher. In 1947, he became a professor of law at the Columbia University Law School, where he wrote legal briefs for the successful 1954 Brown v. Board of Education suit. He also was involved in civil rights cases in the south.
In 1956, he joined Yale Law School as its first Henry R. Luce Professor of Jurisprudence. He was appointed Sterling Professor of Law in 1975. During his thirty-one-year career at Yale, he wrote numerous books, including The People and the Court, Structure and Relationship in Constitutional Law, Impeachment: A Handbook, and The Law of Admiralty, which he co-authored with Grant Gilmore. An outspoken critic of the death penalty, Professor Black also authored Capital Punishment: The Inevitability of Caprice and Mistake. With Alexander Bickel, Black made Yale Law School one of the world's leading centers for the study of constitutional law.
A lifelong fan of jazz, he was featured in the Ken Burns documentary Jazz: A History of American Music, where he related hearing Louis Armstrong perform at an Austin hotel in 1931. This experience, he said, fomented his interest in race and civil rights.
In his New York Times obituary, former student Akhil Amar commented, "He was my hero. So many of the great moral issues of the twentieth century seem clear in retrospect, but were quite controversial at the time. He had the moral courage to go against his race, his class, his social circle."