Graham was born in Hillsborough, North Carolina on August 9, 1890. His father, Major John Washington Graham, was a five-term state senator. His grandfather, William Alexander Graham, had served as governor, United States senator, and Secretary of the Navy.
A. H. Graham, a graduate of Harvard Law School, served in the U.S. Army during World War I and later was elected to the North Carolina House of Representatives (1921-30). He was elected Speaker of the House in 1929. In 1930, Graham chaired the search committee that hired Frank Porter Graham (no known relation) as president of the University of North Carolina at Chapel Hill.
Limited to one term as Lieutenant Governor by the state constitution of the time, Graham ran for Governor in 1936, but came in third in the Democratic primary, behind winner Clyde R. Hoey and runner-up Ralph A. McDonald.
Graham later served as head of the State Highway and Public Works Commission (1945-1949 and 1953-1957).