Long isolated by the surrounding mountains, Hazard met the outside world with the arrival of the railroad in 1912. Previously, the only ways in out of the valley were 45 miles down the North Fork of the Kentucky River, or a 2-week trip over the surrounding mountains. The railroad brought boom times to the town, but the Great Depression saw prosperity end as quickly as it had begun.
The early 1980s CBS television series The Dukes of Hazzard brought visitors to the town, even though the fictional "Hazzard County" was set in Georgia. During and after the popular show's run, members of the show's cast were frequent visitors to Hazard's annual Black Gold Festival.
Beginning in the early 2000s Hazard has seen rapid rise in population, along with several new commercial and residential developments. The city is also actively working on a downtown renaissance plan to rejuvenate its business district.
According to the United States Census Bureau, the city has a total area of 7.0 square miles (18.2 km²), all of it land.
As of the census of 2000, there were 4,806 people, 1,946 households, and 1,266 families residing in the city. The population density was 684.6 people per square mile (264.3/km²). There were 2,291 housing units at an average density of 326.4/sq mi (126.0/km²). The racial makeup of the city was 90.26% White, 6.58% African American, 0.08% Native American, 2.06% Asian, 0.15% from other races, and 0.87% from two or more races. Hispanic or Latino of any race were 0.46% of the population.
There were 1,946 households out of which 30.2% had children under the age of 18 living with them, 42.9% were married couples living together, 18.3% had a female householder with no husband present, and 34.9% were non-families. 31.7% of all households were made up of individuals and 13.7% had someone living alone who was 65 years of age or older. The average household size was 2.30 and the average family size was 2.88.
In the city the population was spread out with 21.9% under the age of 18, 8.5% from 18 to 24, 28.0% from 25 to 44, 23.9% from 45 to 64, and 17.7% who were 65 years of age or older. The median age was 39 years. For every 100 females there were 85.6 males. For every 100 females age 18 and over, there were 82.3 males.
The median income for a household in the city was $20,690, and the median income for a family was $27,226. Males had a median income of $34,398 versus $22,386 for females. The per capita income for the city was $14,782. About 30.9% of families and 30.5% of the population were below the poverty line, including 44.3% of those under age 18 and 13.9% of those age 65 or over.