Born into a poor family in Waco, North Carolina, Patterson was the youngest of eleven children and experienced an insular and troubled childhood. His family moved to Brooklyn, New York, where Floyd was a truant and petty thief. At age ten, he was sent to the Wiltwyck School for Boys, a reform school in upstate New York, which he credited with turning his life around. He stayed there for almost 2 years.
At age fourteen he started to box, trained by Cus D'Amato at his Gramercy Gym. Aged just 17, Patterson won the Gold medal in the 1952 Helsinki Olympics as a middleweight. 1952 turned out to be a good year for the young Patterson; in addition to Olympic gold Patterson won the National Amateur Middleweight Championship and New York Golden Gloves Middleweight championship.
By the 1950s, Patterson was a resident of Rockville Centre, New York.
Patterson's amateur record over 44 fights was 40-4, with 37 knockouts.
Patterson carried his hands higher than most boxers, in front of his face. Sportswriters called Patterson's style a "peek-a-boo" stance.
After beating Tommy "Hurricane" Jackson in an elimination fight, Patterson faced light heavyweight champion Archie Moore on November 30, 1956, for the world heavyweight championship. He beat Moore by a knockout in five rounds, and became the youngest world heavyweight champion in history, at the age of 21 years and 10 months. As a result, he was the first Olympic gold medalist to win a heavyweight title.
After a series of defenses, Patterson met Ingemar Johansson of Sweden, in the first of three fights. Johansson triumphed over Patterson on June 26, 1959, with the referee Ruby Goldstein stopping the fight in the third round after the Swede had knocked Patterson down seven times. Johansson became that country's first world heavyweight champion, thus becoming a national hero in Sweden as the first European to defeat an American for the title since 1933.
Patterson knocked Johansson out in the fifth round of their rematch on June 20, 1960, to become the first man to recover the world's undisputed heavyweight title. Johansson hit the canvas with a thud, out before he landed flat on his back. With blood trickling from his mouth, his glazed eyes staring up at the ring lights, and his left foot quivering, the Swede was counted out. Johansson lay unconscious for five minutes before he was placed on a stool brought into the ring.
A third fight between them was held on March 13, 1961, and while Johansson put Patterson on the floor, Patterson retained his title by a knockout in six to win the rubber match in which Patterson was decked twice and Johannson once in the first round.
Patterson faced a variety of lesser contenders, including 1956 Olympic Champion Pete Rademacher fighting in his first professional match, which led to charges that Patterson was ducking number one contender Sonny Liston. Patterson, stung by the criticism, agreed to fight Liston.
Patterson lost his title on September 25, 1962, by a knockout in the first round to Liston. The two fighters were a marked contrast. In the ring, Liston's size and power proved too much for Patterson's guile and agility. Ten months later on July 22,1963, Patterson attempted to become the first boxer ever to win the world's heavyweight title three times, but Liston once again knocked him out in the first round.
Following these defeats, Patterson went through a depression. However, he eventually recovered and began winning fights again, including victories over Eddie Machen and George Chuvalo. Patterson became the number one challenger for the title now held by Muhammad Ali. On November 22, 1965, in yet another attempt to be the first to win the world's heavyweight title three times, Patterson lost by technical knockout at the end of the 12th round. Despite this loss, Patterson was still a legitimate contender. In 1966 he traveled to England and knocked out British boxer Henry Cooper in just four rounds at Wembley Stadium. In comparison, Ali never scored a knockdown against Cooper in two fights with the Briton.
The World Boxing Association staged an eight-man tournament to determine Ali's successor. Patterson lost a controversial twelve round decision to Jerry Quarry in 1967. Then Patterson, in a third and final attempt at winning the title a third time, lost a controversial fifteen-round referee's decision to Jimmy Ellis in Sweden despite breaking Ellis' nose. This fight also included a disputed Patterson knockdown of Ellis.
Patterson went on, defeating Oscar Bonavena in a close fight over ten rounds in early 1972. However, a final defeat to Muhammad Ali in a rematch for the North American Boxing Federation heavyweight title on September 20, 1972, convinced Patterson to retire at the age of 37.
In 1982 and 1983 he ran the Stockholm Marathon together with Ingemar Johansson.
His adopted son, Tracy Harris Patterson, was a world champion boxer in the 1990s and was trained by Floyd during part of his career.
The New Paltz High School football field was named "Floyd Patterson Field" in 1985.
Floyd Patterson suffered from Alzheimer's disease and prostate cancer and had been hospitalized for a week prior to his death. He died at home in New Paltz in 2006 at age 71. He is buried at New Paltz Rural Cemetery in New Paltz, Ulster County, New York.