The club was a member of the Athenian League from 1921 to 1963, when it joined the Isthmian League and made its first appearance in the final of the FA Amateur Cup, losing 4-2 to Wimbledon (a game famous for Eddie Reynolds, the Wimbledon forward, scoring all four goals with his head). The club reached the final again in 1969, when, as strong favourites, they lost 2-1 to North Shields. The following year saw the club reach the 4th round of the FA Cup, when they were drawn at home to English champions and European Cup semi-finalists Leeds United. A club record crowd of 14,000 saw Sutton lose 6-0, but they drew praise from all quarters, including the Leeds and later England manager Don Revie. The club reached the final of the FA Trophy in 1981, but suffered their 3rd defeat at Wembley, 1-0 to Bishop's Stortford.
The club won the Isthmian League championship in 1985. After refusing promotion to the Football Conference, they retained the championship the following year and this time accepted promotion. In January 1989, the club beat First Division Coventry City 2-1 in the third round of the FA Cup (the goalscorers were Tony Rains and Matthew Hanlan), joining a small number of non-league clubs to beat top-division opponents in the competition. In 1979 the club became the winners of the Anglo-Italian Cup, beating Chieti 2-1.
In 1991, the club suffered relegation to the Isthmian League, but gained promotion again as champions in 1999. Unfortunately, the stay in the Conference lasted just one season as the U's were relegated again.
Sutton United finished midtable 14th in the Conference South at the end of 2006/2007 season.
After a poor start to the 2007–08 campaign, Ian Hazel resigned to be replaced with Ernie Howe, who was in turn replaced by Jimmy Dack towards the end of the season. It was too late to make any big changes in position and so the team fell down to the Isthmian Premier League.
Sutton United have a fierce rivalry with their nearest neighbours Carshalton Athletic as they are both based in London Borough of Sutton and both used to play in the Conference South together attracting crowds of over 1,000. The most recent game was on 2 Jan 2006 at Borough Sports Ground, ending in a 1–1 draw. Due to Sutton's relegation in 2007-08 from the Conference South and Carshalton Athletic's survival in the Isthmian League Premier Division, the two will meet at least twice next season.