He played one season in the National Football League for the New Orleans Saints before returning to the University of Alabama to begin his coaching career.
He then spent 17 years in the professional ranks as running backs coach at Tampa Bay, Indianapolis, San Diego, and Green Bay. Before going to Green Bay he served as offensive coordinator for Detroit from 1997-2000, and during his tenure in San Diego was on the Chargers' staff for Super Bowl XXIX.
He was a finalist for the head coach position at the University of Alabama in 2003, but the job ultimately went to Mike Shula. In March 2004, Alabama's Sylvester Croom Commitment to Excellence Award, given annually for 16 years to outstanding players, was changed to the Bart Starr Commitment to Excellence Award, reportedly at Shula's request. The award has since been changed back and has Croom's name attached. Shula originally changed the award because he did not want an award named for a rival coach. After complaints by alumni and fans, the award was changed back to its original name.
After the 2007 season, during which his team won 8 games, including a Liberty Bowl win, Croom garnered Coach of the Year awards from three organizations. On December 4, 2007, Croom was named coach of the year by the American Football Coaches Association for region two. The AFCA has five regional coaches of the year and announces a national coach of the year each January. That same year, on December 5, Croom was named SEC Coach of the Year twice, once as voted by the other SEC coaches and once as voted by The Associated Press. It was the first time a Mississippi State coach received the AP honor since Charley Shira in 1970 and the first time a Mississippi State coach received the coaches award since Wade Walker in 1957.
Elsewhere, in an interview shortly before his first season as a head coach, when asked if as the first African-American coach in the SEC he considered himself "a trailblazer," Croom responded "I'm just a guy trying to do the best job he can. It just happens that the timing of my hiring puts me in that position. I don't see myself that way. If other people perceive that, so be it. I'm just trying to do the best I can here."
However, the initial response to his hiring was lauded by many as a moment of relative cultural significance. An article published in USA Today on the day that Croom was hired listed a few responses from members of the political, cultural, and athletic communities. In it Julian Bond, chairman of the NAACP, noting that the hire was in Mississippi, a state often regarded as having one of the poorest civil rights records, said that "For Mississippi State to place the fortunes of its team in black hands is more than welcome, however long it has taken." In the same article Bennie Thompson, the lone black member of Mississippi's U.S. congressional delegation, said that the hiring "speaks well of Mississippi State. Mississippi State alumni and friends are more concerned about winning than the color of the coach. There's still a lot of work to be done by other schools." 
On February 12, 2007 in observance of Black History Month President George W. Bush, at a gathering of African-American leaders and dignitaries where Croom was present, recognized the efforts and achievements of NFL football coaches Tony Dungy and Lovie Smith, both acquaintances of Croom and a friend in Dungy's case. The President went on to state that he was "proud to be [there] with another football coach who deserves a lot of credit, Sylvester Croom, who is the head football coach from Mississippi State University. His achievement is the first African American coach in the Southeastern Football League -- Southeastern Conference. He was picked because he's a strong leader and a fine man. And I thank you for blazing trails." 
In February 2008 Croom was featured in a half-hour segment of "Say it Loud," ESPN's documentary celebration of Black History Month. In it are featured interviews with Croom along with coaches and players, among others. Relevant to the topic he speaks generally in this documentary on his decision to accept Mississippi State's offer for the head coaching position and on his race and the history of race relations in the region being major contributing factors.
A 2003 article from the Milwaukee Journal Sentinel recalls his experiences of integration as a middle-schooler in Tuscaloosa, the near-lynching of his father years before in a case of mistaken identity, and segregated restrooms, an institution which he said “bothered me [then], and it still does to this day.” In that article and elsewhere like the article in the Washington Post Croom and his family have communicated a cultural and historical connection to the South and an insistence that genuine progress has been made in race relations there. His younger brother Kelvin Croom, a pastor and assistant principal of Paul Bryant High School in Tuscaloosa, said in the Journal Sentinel interview of their experiences in the segregated South that "We chose not to be intimidated. We chose to be motivated and hoped that one day we would make a difference. And we have made a difference, because the crosses have been taken down and the ropes have been put away."