Born in Stewartsville, Ohio, Jones began his career with the Cleveland Indians in 1951. When he entered a game on May 3, 1952, 39-year-old rookie Quincy Trouppe, a Negro League veteran, was behind the plate. Together they formed the first black battery in American League history.
After the season, the Tribe traded him to the Chicago Cubs for two players to be named later, one of whom was slugger Ralph Kiner. In , the Cubs traded him to the St. Louis Cardinals in a multi-player deal; prior to the season, he was dealt once again, this time to the San Francisco Giants for Bill White and Ray Jablonski. He was picked up by the expansion Houston Colt .45s in the expansion draft and spent a year with them before being traded to the Detroit Tigers. He rejoined the Cardinals for the campaign and played 1964 with the Baltimore Orioles before retiring.
During his career, Jones was known for his sweeping curveball, in addition to a fastball and changeup. Stan Musial once remarked, "Sam had the best curveball I ever saw... He was quick and fast and that curve was terrific, so big it was like a change of pace. I've seen guys fall down on curves that became strikes."
During his career, Jones led the National League in strikeouts three times: in , 1956, and . On May 12 of the former of these three seasons, he no-hit the Pittsburgh Pirates 4-0 at Wrigley Field, becoming the first African American in Major League history to pitch a no-hitter. His greatest year came with the Giants in 1959, when he led the league in both wins with 21 (tying him with Milwaukee Braves starters Lew Burdette and Warren Spahn) and ERA with 2.83. He was named 1959 National League Pitcher of the Year by The Sporting News, but finished a distant second to Early Wynn of the Chicago White Sox for the Cy Young Award. He was named to the NL All-Star team twice, in 1955 and 1959.
Jones died in Morgantown, West Virginia at the age of 45.