He lived in Indiana for a time before the Civil War. He graduated from the United States Military Academy in 1841 and was commissioned a second lieutenant in the 3rd U.S. Infantry regiment. In the Mexican-American War, he served under both Zachary Taylor and Winfield Scott. He was breveted three times for bravery and was wounded at Churubusco. Between the wars he served in the U.S. Army Adjutant General's office and as an adjutant in California.
At the Battle of Shiloh, Buell reinforced Grant, helping him defeat the Confederates on April 7, 1862. Buell considered that his arrival was the primary reason that Grant avoided a major defeat. There have been accusations that Grant developed a professional grudge against Buell that would haunt his future career; however Grant gave Buell unwavering praise in his memoirs. Buell continued under Maj. Gen. Henry W. Halleck in the Battle of Corinth. In June and July, Buell started a leisurely movement of four divisions towards Chattanooga, but his supply lines were disrupted by Confederate cavalry under Nathan Bedford Forrest and his offensive ground to a halt.
Buell got himself into more political difficulties during this period. Some Northerners suspected that Buell was a Southern sympathizer because he was one of the few Federal officers who was a slaveholder (he inherited the slaves from his wife's family). Suspicions continued as Buell enforced a strict policy of non-interference with Southern civilians during his operations in Tennessee and Alabama. A serious incident occurred on May 2, 1862 when the town of Athens, Alabama, was pillaged by Union soldiers. Buell, noted for his iron discipline, was infuriated and brought charges against his subordinate on the scene, John B. Turchin. President Abraham Lincoln succumbed to pressure from Tennessee politicians and ordered Maj. Gen. George H. Thomas to replace Buell on September 30, 1862. However, Thomas refused the command and Lincoln relented, leaving Buell in command. Turchin was court-martialled but not cashiered from service as Buell wanted, and was in fact promoted to brigadier general.
In the fall of 1862, Confederate General Braxton Bragg invaded Kentucky and Buell was forced to pursue him to defend Louisville, Kentucky, and the Ohio River. A single corps of Buell's army was attacked by Bragg at the Battle of Perryville on October 8, 1862, while Buell, a couple of miles behind the action, was not aware that a battle was taking place until late in the day and thus did not effectively engage the full strength of his army to defeat the smaller enemy force. Although Perryville was tactically indecisive, it halted the Confederate invasion of Kentucky and forced their withdrawal back into Tennessee. When he failed to pursue Bragg's withdrawal, Buell was relieved of command on October 24, replaced by Maj. Gen. William S. Rosecrans. Buell spent the next year and a half in Indianapolis, in military limbo, hoping that a military commission would exonerate him of blame; he claimed he had not pursued Bragg because he lacked supplies. Exoneration never came, and he left military service on May 23, 1864. Although he had been offered a command at the express recommendation of Grant, Buell declined it, saying that it would be degradation to serve under either Sherman or Edward Canby because he ranked them both. In his memoirs, Grant called this "the worst excuse a soldier can make for declining service."