After two years on the frontier, he was moved back to Fort McHenry and Barrancas Barracks and promoted to First Lieutenant on July 1, 1854. In 1855 he was sent back for five more years of frontier-duty, this time serving in Kansas, Nebraska and Indian Territory with a brief stint at Fort Monroe.
When Sumner was summoned to lead the Department of the Pacific in California on June 10, 1861, Long resigned his commission in the United States Army the same day, and accepted a rank of Major and Chief of Artillery in the Provisional Army of the Confederate States in western Virginia. He was later promoted to colonel dating from March 1861. Long accompanied its commander General William Loring in the Trans-Alleghany. During the summer and autumn of 1861, he acted as Inspector General, in addition to his regular duties.
He was summoned to appear before General Robert E. Lee at the end of the year, and the two of them would foster a close friendship over the coming years. Long was transferred to the southern coast as chief of artillery for both Generals John C. Pemberton and Robert E. Lee. When Lee became the military advisor to President Jefferson Davis in early 1862, he appointed Long as his military secretary with the rank of colonel. When on May 31, 1862, Lee took command of the Army of Northern Virginia, Long assumed his military secretary position on Lee's staff. He served Lee through the Seven Days, Second Bull Run Campaign of August 1862, Antietam Campaign, Fredericksburg Campaign, the campaign culminating in the Battle of Chancellorsville, and the Gettysburg Campaign. Long was considered a valuable asset with "vigor and unfailing judgment".
On September 23, 1863, Long was promoted to brigadier general and placed in command of the artillery in Lieutenant General Richard S. Ewell's Second Corps. He was present at both the Bristoe Campaign and the subsequent Mine Run Campaign. Long commanded the artillery through the Overland Campaign, including the battles of the Wilderness, Spotsylvania Courthouse, and Cold Harbor. During Early's 1864 Shenandoah Valley Campaign he served intermittently as artillery chief for the Army of the Valley, but was often absent due to illness. He returned with the rest of the army to rejoin Lee and the Army of Northern Virginia around Petersburg and Richmond. He surrendered with the remnants of Lee's army at Appomattox Courthouse in April 1865.
In 1864, his wife gave birth to their second child, Edwin Vose.
His final engagement was alongside the army of Jubal Early in the Shenandoah Valley, where he again organised the artillery campaign. He was present at Waynesboro, and was engaged in the retreat from Richmond until the units surrendered in April 1865. Suffering from facial Neuralgia, Long was paroled at Appomattox.
Following the war, Long was appointed Chief Engineer at the James River and Kanawha Canal from 1866–1869. On April 12, 1869, his wife gave birth to their third child, Eugene Mclean. The following year Long went blind, ostensibly from his exposure during military service.
On October 28, 1875 Long was elected to a post of Vice President of the Association of the Army of Northern Virginia along with Robert Ransom, Harry Heth and William Terry, under the association's president William Henry Fitzhugh Lee.
President Ulysses S. Grant appointed his wife as postmistress for Charlottesville, Virginia. Although entirely blind, Long moved to Charlotteville and began writing Memoirs of Robert E. Lee. The lengthy biographical compendium of the service of the Army of West Virginia was published in 1886, although his manuscripts on Stonewall Jackson and his own army life were never published.
Following three years of ill health after the death of his eldest son, Long died April 29, 1891 after requesting that his daughter build a fire in his room and bring him coffee. By the time she returned, he had slipped from consciousness and died shortly thereafter. He is buried in Maplewood Cemetery, Charlottesville, Virginia.