O'Sullivan was born in New York City. As a teenager, he was employed by Mathew Brady. When the Civil War began in early 1861, he was commissioned a first lieutenant in the Union Army and, over the next year, fought in Beaufort, Port Royal, Fort Walker, and Fort Pulaski.
After being honorably discharged, he rejoined Brady's team. In July 1862, O'Sullivan followed the campaign of Maj. Gen. John Pope's Northern Virginia Campaign. By joining Alexander Gardner's studio, he had his forty-four photographs published in the first Civil War photographs collection, Gardner's Photographic Sketch Book of the War In July 1863, he created his most famous photograph, "The Harvest of Death," depicting dead soldiers from the Battle of Gettysburg. In 1864, following Gen. Ulysses S. Grant's trail, he photographed the Siege of Petersburg before briefly heading to North Carolina to document the siege of Fort Fisher. That brought him to the Appomattox Court House, the site of Robert E. Lee's surrender in April 1865.
From 1867 to 1869, he was official photographer on the United States Geological Exploration of the 40th Parallel under Clarence King. The expedition began at Virginia City, Nevada, where he photographed the mines, and worked eastward. His job was to photograph the West to attract settlers. O'Sullivan's pictures were among the first to record the prehistoric ruins, Navajo weavers, and pueblo villages of the Southwest. In contrast to the Asian and Eastern landscape fronts, the subject matter he focused on was a new concept. It involved taking pictures of nature as an untamed, un-industrialized land without the use of landscape painting conventions. O'Sullivan combined science and art, making exact records of extraordinary beauty.
In 1870 he joined a survey team in Panama to survey for a canal across the isthmus. From 1871 to 1874 he returned to the southwestern United States to join Lt. George M. Wheeler's survey west of the One Hundredth Meridian. He faced starvation on the Colorado River when some of expedition's boats capsized; few of the 300 negatives he took survived the trip back East. He spent the last years of his short life in Washington, D.C., as official photographer for the U.S. Geological Survey and the Treasury Department.