Intending to become a quiet colonial landowner, he lived instead a turbulent and combative career. During the American Revolution, he played a key role for the Loyalist cause in Georgia.
Instead, he quickly became embroiled in the coming revolution. On 2 August 1775 a crowd of Sons of Liberty confronted him at his house. Brown requested the liberty to hold his own opinions, and finally met their demands with pistol and sword. Taken prisoner with a fractured skull, he was tied to a tree where he was roasted by fire, scalped, tarred, and feathered. This mistreatment resulted in the loss of two toes and lifelong headaches.
The enraged Brown assumed leadership of backcountry Georgia loyalists, and developed a plan to support Augusta area Tories with Indian allies from the West and a landing of British soldiers from the East. He helped bring the plan about by living with the Creeks in 1776 and 1777, gaining their confidence, and establishing a network spreading from Florida to the Carolinas. In 1779 he was appointed Superintendent of Creek and Cherokee Indians and continued his efforts to engage them in the conflict.
Brown came to lead a mounted Loyalist company eventually styled as the King's Rangers, developed this over time into a uniformed and disciplined unit, and became a skilled commander himself with the rank of provincial Lieutenant Colonel. His Rangers fought in LTC Archibald Campbell's 1778 invasion of Georgia, the 1779 Siege of Savannah, and the Loyalist occupation of Augusta in 1780 and 1781, as well as minor backcountry clashes. In September 1780, Brown maintained a stout defense against Elijah Clarke's surprise attack at the First Battle of Augusta, holding the fortified Mackay House until arrival of a relief force. On June 5, 1781 he was compelled to yield Fort Cornwallis in the Second Battle of Augusta after a spirited and creative defense, and was paroled to British authorities remaining at Savannah.
Brown's campaign plan achieved temporary success, but ultimately failed due to tardy or insufficient support from local Tories and his Indian allies. His war career was later vilified, but Cashin's research found no historical evidence that he did anything beyond his duty according to the recognized rules of war. It is unlikely that he hanged thirteen prisoners at the Mackay House with savage relish, rather he imposed (or condoned) a widely approved penalty against parole breakers. Brown angrily denied that he ever encouraged Indians to barbarous behavior.
The British government contined to provide compensation for dispossessed Tories, and Brown's next destination was Abaco Island in the Bahamas. Brown led Abaco Loyalists in protesting lack of representation in the local Assembly, but the point became moot as the Abaco and later Caicos Islands lands proved unprofitable.
In 1799 Brown applied for and received a grant on St. Vincent's Island. His status as a former colonizer still entitled him to a princely grant of several thousand acres, on which he finally lived his gentleman planter dream until his death at Grand Sable Plantation, St. Vincent Island in 1825.