Lorillard was born on September 7, 1764, in New York. He was the son of Pierre Abraham Lorillard and Catherine Moore. He married Maria Dorothea Schultz in 1788 and they had five children. They lived at 521 Broadway in Manhattan.
Lorillard's father, also known as 'Pierre Lorillard I', was the founder of the Lorillard Tobacco Company. Lorillard's father made the first American tobacco fortune by developing a tobacco firm that he started in 1760. Originally the business was a snuff-grinding factory located in a rented house in lower Manhattan. It was called Lorillard's Snuff and Tobacco company and sometimes, the name was abbreviated as, J. Lorillard. Later the firm moved to a better location that was on the Bronx River. Lorillard II took over and continued to manage and operate the family business after his father's death in 1776.
Lorillard was a prominent citizen of Manhattan. One New Yorker wrote of him, concerning a prison break and the old jail bell and its peculiar sound,
I remember its sounding for a break-out by the prisoners, about the year 1800. Old Peter Lorillard, the tobacconist, was shot by a prisoner whom he tried to arrest. It was some months before he recovered.
Lorillard II was a member of social clubs, one being a fox-hunting club called the Meadow Brook Hunt Country Club and another as the Narragansett Gun Club. He often is associated with Tuxedo Park, sometimes called the first American country club, since between 1802 and 1812 he purchased the first tracts of land upon which it later would be developed. The village and the surrounding area were developed in 1886 by his grandson, Pierre Lorillard IV, as a resort for socially prominent people and was named a country club. The forerunner for the concept was founded earlier, however, in 1882 at Brookline, Massachusetts at The Country Club, Chestnut Hill.
Lorillard died in May of 1843 at the age of seventy-nine outliving his brothers George and Jacob. A newspaper reporter then writing his obituary tried to describe an extremely wealthy American and used the relatively new word, "millionaire".
While the word "millionaire" had been in use since 1816, apparently it was used for the first time in the United States in 1843 when it was used to describe this wealthy tobacco merchant, although Lorillard was not the first American to own one million dollars worth of property. He may have been one of the wealthiest men in America, however, he definitely was not the richest at the time. John Jacob Astor was the wealthiest man in America at that time. Lorillard just happened to have been the first to be called a millionaire in newspapers. Cleveland Amory incorrectly reports that it was in Lorillard's 1843 obituary that the first use of the word "millionaire" was put in print anywhere.
Philip Hone, one-time mayor of New York, wrote in his famous diary about Lorillard,
He was a tobacconist, and his memory will be preserved in the annals of New York by the celebrity of "Lorillard's Snuff and Tobacco." He led people by the nose for the best part of the century, and made his enormous fortune by giving them that to chew which they could not swallow.