The Lillywhites' father died in 1854 and was buried in Highgate Cemetery, North London. The following year Fred went into partnership with the Sussex all-rounder, John Wisden (1826-84), with whom he established a tobacconist and outfitter in New Coventry Street, near Leicester Square, in the West End of London. This partnership did not survive the tour to the United States and Canada in 1859 that Lillywhite (though not himself a player) organised and of which Wisden was a prominent member.
By 1860 James Lilywhite (the elder brother) was cricketing coach at Cheltenham College, Gloucestershire, where he also ran an outfitters. John Lillywhite, who had also joined the 1859 tour, was then running a cricketing warehouse near Euston Square, London. This was the forerunner of the present Lillywhites, established in Haymarket in 1863, that, following its acquisition in 1922 by I H Benedictus, moved to the Criterion site in Piccadilly Circus in 1925.
In 1865 the Marylebone Cricket Club withdrew its support for Lillywhite's Guide. This falling out with the cricketing establishment seems to have arisen from the trenchancy of some of Lillywhite's observations. Significantly, in 1866, Wisden noted that "John Wisden & Co have avoided making remarks upon the play or players".
In the 1865 edition Lillywhite proposed that the standard height of the wicket should be raised from 27 inches to "28, 29, or even 30 inches out of the ground" to help avoid what he regarded as excessively high scores. This recommendation was implemented over sixty years later, in 1931, when the height was increased by an inch and the stumps were also widened .
The touring party of 1859 left Liverpool on the SS Nova Scotian on 7 September and returned on 11 November. Its members, in addition to Wisden and John Lillywhite, were the captain George Parr (1826-91), Julius Caesar (1830-78), William Caffyn (1828-1919), Robert Carpenter (1830-1901, Alfred Diver (1824-1876), James Grundy (1824-1873), Tom Hayward (1835-76), John Jackson (1833-1910), Tom Lockyer (1826-1869) and H H (Heathfield Harman) Stephenson (1833-1896), who later led the first private tour by an England XI to Australia in 1861.
Fred Lillywhite travelled with his groundside tent and printing press. His role on the tour has been described as that of "scorer, reporter, and mentor, not to say Nestor" .
The team won all five official matches against a 22 of Lower Canada (by 8 wickets at Montreal, Quebec on 26 October-27 September), a 22 of the United States (by an innings and 64 runs at Holboken, NY on 3-5 October), a different 22 of the United States (by 7 wickets at Philadelphia on 10-12 October), a 22 of Lower Canada (by 10 wickets at Hamilton, Ontario on 17-19 October) and a further 22 of the United States (by an innings and 68 runs at Rochester, NY on 21-25 October).
There were also some exhibition matches and an impromptu game of baseball when a match in New York was interrupted by snow. The team made two excursions to view the Niagara Falls.
Lillywhite's detailed account of the tour, The English Cricketers' Trip to Canada and the United States, was published in 1860 and reprinted over a century later, in 1980. There was coverage also in the 13th edition of his Guide to Cricketers and Caffyn gave an account in a memoir, Seventy-one Not Out, published at the end of the century.
Lillywhite died on 15 September 1866 at the age of 37.