Whitney was born in Northampton, Massachusetts, the oldest of thirteen children. He was educated at a series of schools in Northampton, Plainfield, Round Hill, New Haven and Andover. In 1836 he entered Yale University where he studied chemistry, mineralogy and astronomy. After graduation in 1839 he continued to study chemistry in Philadelphia and in 1840 he joined a geologic survey of New Hampshire as an unpaid assistant to Charles T. Jackson
In 1841 he was preparing to enter Harvard Law School when he happened to hear a lecture on geology by Charles Lyell. He decided to change career plans and sailed to Europe in 1842 to continue his studies in science. For the next five years he traveled through Europe and studied chemistry and geology in France and Germany.
When Whitney returned home in 1847 he and John Wells Foster were hired to assist Charles T. Jackson in making a federal survey of the Lake Superior land district of northern Michigan, which was about to become a major copper and iron mining region. When Jackson was dismissed from the survey, Foster and Whitney completed it in 1850 and the final report was published under their names. Building on this experience Whitney became a mining consultant and eventually wrote the book, Metallic Wealth of the United States (1854). It was considered to be the standard reference for the next 15 years. During the 1850s Whitney participated in geological surveys of Iowa, Illinois, and Wisconsin.
In 1860 he was appointed the state geologist for California and was instructed by the legislature to undertake a comprehensive geologic survey of the state. To carry out the California Geological Survey, he organized an eminent, multi-disciplinary team, including William H. Brewer, James Graham Cooper, William More Gabb, Charles F. Hoffmann, and Clarence King. They began a survey that covered not only geology and geography, but also botany, zoology, and paleontology. Although significant progress was made, Whitney made a tactical error by first publishing two volumes on paleontology when the legislators were clamoring for information about gold. Whitney argued that the survey should do more than simply serve as a prospecting party. The legislature grew impatient with the scope and pace of the survey work and slowly cut the budget. Whitney tactlessly complained, telling legislators, "We have escaped perils by flood and field, have evaded the friendly embrace of the grizzly, and now find ourselves in the jaws of the Legislature." In 1867 the survey was eliminated from the budget, and work was suspended in 1868.
Although the California Geological Survey ceased work when funds were eliminated, Whitney managed to retain the title of state geologist until 1874. The survey's field work never resumed. In fact, California was left without a geological agency until 1880 when the legislature created the State Mining Bureau. The legislature, after their experience with Whitney, empowered to address only mining issues and set up a board of trustees to keep the new agency focused on that narrow purpose. One or two bureau chiefs tried to broaden the scope to include geology but the bureau was not allowed to hire a geologist until 1928, six decades after the old survey's demise.
The state funded publication and printing of the first three volumes of the survey's results, and Whitney published the remaining reports using his own money. In spite of financial difficulties and political problems, the survey was significant not only for its published results but also because of the men involved and the survey methods developed – in particular, topographical mapping by triangulation.
Whitney also wrote The Yosemite Book (1869), which was essentially a travel guide to Yosemite Valley and the surrounding area. In this work he advocated the protection of Yosemite and was one of the first to propose creation of a national park.
While in California, Whitney became embroiled in two notable controversies. First, Whitney maintained that Yosemite Valley was created by a cataclysmic sinking of the valley floor. However, John Muir, who was exploring the Yosemite area during the same time, argued that the valley was carved by glacial action. Whitney derided Muir as an “ignoramus” and a “mere sheepherder.” Whitney's survey reports suppressed evidence of glaciers, and he never abandoned his viewpoint. Most scientists eventually dismissed Whitney's hypothesis and accepted Muir’s.
The second controversy involved the discovery of the Calaveras Skull, allegedly uncovered by a miner 130 feet beneath the surface of the earth. Eventually the skull made its way into the possession of Whitney who quickly pronounced it genuine and concluded that it came from the Pliocene era (5.3 mya – 1.8 mya). For a variety of reasons, it is now widely accepted that the Calaveras Skull was a hoax; radiocarbon dating puts its age at about 1,000 years.
Josiah Whitney died at Lake Sunapee, New Hampshire, on August 18, 1896.
He was the brother of grammarian and lexicographer William Dwight Whitney.