The word is derived from water carts made by a company established by John Furphy: J. Furphy & Sons of Shepparton, Victoria. Many Furphy water carts were used to take water to Australian Army personnel during World War I. The carts, with "J. Furphy & Sons" written on their tanks, became popular as gathering places where soldiers could exchange gossip, rumours and fanciful tales—much like today's water cooler discussion.
Another suggested explanation is that the rumbling of an approaching water cart sounded like the firing of artillery, thus causing a false alarm.
It is possible that the word was also influenced by John Furphy's equally prominent brother, the popular 19th century Australian author, Joseph Furphy (1843-1913). However, Joseph was generally published under the pseudonym "Tom Collins".
Scuttlebutt has a similar etymology, a scuttlebutt originally being a cask of drinking water on a ship.