Together with his brother Richard he took over the family business in 1861. In 1878 they acquired 14 acres (57,000 m²) of land in open country, four miles (6 km) south of Birmingham, where they opened a new factory in 1879.
The Cadbury brothers were concerned with the quality of life of their employees and provided an alternative to grimy city life. As more land was acquired and the brothers moved the factory to a new country location, they decided to build a factory town (designed by architect William Alexander Harvey), which would provide houses at low cost for the employees and their families. This village became known as Bournville after the nearby river and French word for "town". The houses were never privately owned, and their value stayed low and affordable. Bournville was a marked change from the poor living conditions of the urban environment. Here, families had houses with yards, gardens, and fresh air. To this day, the town offers affordable housing.
The brothers cared for their employees; they both believed in the social rights of the workers and hence they installed canteens and sport grounds. Nineteen years after brother Richard died, George opened a works committee for each gender which discussed proposals for improving the firm. He also pressed ahead with other ideas, like an annuity, a deposit account and education facilities for every employee.
George Cadbury was one of the prime movers in setting up The Birmingham Civic Society in 1918 and was married to Dame Elizabeth Cadbury. One of his daughters was Marion Greeves the first lady elected to the Senate of Northern Ireland.