Kelly's success had humble beginnings. He was born in Quebec City, Quebec, to a carter, John Kelly, and Marguerite Migneron. If Joseph-Octave Plessis had not encountered him on the street and sent him to primary school, Kelly might have lived an unremarkable life. As it happened, Kelly, after that schooling, and reaching the age of 14, was sent to the Petit Séminaire de Québec, where his "very great application in all things" could be demonstrated. Elected prefect of the Congrégation de la Bienheureuse-Vierge-Marie-Immaculée in 1802, Kelly was assisted by Jacques Labrie and future Lower Canada Rebellion leader Louis-Joseph Papineau.
Plessis had also furthered his own career, becoming coadjutor bishop of Quebec. In 1803, Kelly had a chance to repay Plessis' investment in him, becoming Plessis' assistant secretary. Two years later, when Plessis was made bishop, Kelly served as diocesan secretary. The next year, 1806, Kelly left his mentor, and Quebec City, to work under François Cherrier, Vicar General at Saint-Denis, a town south of Montreal (and, coincidentally, where Kelly's former assistant Papineau would win a major Rebel victory three decades later). Cherrier was also impressed by Kelly, describing him as "his charming lieutenant vicar". On 9 November 1806, Plessis ordained his protegée as a full Catholic priest.
Kelly found himself (and four of his sisters, whom he brought with him) on a mission to Madawaska, New Brunswick in 1808. Madawaska was both a very large area for one priest, and involved in a territory dispute (see Republic of Madawaska). Kelly arrived in the small community of Saint-Basile, New Brunswick. Saint-Basile had come to Plessis' attention; "[its] church is in ruins, that the presbytery is badly maintained, that the tithes are paid negligently, [and] that luxury, entertainment, and licentiousness reign". Kelly quickly came to blame alcohol. "Even my sexton wants to become an innkeeper, and we already have nine!", Kelly complained. In spite of his efforts, Kelly failed to rebuild the church or presbytery. He was also responsible for the Saint John valley, which demanded his attention. The valley, despite its proximity to Fredericton, also lacked proper church structures. An additional problem was that Kelly had come prepared with an Abenaki dictionary, but the local First Nations people were Malecite.
Kelly spent the period addressing these issues, learning English, and in constant communication with Plessis. "I think that if St Jerome had had to go sixty leagues on snowshoe to obtain absolution and had needed confession as frequently as I, he would have given up the solitary life very quickly" , noted Kelly in January, 1810.
Those six years would be the most comfortable for Kelly. In September of 1817, Plessis once again decided a new challenge was in order. Kelly was posted to the large parish of Saint-Pierre, in William Henry (Sorel). Kelly came face-to-face with the notorious fur traders, who were less than receptive to his sermons. Three years later, Kelly refused an even more difficult assignment from Plessis, further into the northwest, wanting less of the fur traders, not more of them. The Saint-Pierre presbytery may have been preferable to the wilderness, but by 1819 it was declared dangerously unfit for habitation. It wasn't until 1832 that a new presbytery was built, using stone from the original.
Only a year into his new position, Kelly was given responsibility for Île du Pads (Dupas) and Drummondville in another of Plessis' shows of confidence in him. Kelly (perhaps ironically, given his surname), especially objected to Drummondville's Irish population, in part out of his French Canadian nationalist tendencies. Nevertheless, he oversaw the building of a church there, Saint-Frédéric, completed in 1822. "The Protestants are jealous of it, theirs will never look as good as this one". 1824 saw Drummondville given into the spiritual care of a John Holmes, but Kelly maintained authority over Île du Pads.
Kelly maintained his interest in building Catholic educational institutions. Along with Plessis, he participated in a boycott of the Royal Institution for the Advancement of Learning. In contrast to his experience in Saint-Denis, Kelly's Saint-Pierre's educational project bore fruit. An English Catholic school opened in 1831.
Plessis died in 1825, and so did not live to see Kelly raised to archpriest. The ceremony was performed in 1835 by Joseph Signay, successor to Panet as archbishop of Quebec.
Kelly blamed the failures of the trip on the Montreal Sulpicians, who were concerned about Bourget's growing influence, and especially on Jean-Baptiste Thavenet, a Montreal Sulpician who was then living in Rome. Thavenet was a notoriously bad book-keeper, and he and Kelly, while both were in Quebec, had a troubled financial relationship. Thavenet, financial disputes aside, was usually quick to campaign against any further extension of the powers of the bishop of Montreal.
Kelly continued to improve educational opportunities for his congregation after his return. By 1846 the parish library contained four hundred volumes. The late 1840s saw a very different community than Kelly had inherited. A monument to temperance was built in 1848, the bells of which Kelly blessed, and, in the same year, a society for the care of the poor and sick, and for the education of girls, was established under Kelly's supervision. The next year marked the completion of another new presbytery, funded privately by Kelly, who turned the old one into another college.
William Henry was renamed Sorel in 1845. Kelly's work in the area had exhausted him, and by the end of 1849 he entered the Hospice Saint-Joseph at Longue-Pointe, physically, mentally and financially distressed. After a four year stay he died in their care, and was buried in Sorel by Bourget.
William Henry, in spite of the town's remoteness, offered Kelly a chance to be of further use to Plessis. Lord Dalhousie Ramsay, the current Governor General, summered in William Henry. Through Kelly, Plessis (and Plessis' successor, Panet), learned of the governor's response to Jean-Jacques Lartigue's controversial 1820 promotion to auxiliary bishop of Montreal. Kelly was concerned by the governor's reaction; "I believe . . . we must not let ourselves be frightened by that man, that it will be necessary to bare our teeth. . . . We have a right to raise our voice and to make ourselves heard together at the foot of the throne. . . . Because we have been quiet spectators in politics it is thought that we feel nothing. It is quite clear that our enemies bear ill-will not only to the constitution of the country but also to its religion".
In spite of his strong words against the British colonial governor, Kelly was no supporter of either the nationalist Parti Canadien in charge of the House of Assembly. When that local government moved against the authority of the Catholic church in Quebec, Kelly wrote to Panet "It is time that a dike was erected against that body, ambitious and intoxicated by its success with the ministers... it is trying to invade everything and take to itself not only the legislative and ecclesiastical power, but also the executive and judicial power". Kelly also opposed Wolfred Nelson and the nationalists' attempts to build a monument to Louis Marcoux, a nationalist killed in the 1834 elections. As the Lower Canada Rebellion was beginning, Kelly urged Lartigue to speak out against violence; "at this time [your voice] would be much more powerful than all the English bayonets in the country... It would be too late to wait for the government to strike several blows from which the factious would certainly profit to stir up the countryside". Kelly's impression of Lartigue's influence may have been based on Kelly's own position as a powerful voice against outright rebellion in William Henry. In spite of this, Lord Dalhousie attacked Kelly for stirring up anti-government sentiment.
Kelly's involved himself in other disputes in William Henry. In 1839, he was suspected of helping Dr. George Holmes, who had killed Louis-Pascal-Achille Taché, to flee to the United States. Kelly, as a Catholic priest might be expected to, had counselled Holmes against committing suicide. Taché's wife, and Holmes' lover, Joséphine d’Estimauville, was related to Kelly by marriage, which may have encouraged his involvement in the case. Although d'Estimauville visited family in William Henry, and Holmes had briefly been a resident, the murder and the trial took place in Kamouraska, and might not have otherwise called for Kelly's involvement. Kelly's testimony on d'Estimauville's behalf was at least partly responsible for her avoiding a sentence for complicity in the murder. In 1970, Anne Hébert wrote a book, Kamouraska, about the affair, which was made into a film in 1973 by Claude Jutra. Kelly appears in the film as a character, played by an uncredited actor.
Despite, or perhaps because of, Kelly's opposition to both the nationalists and the Protestant colonial government, Kelly was eulogized as "respected by all citizens, Catholic and Protestant".