Fagerholm had in his youth briefly been chairman (1920–23) of the Barbers' Union. In 1930 he was elected member of Finland's Parliament. Both in the 1920s and in the 1930s, the main challenge for the Social Democrats was the rehabilitation after the Finnish Civil War, in which the Social Democrats had belonged to the defeated side. A revival of anti-Socialist opinion had in Finland like in many countries in Continental Europe led to a right-wing turn in the public opinion and the emerge of the semi-fascist Lapua Movement. As one of the consequences, Socialists were barred from Cabinet 1929–37. When, finally, principles of parliamentarism again were heeded in 1937, Fagerholm became Minister for Social Affairs in a series of Cabinets 1937–1943.
In government, Fagerholm was one of the chief executors of the neutralist Scandinavia-orientation, that in the 1930s increasingly had been seen as Finland's deliverance from the danger of Russian expansionism – both by Conservatives and Socialists – a danger that seemed to have increased with fierce Soviet anti-Capitalist sentiments being met with equally fierce anti-Bolshevist sentiments in Finland. As native Swedish speaker, Social Democrat, former Union leader, and head of the Ministry for Social Affairs, he was just cut out for this task. The time given was however too short.
When the Winter War broke out, suspicions against Finland's "hazardous foreign politics" remained strong, most importantly among leading Social Democrats in Sweden. As the Winter War ended with the loss of Finnish Karelia, this was generally seen as the failure of the neutralist Scandinavia-orientation. As Soviet dissapproval ended the discussions on a Swedish-Finnish defence cooperation in 1940, the Scandinavist line had run into a blind alley — and Fagerholm had no more say in the policy discussions that ultimately led to close dependency of Nazi Germany, German troops on Finnish soil, revanchism, and to co-belligerence in the Continuation War.
During the Continuation War, controversies on 68,000 refugees internment in labor camps in the vicinity of German troops, and particularly concerning a scandalous deportation of eight Jewish refugees to Gestapo on November 6, 1942, prompted Fagerholm to raise the question of his resignation. After the Battle of Stalingrad, when it increasingly became obvious that Nazi Germany was about to be defeated in World War II, the Cabinet of Edwin Linkomies was appointed to seek peace with Britain and the Soviet Union. Fagerholm was not included.
The end of hostilities in September 1944, found Finland in a thoroughly weakened state economically. In addition to its human and physical losses, Finland had to deal with more than 400,000 evacuees from the territories once again lost to the Soviet Union. War reparations were another burden for Finland. The reparations paid 1944–1952 amounted to an annual average of more than 2 % of Finland's gross national product. The reparations were delivered according to a strict schedule, with penalties for late shipments. By inciting strikes and other disturbances at moments critical for scheduled delivers, the Soviet Union tried both to strengthen the influence of the domestic Communists, and to exert direct pressure on the government.
It was in this phase that Karl-August Fagerholm was elected chairman of Finland's Social Democrats. Väinö Tanner was seen with the uttermost suspicion not only by Finnish Communists, but by many Social Democrats as well, and by the leadership in Sweden and, most importantly, by the Soviet Union. Tanner necessarily had to resign. For the third time, like in 1920 and 1930, Fagerholm's main task was to rehabilitate and consolidate the Social Democrats. This time, however, the accusations came from the left. Fagerholm appeared as the least enthusiastic of the leading advocates for the Continuation War, and could thus be elected as sort of a compromise candidate, accepted by both the Tanner-supporting majority and by the critics of the War, sympatisants of the Six and the so called "Peace Opposition" concentrated in the Swedophone part of the Labor Union and Social Democracy.
The Soviet victory in World War II had greatly improved the prestige of their supporters in Finland. When in 1946 the Communist Mauno Pekkala became after successful elections the Prime Minister for a coalition cabinet including also the Social Democrats, Fagerholm could concentrate on his role as party chairman, remaining outside the cabinet. It was Fagerholm's conviction that Communists couldn't be defeated with repressive methods, as had been tried since the Civil War. Communism was to be countered in free debate and free elections. The battle of the late 1940s was that in the trade unions, where boards and representatives were elected proportionally. The Social Democrats succeeded to fend off or reverse Communist takeovers more often than not.
When new parliament elections were held in July 1948, the Communists suffered a sharp drop in support, falling from fifty-one to thirty-eight seats. Communists had demanded the posts of prime minister and either ministry for foreign ffairs or internal affairs and as the result were not included in the new minority cabinet led by Fagerholm (there was no communist participation in Finland's government again until 1966.) Fagerholm's minority government mostly relied on support by deputies of the Conservative and Liberal parties plus the conservative wing of the Agrarian League.
As leader of the Social Democrats, he was prime minister from 1948 to 1950, despite fierce Communist opposition and accusations by the Soviet Union that he intended to undermine the Finnish-Soviet YYA Treaty by taking Finland into NATO. Even Finland's growing integration into the world economy, as demonstrated by its joining the General Agreement on Tariffs and Trade (GATT) in 1949, was viewed with similar suspicion. It did, however, serve Soviet interests in as much as it contributed to Finland's ability to meet the harsh war reparations requirements of the 1947 Paris Peace Treaty.
Opposition from the domestic Communists (under Soviet guidance) exemplified by: communist strike agitators who in 1949 tried to take over the dockworker's union and indirectly threatened the stability of the government of Karl-August Fagerholm.
The war times, and the post-war times of economic hardship, had motivated government control of prices and wages attempting to hamper inflation. One of the themes in the elections, and hence one of the important objectives for Fagerholm's government, was to transfer the handling of incomes policy to negotiations between the unions and employers' organizations. In January 1950, the Parliament agreed to increase wages and end wage controls.
Soon after this a minority cabinet under Kekkonen assumed power. Wage negotiations ended in an impasse in the spring. A warning of general strike was issued, in order to speed up the negotiations. In this tense situation, Fagerholm, now Speaker of Parliament, brokered a solution which was supported in ballot by a clear majority of trade union members.
Fagerholm's relation to the aged Finnish president Paasikivi remained controversial, although the release of the latter's diary has thrown some light on the issue. Supporters of Fagerholm argue that Fagerholm's line in foreign politics was considerably closer to that of Paasikivi, although Fagerholm in many aspects was ideologically remote from the Conservative fennoman Paasikivi. Supporters of Kekkonen argue that Fagerholm's contact with the U.S. embassy, and pecuniary support from the US, could impossibly have been approved by Paasikivi. Although Paasikivi considered both contenders for the position as Prime Minister as skilful and politically capable, he did in fact come down more often in favor of Kekkonen, who enjoyed the support of the Soviet leadership.
In the presidential elections of 1956 Fagerholm got one elector's vote less than needed to be elected (149 votes), and Urho Kekkonen was elected (151). Fagerholm did instead succeed Kekkonen as Prime Minister for a four-party coalition cabinet.
During his second term as Prime Minister, in 1956–57, he visited the Soviet Union, and relations appeared to have improved. After a period out of office, the general elections of July 1958 again brought a Fagerholm-led coalition cabinet to office. Their chief opponents were the Communists, who had become the largest party in Parliament. Kekkonen in this situation did nothing to mitigate a Kremlin fear that Finland would abandon the careful course steered by Paasikivi and Kekkonen since the late 1940s that sought to ensure that Finland would do nothing that conflicted with the interests of the USSR. Finnish membership in the Nordic Council in 1955 and a progressive increase in trade with the West was seen in Moscow as a harbinger of the "loss" of Finland to the West, particularly under a politician like Fagerholm known for Nordic sympathies and US connections. As a consequence, over the autumn of 1958, the Soviet government pursued an escalating policy of economic and other sanctions against the Fagerholm government, canceling discussions on a range of economic issues and trying to leave little doubt in the minds of Finns that having Fagerholm as prime minister would be exceedingly costly. Finally, the Soviet ambassador was recalled. The pressure worked: on 4 December 1958, Fagerholm filed his resignation. In January 1959, after Kekkonen had traveled to Leningrad to personally assure Nikita Khrushchev that Finland would be a "good neighbor" and a Prime Minister from Kekkonen's Agrarian Center Party was appointed, all economic intercourse resumed.