A history and literature major while studying at the Moscow Institute of Philosophy and Literature, Shelepin was a guerrilla leader during World War II, becoming a senior official of the Communist Youth League in 1943, and at the head of the successor organization, the World Federation of Democratic Youth, from 1952 to 1958. He accompanied Nikita Khrushchev on the Soviet leader's trip to China in 1954.
Shelepin then became the second head of the Soviet secret police, the NKVD, which had been reorganized and reformed as the KGB after the death of Soviet leader Joseph Stalin. Khrushchev appointed Shelepin in part because of several major KGB defections in the 1950s during the tenure of Ivan Serov as head of the KGB. Shelepin attempted to return state security to its position of importance during the Stalinist era. He demoted or fired many KGB officers, replacing them with officials from Communist Party organizations, and, especially, from the Communist Youth League.
He left the KGB and was promoted to the Central Committee secretariat in November 1961, where it is believed he still exercised control over the KGB, which was taken over by his protégé Vladimir Yefimovich Semichastny. Shelepin became a First Deputy Prime Minister in 1962. He was a principal player in the coup against Khrushchev in October 1964, obviously influencing the KGB to support the conspirators.
Shelepin probably expected to become First Secretary and de facto head of government when Khrushchev was overthrown. Aleksandr Solzhenitsyn suggested that Shelepin had been the choice of the surviving Stalinists in the government, who asked what "had been the point of overthrowing Khrushchev if not to revert to Stalinism?"
Rather, Shelepin's reward was to be made a full member of the ruling Politburo in November 1964--by a significant margin its youngest member. But he still held ambitions of becoming the "first among equals". His colleagues on the Politburo watched him carefully, seeking to halt his ambitions. He survived in that body until 1975, when he rapidly fell from power.