In 1871, Nogi was commissioned as a major in the fledgling Imperial Japanese Army. Around this time, he renamed himself Maresuke taking a kanji from the name of his father. In 1875, he became the 14th Infantry Regiment's attaché, and for his service in the Satsuma Rebellion, against the forces of Saigō Takamori in Kyūshū, he was promoted to lieutenant colonel. In a fierce battle at that time, he lost the 14th Infantry Regiment’s regimental banner to the enemy, which was considered an extreme disgrace. Nogi considered this such a grave mistake that he listed it as one of the reasons for his later suicide.
The next year (1876), Nogi was named as the Kumamoto regional troop's Staff Officer, and transferred to command the 1st Infantry Regiment.
On 27 August 1876, Nogi married Shizuko, the fourth daughter of Satsuma samurai Yuji Sadano, who was then 20 years old. As Nogi was 28 years old, it was a very late marriage for that time, considering that the average age to marry was in the early 20s. On 28 August 1877, their first son Katsunori was born, and Nogi bought his first house at Nizakamachi, Tokyo. In 1878, he became a colonel. The next year, his second son, Yasunori, was born.
In 1887, Nogi went to Germany with Kawakami Soroku to study European military strategy and tactics.
In 1894, during the First Sino-Japanese War, Nogi served as major general in command of the First Infantry Brigade, which penetrated the Chinese defenses and successfully occupied Port Arthur in only one day of combat. The following year, he was promoted to lieutenant general and assigned to the Second Infantry Brigade, tasked with the invasion of Taiwan. Nogi remained with the occupation forces in Taiwan until 1898. In 1899, he was recalled to Japan, and placed in command of the newly formed 11th Infantry Brigade, based in Kagawa.
Nogi was appointed as the third Japanese Governor-General of Taiwan from 14 October 1896 to February 1898. When moving to Taiwan, he moved his entire family, and during their time in Taiwan, his mother contracted malaria and died. This led Nogi to take measures to improve on the health care infrastructure of the island.
However, unlike many of his contemporary officers, Nogi expressed no interest in pursuing politics.
In 1904, Nogi was recalled to active service on the occasion of the Russo-Japanese War, and was promoted to army general in command of the Japanese Third Army, with an initial strength of approximately 90,000 men and assigned to the capture of the Russia port of Port Arthur on the southern tip of Liaodong Peninsula, Manchuria. Nogi's forces landed shortly after the Battle of Nanshan, in which his eldest son, serving with the Japanese Second Army, was killed. Advancing slowly down the Liaodong Peninsula, Nogi encountered unexpectedly strong resistance, and far more fortifications than he had experienced ten years earlier against the Chinese. The attack against Port Arthur quickly turned into the lengthy Siege of Port Arthur, a dilemma lasting from 1 August 1904 to 2 January 1905, costing the Japanese massive losses, including Nogi's second son. Due to the mounting casualties and failure of Nogi to overcome Port Arthur's defenses, there was mounting pressure within the Japanese government and military to relieve him of command. However, in an unprecedented action, Emperor Meiji spoke out during the Supreme War Council meeting, defending Nogi and demanding that he be kept in command.
After the fall of Port Arthur, Nogi was regarded as a national hero. He led his 3rd Army against the Russian forces at the final Battle of Mukden, ending the land combat phase of operations of the war.
At the end of the war, Nogi made a report directly to Emperor Meiji during a Gozen Kaigi. When explaining battles of the Siege of Port Arthur in detail, he broke down and wept, apologizing for the 56,000 lives lost in that campaign and asking to be allowed to kill himself in atonement. Emperor Meiji told him that suicide was unacceptable, as all responsibility for the war was due to imperial orders, and that Nogi must remain alive, at least as long as he himself lived.
As head of the Peers' School from 1908–1912, he was the mentor of the young Hirohito, and was, perhaps, the most important influence on the life of the future emperor of Japan.
Nogi spent most of his personal fortune on hospitals for wounded soldiers and on memorial monuments erected around the country in commemoration of those killed during the Russo-Japanese War. He also successful petitioned the Japanese government to erect a Russian-style memorial monument in Port Arthur to the Russian dead of that campaign.
Nogi and his wife committed seppuku shortly after the Emperor Meiji's funeral entourage left the palace. The ritual suicide was in accordance with the samurai practice of following one's master to death (junshi). Nogi and his spouse bathed together, and changed into white kimonos, before sharing a cup of sake before the tokonoma. Mrs. Nogi proceeded to commit suicide first; while Nogi assisted by plunging a dagger into her neck. He then sliced his own stomach open. In his suicide letter, he said that he wished to expiate for his disgrace in Kyūshū, and for the thousands of casualties at Port Arthur. he also donated his body to medical science
All four members of the Nogi family are buried at Aoyama Cemetery in Tokyo. Under State Shinto, Nogi was revered as a kami and a Shinto shrine in his honor still exists on the site of his house in Nogizaka, Tokyo.