Amidamaru was training to be a samurai when he was still very young in order to drive away bandits and protect the orphans he was living with. He always used to break his sword (katana) when he was young, so his very good friend Mosuke would always made a new sword after scolding him for breaking another one of his swords as they were too poor to afford good strong steel. Amidamaru was an honorable samurai who refused to steal swords from the corpses and skeletons that were found scattered across the land at the time.
Mosuke, being a very skilled swordsmith, was able to make a sword using his father's knife. The sword cut very well and Amidamaru was grateful for the sacrifice that Mosuke made, so touched in fact that he cried. Mosuke asked if he was crying, but Amidamaru pretends that some raindrops splashed on his face, therefore he names the katana "Harusame" (春雨), which means "Spring Rain"
When both Amidamaru and Mosuke grew older, they were employed by a daimyo who was impressed with Amidamaru's skills as a samurai and Mosuke's skills as a swordsmith.
They worked for him happily, but it did not last long. Amidamaru found out about their daimyo's greed when he asked him to kill Mosuke so that he wouldn't make a sword which would be better than Harusame. Amidamaru was at a dilemma because he owes unrequited loyalty to both the daimyo and his friend.
Amidamaru and Mosuke met at the tree on Monument Hill where Amidamaru told Mosuke what the damiyo told him. At first Mosuke asked Amidamaru to just kill him because an order from a damiyo was an absolute order for a samurai. Amidamaru, being loyal to Mosuke, told him to run away and that he would accept the crime. Mosuke asked him to give him the Harusume so he could fix it up and give it to him later that night (the English anime changes this; Mosuke instead forged the finest sword he could create for Amidamaru, which was named the Sword of Light). Amidamaru agreed and he gave Mosuke his oath that he would wait for him at that very spot. Sadly, someone spied on their conversation and informed the damiyo who ordered his men to kill Mosuke (which was done after they went to the hill). The men then went up the hill hoping to kill Amidamaru, but Amidamaru killed them all. He died of exhaustion because he did not have the Harasume. Mosuke was killed not long after, arriving at the hill too late to present the sword because he had chosen to spend more time perfecting the blade.
Amidamaru's numerous killing earned him a bad reputation which was carried all the way to the present time. In the English manga he is called a fiend for killing so many people.
Yoh was able to find Mosuke inside a museum guarding the Harusame and crying over Amidamaru's loss. Yoh returned the Harusame to Amidamaru, telling him that Mosuke was waiting for him in the other side.
In the anime, while Amidamaru was rising to the other world, he saw Yoh and his friend Manta being beaten up by Ryu, who was enraged because he was beaten by a mere teenager. Amidamaru decided to join Yoh and become his power spirit, and that was how they came to be friends. Yoh was the first one who trusted Amidamaru and didn't judge him based on his reputation in the past because he knew that there was always two sides to the story.
In the manga when Ryu is first possessed by Tokagero, Tokagero steals Harusame from the museum and wields it against Amidamaru. Tokagero explains that this is his revenge on Amidamaru for killing him six-hundred years ago, but Amidamaru and Yoh slice and break the sword saying that they did it because of their friendship with Mosuke.
Eventually, they defeat Tokagero, but they wonder about how they should fix the sword. Anna tells Ryu that this is his chance to make up for what he'd done, and summons Mosuke, making him integrate with Ryu. Yoh in turn integrates with Amidamaru, and the first thing Mosuke and Amidamaru do is punch each other square in the face. Both apologize to each other; Amidamaru for breaking the sword and Mosuke for making him wait so long.