Fish will not taste fishy if it's handled properly after buying it at the market. Fresh fish doesn't have a fishy or bad smell. It's only after it's caught and killed that a fish develops a fishy odor.
Washing fish with tap water and treating or cleaning it with vinegar, lemon or tomato juice can reduce the fishy smell. Soaking it in milk for an hour can also remove the fish stench. This is because acidic ingredients and milk help neutralize the fish compounds that cause the fishy smell.
Proper handling and storage of fish should also be observed to ensure that the fish will stay fresh and won't develop a strong odor. When storing fresh fish in the fridge, choose a package that lets the air flow freely and place the packed raw fish in the freezer or the coldest part of the fridge. Freezing is another way of storing raw fish longer. Use an airtight wrapper or container, like a plastic freezer wrap, a heavy-duty aluminum foil or freezer bags for freezing fish. Keep the temperature at zero degrees or lower if freezing fish for about four to six months. This will keep and preserve the freshness of the fish and prevent it from developing a bad smell.