Palmieri attended the city's public school system and here was constantly exposed to music, specifically jazz. He took piano lessons for and performed at Carnegie Hall when he was 11 years old. His biggest piano influences were Thelonious Monk and McCoy Tyner. He was inspired by his older brother and was determined to someday form his own band - something he achieved in 1950, when he was just 14. During the 1950s, Palmieri played in various bands, including Tito Rodriguez's.
In 1971, Palmieri recorded Vamonos Pa'l Monte (Going to the Mountain) with his brother Charlie at the organ. That same year he also recorded Eddie Palmieri & Friends in Concert, At the University of Puerto Rico. In 1974, Eddie won the first ever Grammy Award for Best Latin Recording with The Sun of Latin Music. On July 21, 1979, he appeared at the Amandla Festival along with Bob Marley, Dick Gregory and Patti LaBelle, amongst others.
In the 1980s, Ismael Quintana returned to the band, which also included Cheo Feliciano. Palmieri won two Grammys for the recordings of Palo Pa Rumba and Solito. He also recorded the album La Verdad (The Truth) with salsa singer Tony Vega in 1987. Next year the happiness of his success was set back by the sudden death of his brother, Charlie.
In the 1990s, Palmieri had participated in various concerts and recordings with the Fania All-Stars and the Tico All-Stars; he also introduced La India with the production of Llego La India via Eddie Palmieri (La India has arrived via Eddie Palmieri), released in 1992. In 2000, Palmieri announced his retirement from the world of music. However, he recorded Masterpiece with Tito Puente and won 2 Grammys; additionally he was also named the "Outstanding Producer of the Year" by the National Foundation of Popular Culture. Palmieri has won a total of 9 Grammy Awards in his career, most recently for his 2006 album Simpático.
Palmieri's "La Perfecta" departed from the traditional Caribbean sources of salsa instrumentation by introducing a new stylistic device into the New York Latin sound. Their signature sound relied heavily on two trombones and a flute instead of trumpets. On the liner notes of their first album, Eddie's brother Charlie dubbed this combination the 'trombanga', referring to the trombones and the still popular charanga which typically featured the flute . The combination helped to give La Perfecta a rich and bold sound which contributed to Palmieri's success with his new band.
The trombone-based sound was later adopted by salsa band leader Willie Colón among others.
In November/December 2005, Palmieri teamed up with longtime trumpeter and bandmember Brian Lynch to record the Artistshare CD release, "The Brian Lynch/Eddie Palmieri Project: Simpático." This CD and accompanying multimedia web site features music by an all-star roster of jazz and Latin jazz artists, including: Phil Woods, Lila Downs, Donald Harrison, Conrad Herwig, Giovanni Hidalgo, Gregory Tardy, Mario Rivera, Boris Kozlov, Ruben Rodriguez, Luques Curtis, Robby Ameen, Dafnis Prieto, Pedro Martinez, Johnny Rivero, Edsel Gomez, Yosvany Terry. In 2007, the recording was awarded a Grammy as the best Latin Jazz Recording.