Born Frederick James Karlin in Chicago, Illinois, he studied jazz composition with William Russo and earned a Bachelor of Arts degree from Amherst College, where he wrote his String Quartet No. 2 as his honors thesis. Following graduation, he moved to New York City, composing and arranging for various bands, including those of Benny Goodman, Harry James, and Chubby Jackson. During this period he also composed and arranged for documentaries, the Radio City Music Hall orchestra, and television commercials.
Karlin began his film career with Up the Down Staircase in 1967. Following in quick succession were Yours, Mine and Ours, The Sterile Cuckoo, and Lovers and Other Strangers. For the latter he wrote the music for the song "For All We Know", which won the 1971 Academy Award for Best Original Song and was a major hit for The Carpenters. The Sandpipers charted with another of his compositions, "Come Saturday Morning." Other Karlin scores were nominated for three Academy Awards, including one for the movie The Little Ark (Based on a novel by Jan de Hartog) in 1972, his wife, Marsha, was also nominated for the same film.
Although Karlin continued to score films on occasion (The Baby Maker, Westworld, Loving Couples), the bulk of his work was in television. His compositions were nominated for the Emmy Award eleven times, and he won for The Autobiography of Miss Jane Pittman in 1974. Other notable television projects include Dawn: Portrait of a Teenage Runaway, Alexander: The Other Side of Dawn, Strangers: The Story of a Mother and Daughter, and the Minstrel Man, for which he received an NAACP Image Award.
Karlin wrote three books about film composition, On the Track: A Guide to Contemporary Film Scoring (1990), Listening to Movies: The Film Lover's Guide to Film Music (1994), and 100 Great Film Scores, which was published posthumously in 2005. He also wrote a reference book detailing and cataloging the thousands of recordings the Edison Company distributed between 1914 and 1929.