He was particularly noted for his titanic saxophone duels with fellow tenorman Wardell Gray, a popular live attraction that also produced several albums between 1947 and 1952.

Gordon's sound has been characterized as "large" and his tendency to play behind the beat has often been emphasized. While never considered as revolutionary as Charlier Parker or John Coltrane, his influence can still be heard in many jazz saxophonists. He was also known for his habit of reciting the lyrics of all ballads before performing them.

Gordon notably made two appearances in cinema during his life. The first movie appearance occurred, oddly enough, while in prison for possesion of heroin. He portrayed an inmate playing in the prison band in Unchained, though the soundtrack was later overdubbed.

He was composer, musician, and actor in the play The Connection in 1960. After that, he spent 15 years playing and living in Europe, mostly in Paris and Copenhaegen. He concerted regularly with fellow ex-pat jazzmen like Bud Powell, Freddie Hubbard, Bobby Hutcherson, Kenny Drew, Horace Parlan and Billy Higgins. He also visited the States occasionally for recording dates, resulting in several albums being released under his name.

Gordon finally returned to the United States for good in 1976, and appeared in the Village Vanguard, NY for a gig that was dubbed as his "homecoming". After this appearance he would release several more albums that proved he was as good if not better than before his years in Europe and finally gained appreciation as one of the great jazz tenors.

In 1986, he starred in the movie 'Round Midnight, in which he played the role of a musician much like himself. He was nominated for the Academy Award for Best Actor. Gordon is a member of the Jazz Hall of Fame and was voted musician of the year by Downbeat magazine in 1978 and 1980.