The recoilless rifle functions somewhat like a rocket launcher in that its projectile is propelled with a rocket charge that vents at the rear of the weapon. However the difference is that the RCL is not open at the end as in a rocket launcher, and therefore avoids the "backblast" that would otherwise make rockets of this size dangerous to use. Instead the RCL uses a breech-block that vents to the front of the weapon, allowing the propellant gases to be redirected forward to balance the recoil of the projectile. Hence the launcher is "recoilless." The venting of the gas, expensive in terms of propellant, reduces the range.

Unlike a rocket launcher, which fires fin-stabilized rockets from a smooth bore, recoilless rifle rounds resemble conventional artillery shells. They generally have a rifling band preformed to engage the rifled launch tube, spin-stabilizing the projectile, hence the term "rifle.". The "case" area of the shell is perforated to vent the propellant gases to the rear. The rocket stops burning when it leaves the tube, preventing injury to the operator.

The first recoilless rifle to enter service was the Panzerabwehrwerfer 7,5 (or PAW), a 75mm gun developed to give German airborne troops some useful anti-tank support before the Panzerfaust became widespread. The 75 was found to be so useful during the invasion of Crete that a larger 105mm version was developed on the same basic pattern. Interestingly both of these weapons were copied almost verbatim by the US Army, reversing the flow of technology that had occurred when the Germans copied the Bazooka. PAWs remained fairly rare during the war, but the US versions of the 75 started becoming increasingly common starting in 1945. By the time of the Korean War recoilless rifles were found throughout the US forces. The original 75mm and 106mm versions had also been joined by new 57mm and 90mm versions.

As the wire guided missile became more and more popular in the late 1960s and early 1970s, the recoilless rifle started to disappear from the military. The last major use was the Ontos tank, which mounted six of the US 106mm on a light (9-ton) tracked chassis first developed for use by the US Army airborne troops in 1950. However the Army considered them useless, and the Marines picked them up instead, albiet only 176 of them. They used them to great effect as a fire support vehicle during the Vietnam War. The crews continued to report the Ontos was a very effective fighting vehicle in this role, but the military brass continued to argue for heavier designs, and in 1970 the Ontos was removed from service and most were broken up.

Today the only remaining front-line recoilless rifle is the famous Carl Gustav rifle, an 84mm man-portable anti-tank weapon. Similar in conception and use to the Bazooka, the weapon differs primarily in using rifling for stabilization rather than fins, and does not include the complex breech that is the mark of most RCL designs. First introduced in 1946, it is still in widespread use throughout the world today, and has even been re-introduced into the US Army as an anti-bunker weapon.