Carman was born in Trenton, New Jersey. As a child he played drums in his mother's band. Then as a teen he formed his own group. Around the age of 20, he moved to California.
While attending an Andraé Crouch concert, Carman became a born again Christian, and embraced evangelicalism. In 1980, he made a custom album titled God's Not Finished with Me. The following year, he was invited by Bill Gaither to tour with The Bill Gaither Trio. After relocating to Tulsa, Oklahoma, he released a moderately successful eponymous debut album (later issued as Some-o-Dat) in 1982—which contained mostly novelty songs. Then with the release of the live album Sunday's On The Way in 1983, began a string of CCM chart successes beginning with the title song. As he continued his musical career, Carman established the non-profit organization Carman Ministries. With the 1985 release of The Champion came his first number one song, of the same name. Another live album—Radically Saved—came in 1988. Then 1989 brought Carman his first number one album—Revival in the Land.
Between 1987 and 1989, he was named Charisma magazine's readers' choice for favorite male vocalist. In 1990 and 1992, Billboard Magazine recognized Carman's influence in Christian music by naming him the Contemporary Christian Artist of the Year and in 1995 he recorded his first album in Spanish-Lo Mejor-which includes some of the already recorded songs like Who's in the House and The River. In 2000 the album Heart of a Champion came out with 30 of his best songs including new songs. In 2007 he updated his catalog with Instrument Of Praise.
In November 2011, Carman was involved in an auto accident. He was in the passenger seat when the driver made a left into the church parking lot in front of a crowd of fans waiting to get into the concert. A truck from oncoming traffic struck Carman's side at 50 mph. Carman was able to walk into the church attempting to continue with the scheduled concert, but collapsed soon after. He was taken to a hospital just outside of Houston, Texas, where he was examined and x-rayed. No broken bones or spinal injuries were found. However there were internal injuries that required surgery.
In a photo post made on both his official and personal Facebook accounts on February 14, 2013, Carman revealed that he had been diagnosed with myeloma and given a prognosis of only three to four more years to live.
In late March 2013, Carman announced a Kickstarter campaign for a new album and music video. A short time later, he announced an upcoming sixty-city tour as well, additionally noting that the online fundraising campaign had raised more than $230,000 within several weeks.  As of December 2013, Carman had been in the hospital receiving chemotherapy treatments. By early 2014 however, Carman announced that medical tests had indicated that his body was free of cancer. He plans to continue preparing for his "Live Across America" album and tour.