The hymn was originally written as a Baptist hymn and it is also used by the Salvation Army.[4] The lyrics are based on the Biblical verse in Hosea 6:3.[5]
Cushing wrote this hymn in New York after he became a Christian minister in 1854; he started writing hymns in 1870 when his health declined, forcing him to retire. He wrote more than three hundred hymns.[6]
Cushing once said about his inspiration to write it:
The hymn started to get dropped from certain hymn books during the 1960s; however it is still contained in Baptist hymnals.[7] There is now a movement to bring back certain traditional hymns such as "Follow On" into current usage.[8]
The hymn's music has been adopted for the anthem of the Scottishassociation football club, Rangers, in their club anthem, "Follow Follow".[9] There have been disagreements over the Rangers fans' usage of the hymn's music as their anthem with claims such as from the Boys' Brigade complaining that using it and changing the lyrics prevented it from remaining a major hymn in their organization because of the association with Rangers.[10] The original hymn was sung at their Ibrox Stadium in Glasgow on 3 January 2011 at a memorial service for the sixty-six victims of the Ibrox disaster and their families and friends.[9]