Student of Hans Bachem, Dominique Heinrich Lemacher Johner and it is choirmaster and organist at the parish church in Esch, and later in Saint-Étienne in Cologne.
After 1933 Zimmermann was organist and choirmaster of the Cathedral Basilica of St. Apostles before being organ professor from 1945.
Expert in restoration of historic organs. Primary responsibility for an organ class, many students today have important functions in the musical life of the Church, both in Germany and abroad.
It deploys an intense concert activity recognized as a true connoisseur of the French organ music (Dupré, Widor, Vierne, Messiaen, Langlais…), he is also dedicated to many mistakenly forgotten works, Rheinberger, Karg-Elert, Sibelius and Glazunov… It creates almost all organ works of Hermann Schroeder especially the Prelude and Fugue “Christ lag in Todesbanden” which specially dedicated to him.
December 18, 1996, he was named an honorary member of the Hermann-Schroeder-Company.
Joseph Zimmermann died on 1 August 1998 in Cologne.