While the "Index" of banned media is continuously shrinking since years, this work might be one of the few new additions to it. Once the BzKJ (Bundeszentrale für Kinder- und Jugendmedienschutz - Federal central office for children- and youth-media-protection) determines that the work violates § 18 (1) JuSchG, the work is no longer allowed to be sold over the counter, put in display, or advertised to anyone in a place where children legally can be present. Sales in kiosks are illegal, and internet sales are seriously restricted. Those restrictions follow from § 15 (1) Nr. 3 JuSchG.
Among others, the text of "Heil Hitler" qualifies because it violates at least § 86 StGB (publication of propaganda of a banned organisation - here: NSDAP and the Hitler speech at the end), § 86a StGB (use of the propaganda means and marks - here: Heil Hitler. See also: this and this) § 130 StGB (Volksverhetzung - Hate speech).
As a result, I suspect the work will end on the Index with the may or at latest june report. However, even before the work is on the Index, the violation of the §86 & 86a StGB makes public use of a recording illegal: the person causing the music to play is liable and will be prosecuted for the song. Should the work be deemed to violate § 86 & 86a StGB, selling and trading of a copy of the song becomes a crime outside of a very narrow exception for research and teaching, and possession is illegal if an intention to distribute can be proven.
Conclusion
As such, yes, it's illegal to play an "Indexed" song in public so you can listen to it in a fashion that it can be noticed by the public. This song in particular is most likely illegal to be played even without an Index-entry. Note that the crucial part here is that it's not the (passive) listening but the (active) playing of the recording that is illegal.