Off with Their Heads - in Desolation. Minneapolis punk band Off with Their Heads are well on their way to carving out their own punk rock story with their modern take on anthemic four-to-the-floor rock and roll. In the gritty tradition of bands like the Bronx and Against Me, Off with Their Heads write songs that invite fist-pumping singalongs.