From 1930 to 1933, he was a member of the "guard formation" of the republican Reichsbanner (Reich Banner). He was a member of the security detail at political functions and a bodyguard for well-known Social Democrat politicians. After emigrating to Czechoslovakia in March 1933, Herbert Gossmann returned to Germany several times as a courier and collected information for the SOPADE, the Social Democratic Party in exile. After his first court conviction in 1933, Gossmann continued resistance activities together with his future wife Else Ratz.
In September 1935, he was again arrested and initially detained in the Sachsenburg concentration camp. Convicted of "conspiracy to commit high treason" by the state supreme court of Dresden, Gossmann first served a penitentiary sentence and was then sent to Buchenwald concentration camp in 1938. Cleverly speaking on his behalf, his fiancée was able to bring about Gossmann's release in October 1940. In the summer of 1943, Gossmann was drafted into Probationary Battalion 999 and severely wounded shortly before the end of the war. He survived.