Nicolas Pepe gave the hosts the lead at the Stade Jean-Bouin and they were ahead until the 59th minute when Florentin Pogba - brother of Manchester United's Paul Pogba - levelled affairs.
Oussama Tannane notched the winner on 77 minutes to leave Les Verts eighth in the Ligue 1 table, with 21 points from 14 games played.
It was Saint-Etienne's first league victory since the end of October and Galtier hailed the way his charges were able to turn things around to claim the spoils.
"We were slow getting into the game, although there were mitigating circumstances as there were a lot of changes in our team," he said.
"We gifted them a penalty, didn't have any rhythm and made a lot of errors. But we managed to stay in the game, or perhaps our opponents missed the opportunity to bury us.
"It was much better in the second half. I'll take this result with a lot of satisfaction for what the players have done this week to get us back to winning ways, as it had been a while since we won in the league."