Pierre Lees-Melou gave the visitors a 22nd-minute lead at Stade Geoffroy-Guichard and it took a penalty deep into stoppage time by Nolan Roux for Les Verts to claim a share of the spoils.
The result sees Saint-Etienne down in ninth place in the Ligue 1 standings with 13 points from nine games played, but Galtier feels his charges should have done better.
But, far from blaming his players for the loss of two home points, the 50-year-old tactician preferred to shoulder the blame himself.
"We started very badly and didn't turn up in the first 30 minutes. We had lots of shots but few of them were on target," he said.
"Overall we should have done better. The first person responsible for this draw is me. This was no longer a team that had been patched together. My choices in the starting line-up were not the best. I am a little bit angry with myself."