Faris Haroun returned from the calf injury which had kept him out of Boro's last four fixtures and soon made up for lost time with a first-half brace.
His first goal came after nine minutes when he provided a cool finish to a blistering counter-attack which left Posh stunned.
Any home supporters privy to the knowledge Peterborough had not taken a single point after going behind would not have been surprised to see Haroun head home the second shortly afterwards.
However, Peterborough staged an unlikely comeback when on-loan striker Dwight Gayle struck either side of half-time to turn the game on its head.
The striker, who grabbed the first goal of his loan spell from Dagenham last week against Blackpool, feigned one way then shuffled the ball on to his right foot before rifling into the top corner.
The London Road faithful were taking a shine to Gayle's work ethic and he endeared himself to them even more with the leveller seven minutes into the second period.
The 22-year-old won the corner himself, drawing Jason Steele into a smart stop from a tight angle, before he prodded home Lee Tomlin's set-piece delivery.
Both sides pushed hard for a winner but Miller's speculative effort surprised Posh stopper Robert Olejnik to extend Darren Ferguson's side's winless run to eight games.