Andre Villas-Boas picked a strong starting line-up for the Europa League trip to Greece and his team dominated the majority of the game.
Spurs had a strong penalty shout on only two minutes when Andre Pinto looked to have handled in his own area, but the referee waved away their protests.
Clint Dempsey and Aaron Lennon both had good opportunities before Michael Dawson put the Premier League side ahead on 34 minutes.
The central defender's flicked header from a Tom Huddlestone cross gave Tottenham a deserved lead, before Gareth Bale and Huddlestone both forced saves from the Panathinaikos keeper before half-time.
A sedate second-half looked to be going Spurs' way as they dominated proceedings, but Toche pounced on 76 minutes to slot home from a Giourkas Seitaridis ball and pull the home side level.
Spurs could not find a second goal and will regard the draw as two points lost after they let a game they were controlling, slip through their grasp in Greece.