The hosts went ahead in the 23rd minute, Amadou Bakayoko drilling his first league goal past Jayson Leutwiler from Oztumer's fine through ball.
Shrewsbury levelled on the half hour as former Walsall player Oliver Lancashire looped home a header from Ian Black's free-kick.
The Saddlers regained the lead three minutes before half-time, Kieron Morris bundling home Joe Edwards' cross from inside the six-yard box.
Oztumer should have sealed the points 10 minutes into the second half but fired wide one-one with Leutwiler.
And the Shrews levelled in the 58th minute as Walsall goalkeeper Neil Etheridge missed his punch from a corner and Junior Brown bundled home from close range.
Shrewsbury were then denied a goal and a penalty by the linesman as both an Andy Mangan strike and a foul on George Waring in the box by James O'Connor were cancelled out by offside flags.
The visitors went down to 10 men after 65 minutes as Ryan McGivern received a second yellow card for an off-the-ball clash with Bakayoko.
And Walsall won it two minutes from time as Oztumer's corner crept in direct at the near post, although they also ended with 10 men as Bakayoko saw a second yellow for his celebrations.