John Marquis' double sandwiched efforts from Tommy Rowe and Mathieu Baudry, while Tom Parkes had levelled for the O's just before half-time.
It was a 16th home defeat of the campaign for Orient, who due in the High Court on Monday having been served with a winding-up order by HMRC.
Rowe's 12th goal of the season put the League Two leaders in front nine minutes before the break when he seized on a loose clearance and crashed a 20-yard shot past recalled goalkeeper Alex Cisak
The O's equalised a minute before the interval when Parkes coolly slid the ball past Ian Lawlor after a counter-attack.
Rovers restored their lead 10 minutes after the break when Orient defender Aron Pollock played the ball across goal and into the path of Marquis, who fired in off the post from 20 yards.
Substitutes Andy Wlliams and Gary McSheffrey both missed easy chances for the visitors before Marquis showed them the way in the 82nd minute when he latched on to a McSheffrey cross to net his 23rd league goal of the season.
Former Orient defender Baudry volleyed home a fourth following a McSheffrey corner three minutes later, leaving Rovers 17 points ahead of fourth-placed Stevenage.