Bayern edged the clash 2-1 courtesy of headed goals from Thiago and Mats Hummels at the Allianz Arena, moving the defending champions to within three points of leaders RB Leipzig.
However, Voller believes Leverkusen should have been afforded the chance to get back on level terms from the penalty spot on two occasions.
The travelling side were left feeling aggrieved when Javi Martinez appeared to handle the ball in the box, while the Spaniard later bundled into Aleksandar Dragovic.
Voller said: "It was a clear red card for Martinez and a very clear penalty.
"Although we did not score our last [spot-kick], but I would have liked the penalty.
"We were hoping we could get something here, but everyone must play their part, including the referees and linesmen. It's as simple as that.
"Football is not so complicated - sometimes it's quite simple. There are simple things about victory and the game.
"If the penalty is a penalty, then it is a penalty - and [the referee] has to see it."