After a 1-1 draw in Russia in the first leg United went into the tie knowing they still had work to do and the hosts made a bright start at Old Trafford.
Henrikh Mkhitaryan found himself through on goal but fired wide with only the keeper to beat, while Zlatan Ibrahimovic was twice denied by the woodwork before the break.
Rostov barely threatened in the opening exchanges but they nearly took the lead in the second half when Sardar Azmoun forced a save from United stopper Sergio Romero.
Mata broke the deadlock from close range with what proved to be the winner 20 minutes from time after being set up by Ibrahimovic.
Romero was on hand again in the closing stages to deny Christian Noboa from a free-kick as United booked their place in the last eight, with the quarter-final draw set to take place on Friday.
The victory did come at a cost for United however, as Paul Pogba could face a stint on the sidelines after sustaining an apparent hamstring injury in the early stages of the season half.