David Worrall had opened the scoring for Vale inside 10 minutes, and the Mariners had to wait until the death to get back on level terms.
James McKeown also saved a Michael Tonge penalty shortly after the hour.
New boss Jolley, who has succeeded Russell Slade, made five changes while the Valiants were missing top-scorer Tom Pope.
Once the action began, the hosts looked to get on the front foot, and they forged a first chance when Siriki Dembele struck over on the half-turn.
However, soon after at the other end, the visitors dealt a hammer blow.
A cross from the left was not dealt with by Town – and Worrall was on hand at the far post to take a touch and drill into the bottom corner.
Grimsby had chances in the second half, but they left it late as Berrett popped up in the Vale box to equalise with a cool finish.
England manager Gareth Southgate has claimed that the country needs to boot racism out of its own football before criticising World Cup hosts Russia.
Newcastle have been linked with a shock move for former Sunderland goalkeeper Simon Mignolet during the summer.