Jamal Lowe gave Pompey a 19th-minute lead when he fired a close-range effort into the roof of the net for his seventh goal in all competitions this season.
And the 23-year-old should have doubled his side’s advantage just before the half-hour mark when he was in on goal but Gillingham defender Luke O’Neill got back to make a last-ditch tackle.
The visitors struck back through half-time substitute Conor Wilkinson in the 48th minute after he took a touch to set himself up before curling his shot into the bottom corner.
The Gills then took the lead through Martin when his free-kick managed to drift past everyone and find the back of the net.
And Martin got his second and secured all three points for the away side with a spectacular goal in the 80th minute, the Gillingham captain curling the ball into the top corner from outside the area.
Nemanja Matic believes Manchester United boss Jose Mourinho's driven methods and high expectations will bring the club more success.
Cardiff remain unhappy with Derby's decision to postpone Sunday's Sky Bet Championship game at Pride Park.
Dundee United boss Csaba Laszlo has been left hoping for St Mirren to slip up in the race to seal promotion to the Ladbrokes Premiership.