The Millers leapfrogged Pompey in the table and are now just two points off the play-off places after Joe Mattock headed in the winner in the first minute of stoppage-time.
Rotherham, who are unbeaten in eight matches, dominated the second half but solid defending from the visitors looked like it was going to earn them a draw.
However, their former Pompey striker Michael Smith carved out the chance for Mattock to finally break their resolve.
Warne said: “I thought first half we looked the better team but not the most threatening.
At half time I told the lads what I expected and I asked them to make their defenders defend.
“We just kept knocking on the door and had we come off with a 0-0 draw I’d have still been proud. The second half performance was excellent.
” Warne feels his side are becoming more resolute and that is helping with their form.
He added: “They are getting mentally tougher. They are a real tight group and it’s nice to be on a run.
“Portsmouth defensively are the best team we have played here. They threw bodies on the line. On 80 minutes I couldn’t see where we were going to get a goal from.
“When we are really on song we can give everyone a game. Our run in form has probably been down to tightening up at the back because we always look like we are going to score.
“After these next couple of games we will know really what this team can achieve.
” Portsmouth manager Kenny Jackett, returning to his former club, had hoped his side would see the game out.
He said: “There have been a few late goals lately and I thought we had ridden the storm and got through it and the game was petering out.
“We had three chances in the first half and then the second half was an onslaught. We rode it well and battled.
“I thought we had come through it.
“For me it was just another game. We are disappointed to lose because we played the team below us in the league.
“Paul is someone I have always go on very well with a few of the staff.
“It is ever so frustrating to concede in injury time. It is not good, late goals like that are not good in close games and it is a sickener.
Pivotal game in Group E almost a pick ‘em with the bookies