Rotherham had looked the more likely through the second half and their pressure paid off when Joe Mattock headed in the winner in the first minute of stoppage-time.
Kai Naismith had the first opportunity of the match for Pompey but he sliced Matty Kennedy’s cross high and wide. before Brett Pitman then headed straight at Marek Rodak when well placed.
David Ball had Rotherham’s first effort but his shot was deflected just over before the hosts applied the pressure early in the second half and had a succession of chances from corners.
Richard Wood’s goal-bound header was blocked and Will Vaulks’ follow-up was deflected behind, with Semi Ajayi also having a shot scrambled away. Kennedy threatened for the visitors but his shot from the edge of the box ballooned over.
Former Portsmouth striker Michael Smith was brought on for his home debut and a chance fell to him quickly but he was denied with a well-timed block.
England manager Gareth Southgate has claimed that the country needs to boot racism out of its own football before criticising World Cup hosts Russia.
Newcastle have been linked with a shock move for former Sunderland goalkeeper Simon Mignolet during the summer.