The 25-year-old, who finished the season on loan at AFC Wimbledon, was told earlier this month by manager Chris Powell that he would be released when his contract expired on July 1.
Powell said: "John has been a terrific character, trainer and goalkeeper for us, but he needs to play regularly."
Sullivan has turned down the opportunity to stay with the Dons by agreeing a two-year deal at Fratton Park.
He told Portsmouth's website: "I'm very excited to be joining Pompey. I was down at Wimbledon for the final part of last season. It's a great club and I owe them so much.
"But this is a great opportunity to come to a big club at the start of a new project. As soon as I heard of Pompey's interest, it was an easy decision. This is where I wanted to come.
"I have been at big clubs like Brighton, Millwall and Charlton, but never cemented myself as number one.
"Pompey is definitely the biggest club of my career and I am coming here with one aim - to play and help the club get promoted to where they need to be."