It appeared as if the Premier League club were heading out of the competition in extra-time when Newcastle took the lead, with Hull already down to 10 men after Dieumerci Mbokani was dismissed at the end of normal time.
However, the Tigers levelled through Robert Snodgrass and went on to claim a win on penalties, as Eldin Jakupovic saved two spot-kicks.
“On the night we performed heroically in the end and the players stuck to the task,” Phelan told the club's official website.
“We played against adversity at times, but we came through. We came from behind again and that is something we seem to have in our locker at the moment, although we’d rather not have to do it.
“We showed tremendous courage and spirit throughout against a very good Newcastle team that are serious contenders for promotion back to the Premier League.
“We’ve put in an exceptional performance under a lot of stress. Ten men, extra time and penalties, we’ve gone through the lot."