Kaboul saw red as Spurs ended a game with ten men for the second time in a row after clashing with Magpies midfielder Cheik Tiote in the second-half.
The former Portsmouth defender squared up to Tiote after reacting to a tackle by him with 25 minutes remaining, leaving referee Anthony Taylor with no option but to order him off.
He must now serve a three match ban which has infuriated Redknapp, who had been seen Jermain Defoe dismissed only two days earlier in the game against Aston Villa.
"We shouldn't have gone down to 10. It was stupid," said Redknapp.
"It's a lack of discipline. He has got to learn. You are 1-0 up so why do you have to get involved in something like that?
"It's absolute nonsense and he has to learn.
"He is a young boy and has been fantastic for us this year.
"He has been everything I always thought he would be. Now he has become a topclass centre-half but he has gone and got himself suspended for three games."
Aaron Lennon and Gareth Bale scored the goals for Spurs in the victory which has put them up to fourth in the Premier League.
Tottenham last week