The Blues boss is set to put more emphasis on making his side hard to beat by changing personnel and formation after the 2-1 defeat at his former club West Brom.
Chelsea have now conceded 18 goals in their last eight games in all competitions and collected two points from their last four Premier League matches.
The absence of Terry with a knee injury for three weeks after only just returning from a four-match domestic ban has added to their woes ahead of Tuesday’s away Champions League clash with Juventus.
Di Matteo said: “Clean sheets certainly give you a chance to win games and we certainly have to improve on that side. Defensively we just weren’t good enough. Of course we miss John (Terry). He is an important player for us. He gives us a lot of confidence at the back and in general to the team.
“It’s a shame that he is injured – but we will have to do without him until he comes back – and improve.”
Pedro stepped off the bench to send Chelsea into the FA Cup semi-finals after they beat Leicester 2-1 at the King Power Stadium on Sunday evening.
RB Leipzig came from behind at the Red Bull Arena to end a run of four winless games and defeat Bundesliga leaders Bayern Munich 2-1 on Sunday.