Cole has been struggling with the problem and was forced to sit out Wednesday night's thrilling Capital One Cup victory over Manchester United at Stamford Bridge.
The injury is not thought to be serious but Di Matteo will not take any risks with the defender and will rest him again if he is not fully fit for the Liberty Stadium clash.
When quizzed about the player's chances of making the weekend the Italian revealed: "Hopefully, he'll be okay for Saturday. But I can't say yet."
Ryan Bertrand slotted in at left-back against the Reds and despite being part of a defence that conceded four goals, the youngster will keep his place if Cole misses out again.