Antonio Conte's Blues were held to a goalless draw in Wednesday night's Carabao Cup semi-final first leg with Arsenal.
They also failed to score at Norwich on Saturday, prompting an FA Cup third-round replay next week.
"Against Stoke we scored five goals but against other teams we have maybe missed too many chances," Courtois told the official club website.
"Like 2-2 at the Emirates (on January 4) and 0-0 last night and at the weekend against Norwich, maybe we have to score more goals from the chances we create because I think we create enough danger, but we need to finish it as well."
Defender Antonio Rudiger added in the London Evening Standard: "I think the last pass is missing, the final part to score the goal. There weren't a lot of chances against Arsenal but some were there. We have to kill a match off."
Chelsea head coach Conte insisted it was not merely the forward players who are culpable, pointing to two missed headed chances by defender Andreas Christensen against Arsenal, who host the second leg on January 24.
"We are conceding less than last season, but we are not scoring like last season. We have to improve," Conte told Chelsea TV.