England go into the Brisbane clash on the back of three disappointing performances in the competition, in which they have reached the last-four despite an unconvincing 36-22 win against Papua New Guinea, followed by defeats to hosts Australia and the Kiwis in their pool games.
The media in the UK and Australia have since rounded on England and fiercely criticised them, but Smith is urging the players to ignore outside influences and do their talking on the pitch this weekend.
"If you start believing what other people say about you, then you are in trouble," he told BBC Sport.
"We've got a lot of faith in ourselves. We always have. It's only the people outside the group that have any doubt."
He added: "We need to make some adjustments and play better as a squad, which is taking more time than we would have liked.
"But we are in the semi-finals of the World Cup and you've got to produce your best by the end of it. That's what we're hoping to do.
"There are some areas that we need to tidy up. If we do that, it will be very difficult for us to be beaten.
"We stick tight and that's the only way, in a team sport, that you can go ahead.
"What anybody wants to say, that's fine. We're not too bothered about it. Those who lose faith, I'm glad they are not in my team, because the team hasn't lost faith."