The Swans face a huge fight to stay in the Premier League this season and they are now bottom of the league table.
Under Paul Clement, Swansea have won just two games so far and they have only picked up nine points from their first 15 matches.
Swansea's last win came back in October and they threw away a one-nil lead at the Bet365 Stadium on Saturday.
Olsson admits some of the players had words but he claims it was needed as they try to respond against West Brom at home.
He told the Guardian: "Now it's time to get tough and if you need to dig some players out, you need to dig some players out. It's time for us to be men and go and fight on the pitch.
"When we conceded the first goal, most of the heads go down and we're not playing as aggressive. It cannot happen. We need to be stronger mentally, much stronger, because it's going to be a long season.
"[Saturday] was one of those days where some frustration came out in the dressing room but it happens in a good way – and it had to happen. We're still behind each other, there's some stuff we need to work on.
"Of course it gets a bit heated but in a good way. We all need to wake up as a team and we need to run for each other, create chances, offensively, defensively, like the end of last season."