Kean, who was in charge of Blackburn between December 2010 and September 2012 prior to his resignation, was incredibly unpopular during his stint at Ewood Park, during which time the team were relegated from the Barclays Premier League.
A large number of fans were vocal in their dislike of both Kean and the Venky's, who still own the club today, and the Scot thinks he was held up as a public scapegoat.
"I was appointed by the owners and I don't think the relationship between the fans and owners has got any better. Because of that I was getting a hard time," he told Sky's Goals on Sunday programme.
"There were other sections of the (Blackburn) fans and fans of other clubs who were struggling to get their head around it (the hatred).
"Being in the club that's what I felt. I can count on one hand when we were off it. We had a great dressing room, a lot of the performances show we had a great togetherness."
When pushed on if he thought the Venky's regime had had a detrimental effect in east Lancashire, Kean sat on the fence.
"I don't want to say if it's good, bad or indifferent," he added. "The stats are there, the amount of players that were sold, but that's out of the control of the manager.
"I can understand why fans would rather see owners buying players but when that's out of your control you've just got to try and do your best on the pitch."