Despite an unbeaten run at home this season, there were only 35,000 present to see them leapfrog fellow European hopefuls Bolton last Saturday and Quinn believes this will hamper them in the transfer market.
"The plan was always to improve and while we are doing that on the pitch, off the pitch the lifeblood of the club is just starting to get to a worrying area," Quinn told the Sunderland Echo.
"I am not here to be the guardian angel of the club and protect it at all costs. We have to be realistic and it would be a shame to start downsizing.
"But I wouldn't be doing my job if I recommended that we should be paying big sums for players and the people weren't supporting the club.
"I know through my time as a player that you do better with bigger houses and it's a better occasion all round. I thought if we were in the top half of the Premier League, we would be getting crowds well into the 40,000s. The reality is we are in the mid-30s."