The Hammers have won just one of their last seven Premier League matches and slipped back into the bottom half of the table following the 3-2 loss as the Foxes recorded their first away win of the season.
Having faced serious questions about his future earlier in the season before a mini-revival, Bilic appears under pressure once again after the co-owner expressed his disappointment at Saturday's result.
Sullivan came out after the game to apologise to the home fans while also criticising the team for having shipped two early goals, for the second week running, after the 3-2 defeat at Bournemouth the week before.
He said: ”I think all we can do after Saturday’s defeat is say is ‘sorry’ to the supporters.
“To be 2-0 down after seven minutes of any football match, be it in the Premier League or in your local Sunday League, makes it very difficult to get a positive result.
“In parts of the second half we were very good. But we need to be good before we are losing.
“We have to find out why we are conceding so many goals and fix it because it is obviously harder to win games when you concede two or three goals in a game than if you keep a clean sheet.
“The next few games now become vital for our season. We go to Hull, then to Arsenal, before we welcome Swansea to London Stadium.”