The Foxes were quick off the mark to stun the hosts who found themselves two down in the opening ten minutes.
Riyad Mahrez opened the scoring in the fourth minute after his cross/shot from the right hand side missed everyone inside the box and a tricky bounce left Darren Randolph little chance to stop the ball from finding the net.
The visitors then doubled their lead three minutes later after Marc Albrighton put in a fantastic ball that found Robert Huth, with the unmarked German defender heading home.
Full credit has to be given to West Ham who got themselves back into the match and deserved their goal on 20 minutes when Manuel Lanzini fired a brilliant free kick right into the top corner, leaving Kasper Schmeichel with no chance.
Jamie Vardy restored Leicester's two-goal advantage six minutes before the break after West Ham failed to clear their lines from a corner, leaving the England striker to pounce and strike home from close range for his second away goal of the season.
Again West Ham fought back and scored their second on 63 minutes when, from a corner, Andy Carroll rose at the back post to nod it across goal where Andre Ayew was on hand to head home.
It was a thrilling finale with West Ham looking most likely to score the next goal and they almost did in stoppage time but for an outstanding save from Schmeichel, who somehow denied a Carroll shot from only a few yards out.
Leicester hung on however and secured their first away league win of this season.