Goals from Christian Benteke and Charles N'Zogbia put Aston Villa in control at Villa Park and all the Hammers could manage in reply was an Ashley Westwood own goal late on as Paul Lambert's men held on to claim a much-needed victory.
With time running out, Lambert was grateful to goalkeeper Brad Guzan who produced two fine saves to deny Kevin Nolan and Carlton Cole from equalising.
Allardyce said: "I thought we had saved it but we should never have been in that position. That's the biggest disappointment.
"We should never have got ourselves in a losing position because we'd created the chances. We can't play any better than we played.
"We then created the chances to get ourselves back to 2-2 and we failed to do so and that's been our problem away from home this season.
"It was great goalkeeping in the end. Guzan pulled off some really good saves.
"I thought we sat comfortable from the chances Villa created and we gifted them a soft penalty when we should not have been in that position.
"It was a great free-kick from Charles N'Zogbia so we can't criticise that too much. The overall situation was disappointing because of the way we played. We dominated and need to put the ball in the back of the net when we get a chance."
Defender Joey Brien will miss the Hammers' warm weather break in Dubai this week after suffering a thigh injury.
West Ham last week