The Wallabies were hoping to lay down a marker during the summer as they welcomed Eddie Jones' side Down Under.
However, the Aussies were left embarrassed as England ran out 3-0 winners in June before Michael Cheika's men then finished second to New Zealand in the Rugby Championship.
Cheika's side have now won three of their last four matches and Larkham admits the current squad want to gain revenge over England this weekend.
"To lose 3-0 in Australia was very difficult, particularly as a former player and being pretty passionate about Australian rugby," Larkham told BBC Radio 5 live.
"I think we let a lot of people down through that series.
"It was very embarrassing to lose the series 3-0 on our home soil, [so] we're certainly looking forward to this one."