Eduardo Vargas' two late goals had the hosts edging towards a deserved and creditable draw, only for Caulker to put through his own net in the fourth minute of added time.
Richard Dunne's own goal put Rangers behind midway through the second half. Vargas equalised with normal time running out, notching his first goal in English football, only for Philippe Coutinho's deflected effort to restore the Reds' lead.
Vargas put QPR back on level terms moments later, but Liverpool sneaked the final word and the points, with Caulker the luckless fall guy.
Gerrard was not impressed by Liverpool's performance and felt his side would have had no complaints had they left west London pointless.
"We're still in shock to be honest. We're very lucky today," he told Sky Sports 1.
"We would have had no complaints if we came away with nothing, we would have no complaints if we came away with a point. I think we're very lucky to come away with three points.
QPR 2 Liverpool 3 player ratings
QPR 2 Liverpool 3 player ratings
1/22 QPR: Alex McCarthy
Did a solid job deputising for Rob Green – little he could do about any of Liverpool’s goals. 6/10
2/22 Nedum Onuoha
Hardly tested during the first half before going off with an injury. 6
3/22 Richard Dunne
At fault for the Reds’ opener – didn’t adjust his feet quickly enough and turned Sterling’s effort into his own goal. 5
4/22 Steven Caulker
Scored the match-winning goal… at the wrong end. Unfortunate, having worked hard to keep his side in touch as Liverpool threatened to run away with the game having gone 1-0 up. 5
5/22 Yun Suk-Young
Solid performance apart from losing concentration for Liverpool’s opener – defended capably and provided width down the left as QPR dominated the first half – should keep his starting place. 6
6/22 Mauricio Isla
A real threat down the right in the first half, forced to move backwards by Onuoha’s withdrawal at half-time. 6
7/22 Karl Henry
Bullish performance in the centre of midfield. Bullied his more illustrious opponents at times. 7
Influential for 60 minutes before going off injured – his departure coincided with Liverpool taking control of the game. 7
9/22 Leroy Fer
Should have given his side the lead in the first half as he twice hit the bar with great chances – aside from that played well. 6
10/22 Bobby Zamora
The captain’s physicality was a threat to the Liverpool defence right from the opening moment – laid on chances for Austin and Fer that should have been taken. 8
11/22 Charlie Austin
A threat to the Liverpool defence throughout, could have got himself on the score sheet, but did well to set up Vargas’ first. 7
12/22 Liverpool: Simon Mignolet
Another shaky performance – made some good saves but looked extremely vulnerable at times. 5/10
13/22 Glen Johnson
Looked a little rusty on his return to the starting XI from injury – helped prevent a goal with some last ditch defending in the first half and crossed for Dunne’s own goal. 6
14/22 Martin Skrtel
The enforcer of Liverpool’s back four was completely bullied by Zamora and can count himself fortunate (or unfortunate) if he keeps his starting place against Real Madrid on Wednesday. 5
15/22 Dejan Lovren
Unable to handle the inspired Zamora until he went off – unconvincing. 5
16/22 Jose Enrique
Caused a lot of trouble by Isla in the first half – unable to impact the game going forward. 5
17/22 Jordan Henderson
One of few Liverpool players who could be reasonably satisfied with his performance. His mixture of effort and quality helped to keep the Reds in the game as they struggled in the first half. 7
18/22 Emre Can
Making his first Premier League start, the German U21 international failed to stamp his authority on the game and was replaced by Joe Allen soon after the hour mark. 5
19/22 Raheem Sterling
It wasn’t his finest game, but his decisive impact speaks for itself. Played a part in all of Liverpool’s goals and can now looking forward to taking on Real Madrid on Wednesday. 7
20/22 Steven Gerrard
Pushed into a more advanced role in the first half, he was anonymous until almost giving his side the lead just before the break – a bit better when dropping deeper in the second half. 5
21/22 Adam Lallana
Gradually working his way back to full fitness, the England man struggled to get involved during the first half but his shot did lead to Balotelli’s big chance. Went off soon after the hour mark. 5
22/22 Mario Balotelli
He didn’t deserve to be on the winning side. Spurned two great opportunities and generally didn’t look after possession very well. 4
"Our performance needs to be better. I think the only credit we can give ourselves is that we never give up and keep going to the end.
"We got rewarded in the end for hanging in and staying in the game but all around it needs improving."
Raheem Sterling, who started for Liverpool against his old club after the controversy over his England omission last Sunday, admitted he was pleased to be back playing football.
The winger has been an almost permanent fixture on the back pages following England manager Roy Hodgson's revelation that the player had complained of tiredness prior to the Euro 2016 qualifier away to Estonia.
He said: "It's always good to play football and it's no different this week. The manager put me out there and I'm just happy that we got the three points."
Liverpool manager Brendan Rodgers echoed his captain's belief that QPR, who remain bottom of the Barclays Premier League with just four points from eight league fixtures, deserved at least a point.
"They certainly didn't deserve to lose," he said. "I thought we were very, very lucky to get the win.
"We showed tremendous character and were a little bit better in the second half but QPR definitely deserved something from the game."
QPR manager Harry Redknapp declared himself angry at the result but took plenty of heart from his side's performance.
"We didn't deserve that, we were fantastic today, absolutely excellent," the former Tottenham boss said.
"You hope you can replicate that (in future games) and get the points that the performance deserves but I can't help but be angry.
"I'm delighted with the performance, I'm proud of the players, they were just fantastic, but they didn't deserve to come away with nothing.
"It was totally unfair but that's football and sometimes that's life but I'm proud of every one of those boys today, they all gave everything.
"Seven or eight of them started last year in the Championship for us and they've showed today that they can play against one of the best teams in Europe in my opinion, and outplay them really, so there's got to be lots of positives as well."