After the 90 minutes ended 2-2, Rangers scored all their spot-kicks, prevailing 8-7 in the shoot-out.
Defender Jimmy Dunne thumped in the winning penalty after Tom Davies’ kick had been saved by goalkeeper Seny Dieng.
In an entertaining third-round tie, Charlie Austin netted twice for QPR in the first half, opening the scoring and then restoring their lead after Lucas Digne’s equaliser.
Andros Townsend hauled Everton level again a couple of minutes into the second half.
The experienced Austin led the line superbly for Rangers and gave them a deserved lead in the 18th minute.
After Ilias Chair and Chris Willock combined nicely on the left, Sam McCallum’s cross was palmed by keeper Asmir Begovic towards Austin, who headed in from close range.
Everton then had to withstand more pressure as QPR dominated. But the Premier League team were gifted a chance to equalise when Yoann Barbet’s slip let in Alex Iwobi, who was denied by Dieng.
Shortly afterwards, on the half-hour mark, Everton did draw level.
Digne drifted in from the left and met Townsend’s ball into the box with a first-time strike which beat Dieng at his near post.
But Austin restored QPR’s lead with a glancing header into the far corner of the net four minutes later.
Chair swung the ball in from the left and Austin got in front of Ben Godfrey to score his second goal of the night and third of the season.
However, for all Rangers’ enterprising football under boss Mark Warburton, they have often been vulnerable at the back.
Their defensive shortcomings were evident again when they made a mess of trying to clear Andre Gomes’ low corner and Townsend was able to hook the ball home from the edge of the six-yard box.
Everton came on strong in the final stages of the 90 minutes and appealed in vain for a penalty when Abdoulaye Doucoure went down under a challenge from Rob Dickie as he tried to reach Jonjoe Kenny’s cross.
Michael Keane then missed a great chance for the visitors, heading over when unmarked shortly after coming on as a substitute.
In the shoot-out, Mason Holgate, Keane, Townsend, Demarai Gray, Anthony Gordon, Doucoure and Godfrey all scored for Everton.
For Rangers, Austin, Dominic Ball, Barbet, Willock, Albert Adomah, former Everton youngster Stephen Duke-McKenna and Luke Amos, who was making his return after a long-term knee injury, all converted before Dunne clinched victory.
At 6-6, Dieng saved Godfrey’s kick but was deemed to have moved too soon, and the defender put away the retaken penalty.
But there was no doubt about Dieng’s subsequent save from Davies – or Dunne’s emphatic strike.