India's one-run victory over South Africa in Colombo was not enough to book a World Twenty20 semi-final berth.
Pakistan's win over Australia earlier in the day had rendered South Africa's hopes of qualification redundant but India needed a large margin of victory over the Proteas in their final Super Eights match to seal their passage and deny Pakistan a place in the next stage.
Suresh Raina top-scored with 45 as India reached 152 for six but they were unable to restrict their opponents to a score of under 122 and they eventually just squeaked to victory after South Africa were all out for 151, with Faf du Plessis' 65 off 38 balls not enough to get them over the line.
Gautam Gambhir got India off to a fantastic start after they had lost the toss, punching Dale Steyn through the covers for four off the first ball of the innings.
It was a rare highlight for the left-handed opener, who had his middle stump knocked out after inside edging Morne Morkel in the fourth over while India were two down eight balls later when the in-form Virat Kohli was caught behind down the leg side off Jacques Kallis.
Needing to rebuild India were then left further in the mire when Virender Sehwag was bowled by Robin Peterson for 12 after brainlessly charging down the pitch, having just hit the slow left-armer's previous delivery for a huge six.
Yuvraj Singh's cameo of 21 from 15 balls boosted India but the all-rounder was cleaned up after being deceived by a slower ball from the impressive Morkel and it was left to Rohit Sharma and Raina to rebuild.
The two put on a crucial partnership of 44 for the fifth wicket and although Sharma was trapped in front by Peterson for 25 off 27 balls, Raina and captain Mahendra Singh Dhoni (23 not out) steadily accumulated in the closing stages to aid India's cause.
Dhoni's men needed to restrict South Africa to a total of 121 or below to qualify for the semi-finals and they got off to a great start when Hashim Amla edged Zaheer Khan's second delivery to slip.
Fellow opener Kallis pulled Irfan Pathan to Rohit at midwicket and India looked to be in a superb position when Proteas captain AB de Villiers chopped Yuvraj onto his stumps.
Du Plessis ensured it would not be plain sailing for India, however, and he completed a fine half-century with an inside edge for four off Yuvraj, who would have his revenge when the South African was caught by Raina at long-off.
His innings, which contained six fours and two sixes, helped De Villiers' side pass 122 - to the delight off the remaining Pakistan fans at the R Premadasa Stadium - although Zaheer took two wickets in as many balls to finish with three for 22 and give India hope of finishing their tournament on a positive note.
Sixes from Albie and Morne Morkel in the final over left South Africa needing two off the final two balls but Lakshmipathy Balaji, having dismissed the former off his second ball, then bowled the latter to claim his third wicket and seal victory for India by the tightest of margins.