The nation ranked 146th in the world, 133 places below their storied opponents, recovered from Andrea Belotti's early opener goal to take a 2-1 lead just before the hour with two goals in three minutes from Illija Nestorovski and Ferhan Hasani.
With nerves jangling the four-time World Cup winners were indebted to Lazio's Ciro Immobile who scored twice himself, the latter in the second minute of added on time to snatch victory.
It ensured Italy kept pace with new Group G leaders Spain, who were frustrated for almost an hour by Albania before Diego Costa and Nolito secured a 2-0 win.
Newcastle striker Aleksandar Mitrovic also scored twice as Serbia beat Austria 3-2 to take over as the new leaders of Group D.
Georgia's 1-1 draw in Wales - after Tornike Okriashvili cancelled out Gareth Bale's opener - put top spot up for grabs and Serbia moved into pole position as Mitrovic's goals came either side of one from Austria's Marcel Sabitzer's to make it 2-1 at half-time.
Marc Janko levelled things up again just past the hour only for Southampton's Dusan Tadic to score a 74th-minute winner.
The Republic of Ireland are level with Serbia on seven points after they beat Moldova 3-1 thanks to two goals from James McClean following Shane Long's opener, with Igor Bugaiov's strike in first-half added time the hosts' only response.
Croatia lead the way in Group I after beating Finland 1-0 thanks to Juventus striker Mario Mandzukic's solitary goal.
Iceland are second thanks to a 2-0 home win over Turkey after Omer Toprak's own goal was followed by Alfred Finbogasson's goal just before the break.
Ukraine's Artem Kravets, Andriy Yarmelenko and Ruslan Rotan scored in a comfortable 3-0 win over Kosovo to move into third place.
Israel made it two wins out of three in Group G with a 2-1 victory over minnows Liechtenstein.
Tomer Hemed scored twice inside the opening 12 minutes but defender Maximilian Goppel scored Liechtenstein's first goal of the campaign to make things difficult for the hosts in the second half.