The 64-year-old had held the post since 2006.
“A prominent Russian diplomat has passed away while at work. We'd like to express our sincere condolences to Vitaly Churkin’s family,” the Russian Ministry of Foreign Affairs said in a statement on its official website.
The New York Post reported that Mr Churkin became ill outside the Russian Embassy on East 67th Street at around 9.30am after suffering heart problems.
He was reportedly rushed to New York Presbyterian Hospital, where he later died.
The announcement his death was met with shock when it was delivered during a session at the UN headquarters.
“He was a dear colleague of all os us, a deeply committed diplomat of his country and one of the finest people we have known,” a UN official said.
He is survived by a wife and two adult children. His daughter is believed to be a journalist for the Vladimir Putin-backed Russia Today network.
Mr Churkin was a strident defender of Russian foreign policy, particularly the intensive bombing of Aleppo last year in order to crush rebels opposed to Syrian President Bashar-al-Assad.
Russia's Deputy Permanent Representative to the United Nations Yevgeniy Zagaynov said that Mr Churkin, who was fluent in English, French, Russian and Mongolian, kept working “till the very end”.
Before he was appointed as the country's ambassador at the UN in May 2006, the diplomat served as ambassador to Belgium and Canada, and liaison ambassador to Nato and the Western European Union (WEU).
In the 2000s he was ambassador at large at Russia's foreign ministry, while in the early 1990s he served as the special representative of the Russian President during talks on the former Yugoslavia.
The UK's ambassador to the UN, Matthew Rycroft, said on Twitter: “Absolutely devastated to hear that my friend & colleague Vitaly Churkin has died. A diplomatic giant & wonderful character. RIP.”
Alexander Yakovenko, the Russian ambassador to the UK, also tweeted his of his sadness: "Shocked and saddened to hear that my friend and colleague Vitaly Churkin has died. A diplomatic giant and a wonderful character".
Mr Churkin gained a degree of notoriety in 1986, when as a relatively young diplomat, he testified on the Chernobyl nuclear power station accident, making him the first Soviet official to appear before a Congressional Committee of the US House of Representatives.
Former US ambassador to the UN Samantha Power said that Mr Churkin had been a force for good in US-Russian relations.
She tweeted: “Devastated by passing of Russian UN Amb Vitaly Churkin. Diplomatic maestro &deeply caring man who did all he could to bridge US-RUS differences.”
The Russian Ambassador to Turkey, AndreI Karlov, was assassinated just before Christmas last year by a police officer who shouted “Don't forget Aleppo” before gunning him down.
Another less senior Russian diplomat, Sergei Krivov, was reportedly found lying on the ground by the Russian Consulate in New York on the US election day last November having died in mysterious circumstances.
BuzzFeed reported that US Consulate officials at first claimed he had fallen to his death, before saying he had suffered a heart attack. The medical examiner found that Mr Krivov died of “natural causes”.
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies