David Rank, who has served in the foreign service since 1990, assumed his position in January 2016 and was set to stay until Terry Branstad arrived later this month.
But it was said he "could not deliver" the news that Mr Trump had decided to withdraw the US from the accord, which has been signed by 195 countries and ratified by 147.
The US leader removed the US from the agreement which nearly 200 countries signed in an attempt to limit global warming and reduce greenhouse gas emissions late last week.
Anna Richey-Allen, a spokeswoman for the State Department's East Asia Bureau, said that Mr Rank "made a personal decision" and did not confirm that it was due to the Paris Agreement withdrawal.
He worked in the Beijing embassy when China signed a bliateral deal with the US under the Obama administration just ahead of the Paris signing.
It was considered a historic deal between two of the world's largest polluters.
China estimated they would need until 2030 to reach a "peak" of carbon emissions – mostly through inexpensive coal burning – in order to build up their economy to a point where they can afford to start transitioning to more renewable sources of energy.
After that time, the country will begin to reduce carbon emissions and increase the proportion of renewable sources of energy. They are already a leader in solar panel production and renewable energy investments throughout the world.
Chinese President Xi Jinping had urged Mr Trump to keep the US in the Paris accord, but said that his country and France must "protect" the deal.
Jon Pomfret a California-based journalist who has written extensively on China took to Twitter to say that Mr Rank held a meeting with his staff because Mr Rank "could not deliver" the statement that the US had withdrawn from the Paris accord.
Jonathan Fritz, the embassy's economics councillor, would serve as chargé in his place, Ms Richey-Allen said.
Mr Branstad, the former governor of Iowa, was confirmed by the Senate for the Ambassadorship in late May 2017. He is expected to take up the role later this year.
On the day Mr Trump decided to withdraw the US, Richard Gowan, a UN expert with the European Council on Foreign Relations, told The Independent the he “may not care, but he has just created an astonishing opportunity for the Chinese to assert themselves in the UN system”.Reuse content