Wild jubilation broke out among MPs when Speaker Jacob Mudenda told the House and suspended the process that had begun to impeach the president.
“I Robert Gabriel Mugabe in terms of section 96 of the constitution of Zimbabwe hereby formally tender my resignation... with immediate effect,” said Mr Mudenda, reading the letter.
The president's letter said he was resigning immediately and voluntarily in order to have a "smooth transfer of power".
The sound of car horns filled the air in Harare, as thousands poured onto the streets to celebrate.
The chief whip of the ruling ZANU-PF party, which Mr Mugabe leads, said that he was happy the president had decided to resign voluntarily.
Lovemore Matuke said that if Mr Mugabe had not taken this step, the situation would have ended in “serious embarrassment”.
The origin of Mr Mugabe's sudden downfall lies in rivalry between members of Zimbabwe's ruling elite over who will succeed him, with the army seizing power after Mr Mugabe sacked the favourite to succeed him, then-Vice President Emmerson Mnangagwa, to smooth a path to the presidency for his wife Grace.
Mr Mnangagwa, a former security chief known as "The Crocodile", is expected to take over as president, with one ZANU-PF official suggesting that will happen within 48 hours - although the speaker Mr Mudenda did not say who would be in charge until that point.
On the streets of the capital, some people held posters of Zimbabwean army chief General Constantino Chiwenga and Mr Mnangagwa.
Mr Mugabe is the only leader Zimbabwe has known since a guerrilla struggle ended white-minority rule in the former Rhodesia. Referring to Mr Mugabe and that time, Nelson Mandela once said that "he was the star and then the sun come up".
During his reign, he took the once-rich country to economic ruin and kept his grip on power through repression of opponents, although he styled himself as the "Grand Man" of African politics and kept the admiration of many people across Africa.
British Prime Minister Theresa May said Mr Mugabe's resignation gave Zimbabwe the chance for a fresh start.
“The resignation of Robert Mugabe provides Zimbabwe with an opportunity to forge a new path free of the oppression that characterised his rule,” Ms May said.
“In recent days, we have seen the desire of the Zimbabwean people for free and fair elections and the opportunity to rebuild the country's economy under a legitimate government.”
Ms May added that Britain would do all it could to support the country as “Zimbabwe's oldest friend”.
Elsewhere, Zimbabweans took to the streets of the Yeoville and Hillbrow districts of the South African city of Johannesburg to celebrate news of the resignation.
Around three million Zimbabweans have emigrated from their home country to South Africa in search of work following Zimbabwe's economic collapse.
Reuters contributed to this report
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies