The former vice president flew into Harare at around 4pm and was driven to the State House where he was due to be briefed on developments, according to an official with the ruling ZANU-PF party.
Mr Mnangagwa had fled the country two weeks after being fired by Mr Mugabe. His ousting led to the military stepping in and the of triggering popular protests against the veteran leader.
Mr Mugabe, who had ruled Zimbabwe since it secured independence in 1980, appeared to believe he could salvage his political career in the middle of the chaos. But amid mounting demands for him to step aside, he resigned on Tuesday.
Footage broadcast by eNCA television showed South African President Jacob Zuma hosting Mnangagwa in Pretoria earlier on Wednesday for what was termed a "courtesy call".
The 75-year-old is due to be sworn in as Zimbabwe’s new president on Friday morning, the speaker of parliament said, after ZANU-PF notified him of its nomination of Mr Mnangagwa to replace Mr Mugabe until the end of the current term next year.
The Associated Press said the privately run Newsday newspaper reported Mr Mnangagwa had been due to be met on arrival by army commander Constantino Chiwenga and ruling party officials. He was also due to meet with Mr Mugabe “for a briefing”.
The news agency said several hundred people gathered ahead of his return to the country. One man in the crowd, Godwin Nyarugwa, said he was “very ecstatic” and that “we need change in this country, change in everything” after years of economic crisis.
Mr Nyarugwa said he had several university degrees but no job, a common theme among Zimbabwe's well-educated population.
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies