Mr Afolami told Talk TV Mr Johnson should also resign from No 10 and said he “can’t serve under the prime minister” after months of turmoil.
He said that after recent allegations regarding the former deputy chief whip, Christopher Pincher, and other damaging scandals, Mr Johnson did not have his support.
“I just don’t think the prime minister any longer has, not just my support, but he doesn’t have, I don’t think, the support of the party, or indeed the country any more,” he said.
“I think for that reason he should step down.”
Confirming he would be resigning, he continued: “I think you have to resign because I can’t serve under the prime minister – but I say that with regret because I think this government has done some great things.”
Sir Keir Starmer, the Labour leader, also said Mr Johnson is “unfit to government” and said he would support a general election if one were called.
Sir Keir told broadcasters: “He is unfit to be prime minister. He is not fit to govern the country. That is dawning on many people across the Conservative party, but they have to reflect on that, that they have backed him for months and months and months.
“Resigning today means nothing against their complicity for all those months when they should have seen him for what he was, they knew who he was. We need a change of government.”
Asked if he would support an election if one were called in the next few weeks, Sir Keir said: “Yes. We need a fresh start for Britain. We need a change of government.”
Join our commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies