Sir Craig Oliver said Mr Cameron would have been hit by a wave of Conservative defections to Ukip if he had not promised to give the country the opportunity of an in/out vote.
In an interview for the BBC Radio 4 series The Cameron Years, Sir Craig said Eurosceptic backbenchers had been “lining up” to become “kamikaze pilots” and bring him down.
Following months of speculation, the then prime minister announced in January 2013 that there would be a referendum if the Conservatives were returned to power in the general election in 2015.
Sir Craig said Mr Cameron and those around him took the decision in part because the issue had become a “massive boulder in the road” for the Government, standing in the way of everything they wanted to do.
“The second reason I think that they wanted to do it was that knowledge that actually they would probably have to go as leadership of the party if they didn't do it,” he said.
“Now you could say, actually you could take a principled decision and say I'm not doing it. It's not the right thing for the country or the party.
“But their view was, if they gave up, then somebody would be elected who would - and probably not do it in the way that they thought was best for the party in the country.”
“There was absolutely no doubt that there were people who were lining up to do it and be kamikaze pilots to make sure that this thing happened,” he said.
“There weren't specific numbers but I think that there was a sense that you could end up with, you know, another six to 10 that were just going.”
Mr Cameron's policy chief Sir Oliver Letwin described how, after the Conservatives were re-elected, he was asked by the prime minister to try to persuade Boris Johnson to back Remain.
“It was more than one phone call, it was a large number of meetings,” he told the programme.
“It was part of a continuing conversation. And we didn't quite get there.”
Sir Oliver said he believed the result “might well have gone the other way” if Mr Johnson had not decided to put himself at the head of the Leave campaign.
“It's impossible in history to write the counter-factual. You don't know what would have happened if Boris hadn't been part of the campaign.
“But my feel, my gut instinct, is that he made a very considerable difference.”
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies