Mr Brown highlighted a recent poll by the campaign group Hope Not Hate which suggested that around 21 per cent of Labour voters who backed Brexit at the 2016 referendum had shifted their views.
The former prime minister claimed those who voted Remain are “despondent” at the current trajectory of Britain’s withdrawal from the bloc while Leave voters believe none of the options can deliver “what they were promised” and “now feel betrayed”.
His remarks come as the The Independent’s campaign – demanding a Final Say referendum on the Brexit deal for the British people – passed 614,000 signatures after launching just three weeks ago.
A fresh poll this week by BMG for The Independent also found that 48 per cent of the British public would back a fresh vote on any deal struck by Theresa May with negotiators in Brussels – up four percentage points from just four weeks ago.
Opposition has also dropped by three points, with only 24 per cent now against a new public vote on Brexit, 16 per cent not having strong feelings either way and 11 per cent saying they did not know.
Addressing an audience at the Edinburgh International Book Festival, Mr Brown said: “Leavers and remainers do have one thing in common – both are increasingly losing hope for a better future for Britain”.
He added: “Remain voters are despondent, fearing that we have moved from a soft Brexit to a hard Brexit to a no-deal Brexit.
“But leave voters believe that none of the Brexit options – a Norway-style deal, a Swiss-style deal or the Canadian option – can deliver what they were promised and now feel betrayed.”
The former prime minister’s intervention also came as he urged Jeremy Corbyn to urgently address the issue of antisemitism in Labour’s ranks and said the current row cannot “keep on going as a running sore”.
“Jeremy Corbyn has got to change,” Mr Brown added. ”He cannot sustain particularly what he is saying about the international agreement on what we do in our attitudes to both the Holocaust and to Israel.
“I predict to you that's going to change within a few weeks. I believe that it will change but even that will not be enough.
“You have got to show by your actions not simply by saying some words that you understand the deep hurt that has been caused. I'm very clear about that.”
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies