When, in 2006, I first raised the issue of immigration and its impact on white working-class communities, I was vilified for giving the "oxygen of publicity" to the British National Party. The false accusation was that talking about immigration legitimised support for the extreme right.
When, in 2007, I raised the question of how to respond to genuine frustrations felt by those working-class communities who were most affected by migration, living in areas where jobs and housing were already too scarce, I was accused by some of "using the words of the BNP".
Few of my fellow MPs who sidled up to express their support were willing to engage in a public debate. So, I'm pleased to hear that the Prime Minister wants a wide-ranging debate, but I'm sceptical of his motives.
David Cameron's timing has more to do with politics than policy. His speech seemed more about shoring up the Tories' right flank before the local elections, just as Vince Cable's response was about shoring up the Liberal Democrats' left flank.
But, whatever the motives and however bad the timing, engage we must.
The issues surrounding immigration are hugely important to voters. If we want to reconnect people to politics, we have to talk about the things that matter to them, not the obsessions of the Westminster village.
And if we do not openly lead the difficult debate on the impact of migrants settling in existing communities, we leave the terrain open for the extreme right, letting them exploit people's fears for their own hideous racist purposes. Indeed, it is not talking about immigration that provides the extreme right with the oxygen it needs to succeed in the democratic arena.
I do, however, despair at Cameron's approach. His main response is to talk tough on numbers. Like others before him, he says he'll dramatically reduce the numbers coming into Britain. Like the others, he will fail.
Migration is a feature of globalisation. People move across national boundaries more often and more easily. Britain is an attractive destination for many in Europe and beyond, with its job opportunities, its language and its open and tolerant culture.
It is disingenuous to pretend that we can turn the clock back to the 1980s – globalisation is a reality and we have to deal with the world as it is now, not with how we might remember it.
It was Labour that got tough on people abusing the asylum system and cut the numbers seeking asylum from more than 84,000 in 2002, to just under 18,000 by 2010. But migrants found new routes. So, for instance, in 2000, 75,000 students came to study in the UK, but by 2010 that figure had become 234,000. Many were genuine students, but some were not.
Now the Tory-led coalition is promising to get tough on students, but, even if it succeeds, migrants will find new routes, as family members or visitors or whatever opens up for them.
Of course, the Government must continue to try to manage migration in a fair and efficient way, but promising to cut numbers substantially offers a false prospectus. When politicians say they will do something and fail to deliver, people lose trust. An honest conversation needs honesty about what the Government can do.
The focus must shift from a single- minded obsession with numbers to a consideration of the impact migration has on affected communities. And that means the Government must put its money where its mouth is.
Everyone agrees that it is important for immigrants to speak our language. But it's no good Cameron talking about the importance of learning English while cutting all funding for English classes for nearly all immigrants.
Above all, my conversations with constituents in Barking and Dagenham convince me that what really angers people is a perceived unfairness in the way recently arrived migrants access public resources, particularly social housing. If we can tackle that perception I believe we can puncture much of the hostility. To do that we need a system for allocating public resources which is open and transparent and that people feel is fair.
That means giving some proper weighting to the length of time people have lived in a community and the contribution they have made.
These are tough issues, especially for the left. These ideas are no substitute for building more affordable homes or redistributing more wealth. But no matter how many new affordable properties we build, we will always need to ration scarce public resources; doing that in a way that enhances community cohesion is important.
Crucially, leading a debate on these difficult issues is the best way of responding to the challenge of immigration and ensuring that we keep it out of the hands of extremists.
The writer is Labour MP for BarkingReuse content