Not many dawns have passed since the sparky Tory chairwoman Sayeeda Warsi spoke up about the "dinner table" libelling of all Muslims, now routine, normalised, unremarkable, intimate, uncontested. I see and hear it, too – prejudices passed around with the balsamic vinegar or ketchup. Some Muslims deserve castigation and worse for the terrible things they do. I frequently denounce them in my columns. But sweeping, indiscriminate execration of any collective is abhorrent and must be confronted. Warsi did that, knowing her words would infuriate right-wing Tories who can't stand the brown little upstart.

Now, how will she react to her leader, who has amplified the small talk of bigotry and boomed it through a megaphone, perhaps to slap her down? I found Cameron's speech in Munich indefensible even though I completely agree with some observations and policy ideas. We discussed these two years back when we met in his office for over an hour. Self-exclusion, special pleading, women's rights, community oppression, anti-democratic attitudes, terrorism, the spread of Wahhabi Islam are serious problems and growing. Laissez-faire multicultural policies do not serve our times. State institutions should fund shared spaces, crossover ideas, openness and modernity. Many of us Muslims would be with David Cameron if his speech hadn't shown him to be selective, hypocritical, calculating, woefully indifferent to Muslim victims of relentless racism and chauvinism. He was speaking the words of white extremists but in posh. There was so much that was objectionable – where he spoke, what he said, the timing, the purposes loitering behind the fine façade of his personality.

By speaking out in Munich he allied himself with the ghastly Angela Merkel who delivered a similarly provocative sermon last autumn. Racism is rife in both countries; in both nations, millions of their own natives rigidly hold on to their languages and cultures. Think of those Germans who go abroad on holiday and stay in walled-off camps where only German is spoken. Countless Britons are similarly against integration with the people of unfamiliar countries they visit or migrate to. I would rather have my tongue chopped off than lose my mellifluous home languages. To learn and love English shouldn't mean the destruction of world languages, most of which we are lucky enough to have on our isles.

Remember the PM was at an international security council when he let rip – an outrage. Diversity is one of our greatest assets, an antidote to militancy not its cause. A new study by the Runnymede Trust in Birmingham shows young citizens are more bonded and at ease with difference than their elders in that multifarious city. As the speech progressed, you realised that Cameron's problem isn't cultural difference. It's the people whom marauding Christian Crusaders called "curs", wretched Mohammedans. Cameron isn't troubled by Hassidic enclaves, Orthodox Jewish dress codes, or their religiously sanctioned gender inequality and stubborn self-removal from mainstream societies. I have been rebuffed by a veiled Muslim woman and a Hassidic Jewish one when I tried to talk to their children. And the other day a young white mum told her daughter to come away from me, the "Paki". Moreover, those who only want to live with their own in white heartlands are thought to be no threat to integration – they are only doing what comes naturally. Little official concern is expressed about crimes committed by various non-Muslim ethnic groups against each other, against Muslims or white Britons. Even more disgracefully, Tories ignore racists who terrorise people of colour. How unfair is that?

Our PM, in effect, identified himself with the abominable English Defence League when he spoke up a day before the league marched through Luton shouting abuse. Are these the laudable British values we must embrace? Hot-headed Muslims will be even more convinced they are not wanted in the land of their birth.

The next charge: hypocrisy. This Government is enthusiastically funding schools for separatists – from snooty white middle-classes, to pedantic, purist Hindus, nutty, evangelical Christians, and introverted, uncompromising Muslims. How does that foster integration? Michael Gove has just been accused by Bradford City Council of encouraging segregation by funding a new free school started up by Ayub Ismail, who wants to ensure his pupils are not "absorbed into the dominant culture". Saudis are allowed by our Government to brainwash Muslims who are then despised. The Tory party's right and left buttocks move in different directions. Not clever nor consistent with the PM's Big Message of the week.

So why is he doing it? When politicians are in trouble they pick on "outsiders", put them into stocks so the people can turn on them and relieve their feelings of frustration. Andrew Lansley, now in charge of health, said shamelessly in 1995 that they were using the anti-immigration card because it played well with voters. Recently he blamed migrants for a rise in TB in Britain, a link that used healthcare concerns to whip up xenophobic panic. Cameron himself designed the disgraceful anti-refugee campaign for Michael Howard in 2005.

The German Marshall Fund has just published a comparative survey of attitudes to migrants in Western countries. Britons, noted The Economist, are shown up as a "mean-minded lot" – negative, hostile, paranoid. I don't believe that is the full picture. Britain is also uniquely receptive – which is why so many of us would not live elsewhere. But it is going through a seriously bad mood and Cameron is exploiting that.

I accept our citizens are unnerved by those British Muslims who make endless demands, are full of wrath and murderous plans, or choose ghettoisation. However, the widespread national unhappiness is created by policies pushed through by this Government. Muslims and migrants are being used to distract people from the planned chaos implemented by this unpopular coalition. It is politicking of the worst kind. Which is why it must be opposed vehemently. As the daughter of a survivor said to me at the Holocaust Memorial Day in January: "We Jews must look to our failings and crimes. But when outsiders try to use that for their devilish reasons, we know where we must stand." Me too.

y.alibhai-brown@independent.co.uk