For Mr Latif that awful morning was a watershed moment. Firstly he had to gently inform six-year-old Ruqayah that there would be no lessons for the next few weeks because her school, which was on the top floor of the mosque, had been damaged in the fire. Then he had to explain to her why someone would even want to set fire to a mosque in the first place.
"I had always tried to shield my daughter from the idea that there are people out there who simply do not like Muslims," the 39-year-old said. "But it wasn't long before she worked it out. That day installed in her a "them and us" mentality – something I hoped she would never have."
The anger that Mr Latif felt following that fire on 4 May could have been directed solely at the bigots who set his mosque alight. But the people he was most furious with were a motley collection of 15 to 20 young men who regularly preached a radical and intolerant brand of Islam from a street stall down the road and had helped foster the image that Luton was an Islamist stronghold.
Two weeks earlier those same men – most of whom are former members of the banned Islamist group Al Muhajiroun – had greeted soldiers of the Royal Anglian regiment who were returning from Iraq with screams of abuse and placards declaring them "Butchers of Basra", "murderers" and "baby-killers".
The protest outraged whole swaths of Britain, not least Luton's 25,000 Muslims who knew all too well that their town would once again be associated with extremism.
Once the Masjid Al Ghurabaa was firebombed, in what police suspect was a retaliatory hate attack, Mr Latif sadly concluded that Luton's ordinary Muslims were paying the price for the actions of the "Al Muhajiroun boys". Which is why he decided to act against them. Shortly after Friday prayers last week he and 300 supporters marched down to Dunstable Road where the sect often set up their stall and told them in no uncertain terms that they were no longer welcome in Luton. His group was spontaneously joined by a number of shopkeepers who were equally fed up with the Islamists putting off their customers with their firebrand preaching.
Despite retaliating with angry taunts of "shame on you" and "go back to your synagogue", the extremists eventually backed down and sloped off towards the town centre. They have vowed to return but many within Luton's Muslim community say they will now get the same treatment if they ever come back.
For outside observers the fact that Luton's Muslims turned on the extremists in their midst shows how angry many British Muslims are at the small number of Islamists who routinely give them a bad name. But within Luton itself, it wasn't the liberal Muslims of the community that turned on the extremists – it was the ultra-orthodox conservatives that finally snapped.
Mr Latif's Masjid Al Ghurabaa follows the Salafi school of thought, the socially conservative Saudi sect in which male adherents tend to grow long beads and dress in simple tunics and women usually adopt the full-length niqab veil. "To outsiders we come across as very traditional," he says. "We don't listen to music – my wife and I, for instance, wouldn't go to a wedding if there was music playing. But that doesn't make us extremists. Islam teaches people to strongly believe in social cohesion and strictly prohibits shedding any innocent blood. The hot-headed young men that belong to Al Muhajiroun promote violence and preach a false version of Islam that reflects badly on ordinary Muslims. That's why we took action."
The spontaneous protest against Luton's firebrand street preachers has prompted a genuine outpouring of support amongst the wider Muslim community. This Saturday local business leaders are holding a meeting to try to persuade the police and local authorities to help them crack down on former Al Muhajiroun members.
Nadeem Chaudhary, a 38-year-old businessman who runs a shopping plaza at the northern end of Dunstable Road, says the Muslims of Luton have finally had enough of the extremists but they need support from police and local authorities to keep them out.
"We've been trying for a long time to persuade the police to move the Al Muhajiroun boys on but either there doesn't seem to be much they can do or they simply don't want to do anything," he said. "Despite what people say, Luton's Muslims are proud to live in a mixed, multicultural society and they've had enough of the extremists."
Spend five minutes on Dunstable Road, the bustling commercial heart of Luton's Asian community, and it is easy to see what Mr Chaudhary means. Although the street is overwhelmingly Asian and Muslim it is by no means homogenous. Women in full niqabs, old men thumbing prayer beads, glamorous girls in high heels, muscly-armed rude boys and young pious men in cotton tunics and downy beards all intermingle in the marketplace. The opinions and beliefs of Dunstable Road are just as varied as any other high street.
But the recent actions of Al Muhajiroun have led to increasing racial tensions within the town and a polarisation of opinion among the area's white and Asian communities.
In response to the protest against the Royal Anglians a group calling itself the United People of Luton have held a number of counter demonstrations against the "scum" that ruined the soldiers' homecoming. When they held a protest last month a mob of skinheads broke away from the police cordon and started attacking Asians. Another UPL march is planned for the August bank holiday and many young Muslims in the Bury Park area say they will do whatever is necessary if the police fail to protect them.
"Everybody is getting ready for a fight," said one 19-year-old man, who strongly disapproved of Al Muhajiroun but was concerned that Luton's Muslims were being targeted by anti-Muslim mobs. "We're fed up with girls in veils getting spat at and having racist abuse hurled at us. If they come here again, we'll fight."
Last night Wayne King, an organiser at the UPL, said he would consider calling off the bank holiday march now that the local Muslim community were confronting Al Muhajiroun.
Mr Latif, meanwhile, hopes that their decision to turn on the extremists within their own community will now prompt Luton's white community to do the same.
"I believe people on all sides are sick of the extremists," he said. "I now hope the white working class will weed out the fascists and hate mongers just like we now have. Otherwise things will only get worse."
Al Muhajiroun: The enemy within
Formed by the now exiled extremist preacher Omar Bakri Mohammed in the mid-1990s, Al Muhajiroun was a radical political Islamist organisation that aimed to replace democracy in Britain with sharia law. The sect officially disbanded in 2004 but activists continue to attract followers – predominantly young, unemployed men. Numbers for the group and its two banned offshoots – Al Ghurabaa and the Saviour Sect, proscribed for glorifying terrorism – rarely exceeded a couple of hundred followers but their visible self-promotion and controversial statements, such as praising the 9/11 suicide bombers, have earned them notoriety and influence. Bakri Mohammed still preaches from Lebanon on the internet. London preacher Anjem Choudhry says the group will keep its stall in Luton: "The brothers have been propagating Islam in Luton for 15 years. There is a huge amount of love and respect for them."
Comments