But Lord Lester QC, who led a seven-strong committee of inquiry, denied his report was a whitewash and accused Labour of using tactics which were just as bad.
The three to be expelled are Jeremy Shaw, a councillor, Jonathan Mathews and Tom Winnifrith, who were involved in issuing leaflets in the East End of London.
Last night, Scotland Yard confirmed that the Attorney General, Sir Nicholas Lyell, had asked the Metropolitan Police to examine whether one of the leaflets was an incitement to racial hatred.
Jack Straw, the shadow Cabinet minister whose allegations led to an inquiry, accused the Liberal Democrats of letting others off the hook.
The inquiry was ordered by Paddy Ashdown, the Liberal Democrat leader, after the British National Party won a council by-election in the Millwall area of Tower Hamlets, east London.
The victory shocked politicians and caused fears about the rise of the extreme right wing in Britain.
Belle Harris, the Labour councillor who lost his seat in the by-election, yesterday accused Liberal Democrat leaders of a damage limitation exercise.
Mr Harris urged the Commission for Racial Equality to put pressure on the Director of Public Prosecutions to take action under the Race Relations Act against Liberal Democrats who had issued racist leaflets.
The inquiry report criticised the party leadership, saying it found 'ample evidence' that the party at federal, national and regional levels had been aware of problems in Tower Hamlets since 1990.
'It is also clear that despite a number of suggestions to this end, no effective concerted action has been taken at any time to ensure these problems were fully and effectively resolved,' it said.
The use of Liberal Democrat 'focus' leaflets had involved 'exploitation of the local prejudice against immigrants in the allocation of housing in order to win votes'.
It condemned the conduct of Mr Shaw, Mr Mathews and Mr Winnifrith, but the committee said it was unfortunate that action was not taken after Mr Harris had taken seven Liberals to court for issuing a fake Labour leaflet in the May 1990 elections. It alleged that if elected Labour would rehouse homeless Bangladeshis in Tower Hamlets.
The report found that Liberal campaign statements such as 'Liberals fight for local people' or 'Liberals fight for those who survived the Blitz' were construed as racist.
It criticised three other councillors for failing to co-operate with the inquiry: Betty Wright, John Snooks, and Jim Smith.
It also criticised the lack of a system by the regional or federal party to guide the Liberal Democrats in Tower Hamlets, ensuring that their campaigning literature 'did not pander to racist sentiment'.
Mr Winnifrith accused the Liberal Democrat inquiry team of using him as a scapegoat, a charge denied by Lord Lester.
'I played a part in two leaflets. There were many senior figures in Tower Hamlets Liberals who played a part,' he said.
'It's quite clear that the small guy is dispensable. That is me. I am being scapegoated because I am dispensable and they are not,' he said in a BBC Radio interview.
Exposing the bitterness against Lord Lester's team, he accused those who wrote the report of 'living in the shires and leafy suburbs sipping claret'.
'We won this borough without help from the national party. They've been a nightmare ever since we got involved with them. It's up to each individual person and their conscience whether they can accept this pile of shit that Lord Lester had produced.'
One Liberal Democrat councillor in the borough, Eric Flounders, last night said he was one of five who had resigned from the party as a result of the inquiry. 'I have to say I disagree with the basis on which the inquiry took place,' he said. There was nothing racist about the leaflets: 'The inquiry was set up as a result of hysterical over-reaction by Paddy Ashdown.'