Nearly 700 shareholders spent two hours debating the proposal, backed by the board, at a meeting at the Bath and West showground, a few miles from Clark's headquarters in Street, Somerset.
Some 37.8 million votes, equal to 52.2 per cent of those cast, went against Berisford, which mustered support from holders of 34.1 million shares.
Alan Bowkett, chief executive of Berisford, said: 'I am saddened that Berisford's proposals have not been accepted, despite having been recommended by a majority of their directors.
'We have therefore withdrawn from discussions with Clark. As we have made clear, we have had discussion with a number of other parties about prospective acquisitions, and attention will now be focused on these.'
A Berisford spokeman said later that the company was irritated that the Clark board had allowed the proposal to get so far without being able to deliver shareholders.
He said the other potential acquisitions on the 'front burner', would be in manufacturing and could be of a size similar to the Clark deal.
The proposed bid for Britain's best- known shoe manufacturer deepened the rift in the Clark family, members of which own 70 per cent of the shares, over the future of the private company, founded 168 years ago. The result has cast a shadow over the future of Walter Dickson, a non-family employee, who was drafted in as chairman last year. Asked after the meeting whether he was going to resign, he said: 'I have not had a chance to think about my position. I'll come to that decision once I've given it some thought over the weekend.'
He said the board 'will now work towards achieving agreement on how best to take advantage of the great strengths of its brands'.
Mr Dickson survived an attempt to remove him last autumn after shareholders voted against proposals put forward by three rebel directors.
The three directors - Lance Clark, Caroline Gould and Richard Clark - yesterday openly rejected the bid at the meeting. Roger Pedder, another director, also dissented.
Strong support for the rebels' stance against Berisford was separately orchestrated by a group called Share Holders Against Enforced Sale (Shoes).
Hugh Pym, leader of Shoes, said after the meeting that he had no ambitions to join the board.
'Our victory shows that there is a widespread belief that the family has not interfered with the running of the company,' he said.
During the meeting there were several impassioned pleas from shareholders to keep the company independent. One said: 'It seems a shame that we have come to this. What exactly does Berisford have to offer us?
'All I can see is their claim that they have a united board. I think that all it needs is for you people (directors) to get together.'
Lance Clark argued: 'We are emerging from the worst recession since the Thirties and this company has still got a strong balance sheet.
'This is an idiotic time to sell - we do have the structure which will provide leadership.'
Mr Dickson argued strongly for shareholders to accept the bid and reject alternative proposals put forward by Shoes. These included cutting family representation on the board to two directors and forming a shareholder council. Mr Dickson said those proposals were 'excellent in principle, but do not offer a certain alternative'.