James Palumbo, 30, and his sister, Annabella Adams, 32, accuse their father of wrongly benefiting from a family trust that he jointly manages. The trust was created by their grandfather, Rudolf, an astute property investor who made a fortune developing in the City just after the Second World War.
A writ lodged in the High Court last month brings to public attention what is said to have been a family row lasting several years. The writ contends that Lord Palumbo and other trustees of the estate have been taking directors' fees - which its rules expressly forbid them doing - as well as other benefits. His children want all monies to be returned, with interest, to the fund.
It also calls for the trustees to itemise exactly how much they have taken and for the High Court to order the replacement of all trustees with 'two or more fit and proper persons'.
Lord Palumbo, his Lebanese-born wife, Hayat, accountant Thomas Tharby and solicitor John Underwood would be liable to damages for 'breach of trust' if shown to have taken benefits from the fund.
James, who owns a London nightclub, the Ministry of Sound, was not available for comment yesterday. Annabella, who is married and lives rent-free in a cottage on her father's estate, said: 'This is a very personal family matter and one which I don't wish to comment on. '
It is understood that relations in the family have been strained since Lord Palumbo married Hayat Calil following the death of their mother, Denia Wigram, in 1986, particularly since he started a second family.
The writ comes at an embarrassing moment for Lord Palumbo, who stepped down as chairman of the Arts Council earlier this year. He is about to embark on the development of a controversial property in the City, partly funded by money from the family trust. Work on the building - Number One Poultry - designed by the late Sir James Stirling, is scheduled to start next month
It is understood that James and Annabella became concerned about changes made two years ago to the way in which the trust, set up in 1955, was managed.
In its annual survey of the the 500 richest people in Britain, the Sunday Times yesterday ranked Lord Palumbo equal 148th with a fortune of pounds 80m, a fall of pounds 50m from the previous year. This reflected the declining net asset value of the Rugarth Investment Trust, which is the management vehicle of the family fortune.
He has previously insisted that when Number One Poultry is completed in three years' time, it will be worth pounds 150m.
Lord Palumbo, who has been a lifelong supporter of the Conservative Party but was critical of government policy towards the arts, was not available for comment yesterday.