The Duke, patron of the Charities Aid Foundation, called for charitable status to be withdrawn from from whole sections of the voluntary sector to slash red tape and simplify rules.
Giving the annual Arnold Goodman Charity Lecture in London he said it had become accepted that almost any organisation which could attract voluntary support and money should become a charity.
'Some are undoubtedly fulfilling the original humanitarian purpose of helping the relatively poor, deprived or disadvantaged to enjoy a better standard of living,' he said.
'Most of the others are providing non-vital services of all kinds to people who are unable, or do not wish, to pay for them themselves.' These ranged from arts, educational and sports bodies to museums, stately homes and trusts to run old steam engines.
The Duke, no stranger to controversy, said: 'The fact is that the combination of the welfare state and the relatively high personal and corporate taxation that is needed to pay for it has removed the need for charity in its traditional sense. In this country at least, poverty is no longer absolute.'