It was bad enough when it was only Ed Lester, the head of the Student Loans Company, who was enjoying lower taxes thanks to his being paid as a private contractor rather than a government employee. The implications grew darker when the Department of Health was forced to admit that two dozen of its senior staff were also pulling the same stunt. Only now, after a much-needed Whitehall-wide review, is the full extent of the rot revealed – and it should make both the non-mandarins themselves, and those who waved through their advantageous arrangements, blush with shame.

That there are no fewer than 2,400 civil servants being paid "off payroll" – more than two-thirds of whom have been in their jobs for upwards of two years – is wrong in every conceivable way. First, there is the cost to the public purse. Although there is no reason to believe that these people are not paying all that is required from them as contractors, the exchequer may be losing as much as £30m each year thanks to their disingenuous status. Then there is the question of governance: it is hardly in the best interests of an organisation for executive staff to be freelance.

Most compelling of all, however, is the moral argument. At a time of unprecedented spending cuts, with welfare reforms biting chunks out of benefits payments and salaried government workers striking over pay and conditions, it is neither just nor politic for Whitehall to be turning a blind eye to the tax avoidance ploys of some of its most expensive staff.

Danny Alexander – the Treasury Chief Secretary who commissioned the review – is trying to regain the initiative for the Government, launching a consultation on new tax rules for "personal service companies" and pledging to tighten up off-payroll appointments before the end of the year. Not a moment too soon.

But the matter cannot end here. So far, there has been no probe into either the NHS or local government, where such arrangements may be rife. As the Chancellor has rightly noted, tax avoidance is "morally repugnant". It is up to his Government to ensure that the public sector is setting the example it should.