A senior minister says Whitehall managers must step up efforts to identify and dismiss poorly-performing civil servants.
Cabinet Office Minister Francis Maude said reforms would provide incentives for them to "take tough decisions and have difficult conversations" with staff.
Detailed proposals are expected to be published within the next few months.
The coalition agreement pledged action to "make it easier to reward the best civil servants and remove the least effective".
And Mr Maude told The Daily Telegraph he would act to ensure private-sector-style ranking of workers by ability was imposed across the service.
He said sacking someone should not be "easy" at any time - but rejected claims that it was any harder to get rid of civil servants than private employees.
"It is just that the recent history has been that performance management hasn't been very good," he said.
"What we will need to end up with is the way performance management is done in most organisations, which is you force managers to do rankings, to rate people in order of performance.
"Otherwise the temptation always is for everybody to be 'above average'.
"This will take time. There has been no incentive for managers to take tough decisions and have difficult conversations.
"That can change and it is one of the issues we will be addressing in our civil service reform plans."
A spokesman for the Public and Commercial Service union said: "When 200,000 of Maude's staff were on strike yesterday in a dispute over pensions entirely of his making, he was nowhere to be seen.
"People will see right through this desperate attempt to grab a headline, by the minister who brought us fuel panic and jerry cans in garages."
Dave Penman, deputy general secretary of the FDA, which represents senior civil servants, said: "We have yet to see details of the proposed white paper on civil service reform.
"The civil service already has rigorous performance management processes, including for the senior civil service a forced ranking approach to appraisal and a revision of these procedures, which was introduced only last month.
"As Francis Maude has said, it is simply a myth that civil servants can't be sacked or that it is more difficult to sack individuals who underperform in the civil service than it is anywhere else.
"We will, of course, expect full consultation if there are any proposals to change performance management for civil servants and have assurances from the Government that there will be full consultation on the white paper once published."
The Prospect union condemned the suggestion that performance management could be used to cull the civil service and said the Government should instead focus on how the service is coping with the loss of 80,000 jobs.
Deputy general secretary Dai Hudd said the idea of identifying and firing civil servants by ranking all government officials was wrong: "This would be a cruel policy and legally challengeable.
"It would set colleague against colleague and destroy collaborative working - worsening morale rather than improving performance, and creating a culture of back-stabbing, uncertainty and fear. It would increase stress and adds insult to the injury of job cuts, pay freezes and attacks on pensions.
"Even Francis Maude distances himself from this rhetoric by admitting that effective procedures already exist in the civil service for dismissal on grounds of performance.
"Blaming hard-working public servants and making ill-thought-through comments will not in any way contribute to making the civil service a more efficient and effective organisation. Comments like this are just another way of trying to push the blame for the Government's own errors - typified by yesterday's U-turn over the F35 jet fighter - on to individuals."