Robin Duval, a senior regulator at the Independent Television Commission, will take over in January from Mr Ferman who has been in the job for 23 years.
Mr Duval has been at the ITC for 13 years and is fully prepared for the brickbats that will come his way, said Andreas Whittam Smith, chairman of the board "There are two armed camps, or lobbying groups. There are the pro-censorship groups and the libertarians, so there is no way of pleasing both of them all. You are always under attack from both sides."
Currently Independent Television Commission deputy director of programmes, Mr Duval has become an expert in screen media standards. For 13 years he has monitored on-screen taste and decency as well as the quality and diversity of programmes on ITV, Channel 4, Channel 5, cable and satellite.
He has also worked for BBC Radio, made television commercials and worked for the Central Office of Information making films. He also worked at the Home Office, forming policy on the rights and privileges of prisoners.
"We could not have found a better all-rounder for the post," Mr Ferman said. "Robin is an experienced film-maker, a sensitive media regulator and a man steeped in the best traditions of public service."
Mr Ferman has already advised his successor to get a "flak jacket". During his tenure Mr Ferman was targeted by the self-appointed protectors of Middle England's morals for his supposed liberalism.
Passing films such as David Cronenberg's Crash, Lolita, Kids and The Last Temptation of Christ had right-wing newspapers frothing at the mouth and calling for his head. Most regularly described him as "American-born" and implied that he was out of touch with the morals of the UK. Critics were especially angered by his calls for European-wide pornography regulations.
However under Mr Ferman's tenure the BBFC moved strongly on violence - particularly of a sexual nature, and moved to regulate videos.
Mr Duval said: "I am honoured to be asked to succeed James Ferman, whose authority and expertise are recognised in Britain and internationally. He will be a tough act to follow."
One television industry source said Mr Duval was well-equipped to handle the new role amid the glare of the media. "He's a man with strong views and considerable artistic interests. I think he's going into this job with his eyes open, well aware of the high-profile nature and very much able to take it on the chin," said the source.
Since Mr Whittam Smith was appointed chairman he has been trying to take the board to meet the people and to reach over the heads of what is usually a negative press. It has conducted road shows to explain its work to the public and Mr Duval will be expected to continue the new strategy of openness.
Mr Duval, with Mr Whittam Smith, will have to take on the board to a period when it will be able to ban less and will instead be looking to promote warnings on material. The use of the Internet and the expansion in the electronic imagery market will make it more difficult for the censor to control what people see. Instead in the long term the board will be more concerned with keeping unsuitable material out of sensitive hands.