Muslim prison gangs are trying to force other inmates to sign up to Islamic radicalism, prison officers said.
Extremists at high security HMP Whitemoor in Cambridgeshire were pushing a "strict and extreme" interpretation of Islamic practice, inspectors were told.
Chief Inspector of Prisons Anne Owers said: "There was a perception among officers that some Muslim prisoners operated as a gang and put pressures on non-Muslim prisoners to convert, and on other Muslim prisoners to conform to a strict and extreme interpretation of Islamic practice.
She found staff were reluctant to tackle "inappropriate behaviour" and was told Muslim prisoners were able to "police themselves".
She was told: "The new gang are the Muslims. The Muslim group is a big group and others are looking for protection.
"Those who are isolated are looking for protection and so are the ones converting as they won't get help from screws."
Earlier this week an EU-commissioned report warned urgent action was required to stop brainwashing by jailed extremists.
Dr Peter Neumann from King's College London said governments across Europe should observe prisons more closely in the future as they were likely to become "major hubs" for terrorist recruitment.
He suggested creating "jihadist prisons" in which to isolate Islamist militants.
Phil Wheatley, Director General of the National Offender Management Service, said: "The Chief Inspector is right to highlight the challenges and risks Whitemoor is facing. It is also important to recognise the action being taken to manage challenging prisoner profiles.
"A more sophisticated approach to addressing bullying and the management of bullies and their victims is now in place and is bringing improvements.
"Work to improve the relationships between staff and prisoners is a priority and measures have been implemented to tackle this, including training to develop staff understanding of the growing Muslim population."