Publish or be damned by the academic police

How far should universities' funding be dependent on the level of research that departments undertake? And just how effective is the current system of assessment?

A monster, they call it. A sterile intellectual exercise that rewards the few, fuels too much theoretical research, downgrades teaching, creates a football-style transfer market for star professors and encourages universities to play games or, worse, to fire staff.

Moreover, it induces academics to behave "with the frenzied energy of battery chickens on overtime", churning out papers of dubious quality, according to one critic. Welcome to the Research Assessment Exercise (RAE), the most derided accountability mechanism in higher education.

But is the RAE as bad as the critics make out? And will it be abolished after the next exercise in 2001, as some are suggesting?

Born in 1986 to channel £850m of Government money a year to the best researchers, the RAE is feared because it can make or break departments - and the reputation of their staff. Organised every five years, it has introduced league tables into higher education. Universities are ranked against one another for the quality of research - the extent to which they push out the frontiers of knowledge - and money flows or dries up accordingly.

It surprised no one that Oxford and Cambridge were top of the last RAE in 1996, followed by the London School of Economics (LSE), Imperial College and University College London (UCL). At the bottom of the league were Luton, Teesside and Wolverhampton.

Everyone wants to work in a tip-top, 5* department - that indicates that most of your work is internationally excellent. To score the top grade 5 is also good: your department is internationally excellent in some areas. A grade 4 means you can hold your head up high because you are getting there - your department is nationally excellent in virtually all areas. But a 3a is not really cutting the mustard, certainly for an old university, though you will receive some funding for it. And a 3b and below brings you no money.

So, when academics learnt that the Higher Education Funding Council (Hefce), the body that runs the RAE, was subjecting research policy and funding to a fundamental review, they were jubilant, especially as the council was looking at whether the hated RAE should continue at all after 2001, the date of the next exercise.

Many academics hope it won't. Although their research is assessed by subject panels of peers, they believe the RAE forces them to spend ridiculous amounts of time spewing forth short papers. There is a widespread belief that four articles in peer-reviewed journals are needed to pass muster in the RAE. That is true in the sciences, but it's not true in arts subjects.

"I think the research community feels the RAE distorts the nature of our work, particularly in the humanities," says Judy Simons, dean of humanities and social sciences at De Montfort University. "People are dissuaded from undertaking long-term projects. They are churning out papers that don't demand the depth that traditionally underpins research in English."

Moreover, the RAE has led to undesirable competition between universities that militates against collaboration and the sharing of ideas, say the critics. It has led to the playing of games such as making poor researchers vanish from a department's books. Queen's University, Belfast, has gone one step further and begun to get rid of staff who are not actively engaged in research. In addition, the RAE is criticised for favouring theoretical over applied research.

Dr Nicholas Barr, senior lecturer in economics at the LSE, explains: "I publish work in academic journals, I write books and I write newspaper articles. I also submitted evidence to Lord Dearing's inquiry into higher education. But in the last RAE it was made clear to me that it was only the academic journal articles that mattered." His concern - and the concern of many other academics - is that this makes the RAE too narrow because it ignores other work that breaks new ground and can be influential.

The RAE is further criticised for discouraging interdisciplinary research, because departments have to enter their work under certain headings. That is unfortunate, according to its detractors, given that much cutting-edge research nowadays occurs at the intersection of disciplines.

Experts such as Roger Brown, director of Southampton Institute, dislike the fact that the RAE emphasises research over teaching. And Paul Cottrell, assistant general secretary of the Association of University Teachers (AUT), says: "If you are not research-active but mainly concerned with teaching, you're a second-class citizen.

"After the last RAE we had some strong research universities putting staff they deemed not to be research-active on different contracts. That meant that they lost status."

Finally, say the critics, the RAE pushes universities into a football-style transfer market by creating serious monetary incentives to hire high-performing academics. Big-name professors are lured away to boost a department in another university. If they are scientists, they may take their whole research team with them, leaving a big hole in the department they deserted. It can also mean that younger academics are overlooked, finding it hard to get a foot on the ladder.

"I don't know whether you call it poaching, but there's no doubt the RAE has had a destabilising effect," says Professor Graham Zellick, Vice-chancellor of London University.

Hefce rejects much of the criticism. Changes have been made to the RAE over the years and the whole process has been refined to take account of complaints, it says. The funding council does concede, however, that selective funding of research can skew the mission of universities, sending them on a headlong grab for research ratings and money. It also concedes that universities are engaged in playing games to do well. But it has ways of checking on "cheats", it says.

Hefce has carried out studies that enable it to argue that the exercise is no monster, but has improved research quality. There's more research being done, it has greater impact, and the ratings and the number of people in 5-rated departments have increased with each cycle, according to David Pilsbury, head of research policy. A study conducted by the University of Leeds on international citations of papers shows there has been a real improvement, he argues.

Moreover, the football-style transfer market is more imagined than real, according to Hefce. The council has looked twice at the movement of academic staff and is doing so again. Around 1,600 research-active staff move between institutions each year, a low figure. Anyway there are real benefits to movement, says Dr Pilsbury. It makes institutions more dynamic and can give researchers better facilities. Moreover, there is no evidence that fewer young researchers are being recruited.

One of the abuses of the transfer market has been stopped. In the past, universities were able to hire academics just before the RAE and present their work as their own. Now the university that academics have left will be able to share credit for work done in the year before the RAE with the university they have joined.

Studies have been conducted on interdisciplinary research. The funding council has found no truth in the charge that such research is being penalised. Research across subject boundaries is flourishing, says Hefce: 80 per cent of researchers are involved in it. So, there is a mismatch between perception and reality.

Nevertheless, devices have been put in place to ensure that disparities between subject panels are ironed out. The funding council has also statedthat applied research should be accorded as much importance as pure research. Material doesn't have to appear in peer-reviewed journals, it says.

In the next RAE, the council is trying to ensure that panels take account of individual circumstances more than they have in the past. This is a response to criticism from the the AUT that the RAE discriminates against women who are part-timers or on maternity leave. The funding council wants universities to flag individual staff circumstances. But the AUT doesn't think that goes far enough, since it leaves too much power with departments who may simply not include women in their submissions.

Given that so much of the criticism can be rebutted, why do so many myths about the RAE endure? Some of it is attributable to academics' perceptions of subject panels. They may think they are taking a terrible risk in believing the funding council, and instead assume that it is safer to try to second-guess their subject panel. In other words, the traditional academic culture asserts itself.

So, will the RAE continue after 2001? The answer is almost certainly yes. The Government needs a method of directing money to research. And the message coming out of Hefce's fundamental review is that people prefer the RAE to any possible alternatives. But there are bound to be changes. Anyone who believes, however, that funds will be spread more generously around the sector is likely to be disappointed. At present, three-quarters of the money goes to 26 universities. Evidence from the review suggests this won't change radically.