How do you rate a university subject?

University vice-chancellors are furious with new proposals to assess the teaching of individual subjects, writes Lucy Hodges. They believe the new system will perpetuate unfair league tables of universities which are of little help to students

Universities and colleges are up in arms about new proposals to check up on standards and the quality of teaching in higher education. The row has brought together old and new universities and higher education colleges in a rare display of unity against what they see as the bossy- boots tactics of the Quality Assurance Agency (QAA), the body charged with maintaining and improving standards.

"It's a disaster," says Peter Knight, Vice-chancellor of the University of Central England. "Does any of it improve the quality of the experience of the undergraduate? No. I despair at what has happened."

Earlier this month, at their private residential meeting in Wolverhampton, university bosses reacted with fury to the notion that the teaching of subjects would be rated by means of differentiated judgements on a four- point scale. The judgements could be converted into numbers to produce league tables, which are much hated. The dons also objected to the idea that some universities - those with a good track record - would be subject to a lighter touch than others in the QAA's scrutiny.

They wrote to the Secretary of State for Education, David Blunkett, saying that the new methodology was fundamentally flawed and would lead to misleading league tables, which could harm both universities and students.

"We think the proposed methodology for rating teaching is not meaningful," said Howard Newby, president of the Committee of Vice-chancellors and Principals, and vice-chancellor of Southampton University. "It obscures more than it reveals. We favour a system which sums up the teaching quality of a department in a series of bullet points giving strengths and weaknesses. If you try to attach numerical scores to each aspect, you get an average overall score which is meaningless. You could have two or three things in which you are absolutely outstanding and four or five things in which you are dreadful, but the summative score would be halfway between the two."

James Wright, vice-chancellor of Newcastle University, complains that judgements that can be translated into numbers lead people to concentrate solely on the score a university receives: they don't read other information about the teaching in a given department: "What we need is a report that tells a reasonably intelligent and engaged person something of what it's like to be a student studying a subject in that university."

Behind all this lie fears about Big Brother watching over universities and eroding their jealously guarded autonomy in the interests of more accountability. The Government's ministers and its agency, the Higher Education Funding Council (Hefce), want teaching assessed with pithy judgements in three areas - teaching and learning, student progression and learning resources. The Higher Education minister Baroness Blackstone has said privately that she doesn't want "an essay on the teaching of every subject".

Ministers, as well as the QAA and the funding council, are adamant that students and their families need information that they can understand, rather than waffly education-speak.

For Peter Milton, the QAA's director of programme review, the compelling argument for differentiated judgements is the need for better public information. "It is quite clear from a survey commissioned by the funding council and the vice-chancellors that the customers want differentiated information," he says. "In fact they want even more differentiated information than we are suggesting."

The current system is numbers-based and rates subject-teaching on a four- point scale in six areas. Universities do not like it, and the QAA has persuaded ministers and the funding council that it should be dropped. Instead, it is proposing a different four-point scale, which would rate teaching in different subjects by using the following kinds of labels: approved highly commended; approved commended; approved; and failing.

Universities are worried that such judgements could be turned into numbers by newspapers - and that they would continue to be compared with one another in league tables. Dr Milton, though, points out that any system based on differentiated judgements can be turned into numbers.

The other point of contention is the proposal for some highly rated universities to be monitored more lightly than others. In her speech to the vice-chancellors' conference this month Lady Blackstone explained the rationale. "To look at all institutions, regardless of their previous record, would be both inefficient in its use of the QAA's resources and unnecessarily burdensome on institutions. Perhaps worst of all, by spreading resources too thin, it would run the risk of failing to spot real problems."

University bosses are not convinced that a lighter touch will reduce the burden on them. In addition, they want to know what criteria will be used to decide who gets the lighter touch and how many will be treated in this way - will it be 20 or 80 per cent?

"I think the light touch is an extremely difficult concept to make work," says Roger Brown, director of Southampton Institute and former head of the Higher Education Quality Council, a predecessor of the QAA. "Institutions are large and complex organisations, and quality assessment is not sufficiently far advanced that one can say this institution is serious about its standards and can be relied upon and this institution isn't."

Another problem is that universities could well be tempted to sue unless the criteria for a lighter touch are openly applied: they are very sensitive about being put into a second division.

For Mr Wright there is also the issue of fairness. "I simply do not see how you can give scores to all universities in all subjects on the basis of an assessment which is very different in some cases from that in others," he says. "In other words, you just have a quick look into a department and give them a set of scores, whereas in another you crawl all over them at great length before you rate them. I don't think that is sustainable in public in the modern age."

The new plan has been 18 months in the making and follows the setting up of the QAA on 1 April 1997 to replace two bodies that were accused of auditing universities to death. Last Friday the vice-chancellors' committee and the Standing Conference of Principals (SCOP), which represents higher education colleges, held crisis talks with the QAA and Hefce.

Agreement could not be reached on the rating of subject-teaching on a four-point scale, so discussions on that issue have been postponed. The proposals have, however, been approved by the English, Welsh and Scottish funding councils and will be published next month.

The changes amount to another nail in the coffin of institutional autonomy. For the Government, however, the changes are justified by our expensive higher education system's being funded largely by taxpayers and students.

"Only an independent body can provide the necessary confidence," says Lady Blackstone.

HOW UNIVERSITY TEACHING WILL BE JUDGED

1 The experience of students in teaching and learning, student progression and resources for learning will each be rated on a four-point scale, but with labels used instead of points. The labels are likely to be: approved highly commended; approved commended; approved; and failing.

2 Standards - what is attained at the end of a course - will be judged on a pass/fail basis in five areas. Such national benchmark standards are being produced in every subject by March 2000, and the quality of teaching will be measured against them.

3 Universities with a good track record will be subject to a lighter touch in the reviews conducted by the QAA.