Education: Old vs new: the battle for points in the ratings system

Do students get better teaching at the former polys, the 'new' universities? Analysis of the 'scores' awarded by assessors suggest that they lag behind, although established names are not immune to the shock of a low mark. Lucy Hodges reports.
Click to follow
The Independent Online
When assessors visited Exeter University's English department to check on the quality of teaching they found most of it to be excellent or satisfactory. But in a few instances they were disappointed. In some sessions staff were unsure of their material, and in others debate was pitched at the wrong level. On two occasions students stopped taking notes and began to talk among themselves.

Postgraduates experienced serious delays in the return of their work, and in some cases essays were never returned. It was perhaps not surprising that more than one-quarter of students on average had withdrawn from or failed their MAs between 1988 and 1992. Exeter's MA in English was found to be unsatisfactory by the higher education funding council assessors.

The university wasted no time in putting its programme right. The following year, assessors found its English department to be excellent.

"We turned round the school in a very short time," says Dr Malyn Newitt, senior deputy vice-chancellor. "We identified what the problems were and tackled them directly, head on. The whole university learnt from this experience. We're now extremely careful that the highest standards that some departments have always practised are applied throughout."

The first act was to replace the head of English. A central monitoring group was set up. The MAs were reorganised. One member of staff took early retirement, another was retrained. All course documents were rewritten.

The experience was a baptism of fire for the university. But, according to Newitt, it has had positive effects on the way Exeter operates.

At the same time the English departments at Teesside University and Chichester Institute of Higher Education were marked down too. Both were found to be satisfactory on a revisit.

Neither an "old" university such as Exeter, nor a new one such as Teesside, nor a college such as Chichester, can afford a bad rating. They can make or break departments, deter students from applying and bring the whole institution into disrepute. That is why assessments are taken so seriously. So important are they that universities prepare for them months in advance, hiring consultants, holding mock assessment visits and staging dress rehearsals of classes to be assessed. "We make sure that the act is as slick as we can get it," says Geoffrey Alderman, head of academic development and quality assurance at Middlesex University.

Nothing is left to chance. Nowadays the assessors, under the auspices of the new Quality Assurance Agency, are using a new methodology to rate departments' teaching. After their three-day visit, they give marks out of four under six headings: curriculum design, content and organisation; teaching, learning and assessment; student progression and achievement; student support and guidance; learning resources; and quality assurance and enhancement.

Universities are not supposed to add up the marks they receive under each heading, but they do. In general, they are very happy to score 20 or more out of 24.

One university, Luton, has compiled a league table of how it and all the other universities have been doing under the new points system. As a former college of higher education (Luton was never a poly) and the newest of the "new" universities, Luton is the butt of all jokes about standards in the new universities. But it can point to a respectable showing in the new teaching quality assessment system: an average of 20.2 in five subjects. Of the 35 new universities in England, Luton came ninth in assessments carried out between April 1995 and June last year.

Dai John, Luton's deputy vice-chancellor, believes the new universities do a better job of teaching students than the old universities. "Their priorities are different," he says. "Many of the staff here see as their prime responsibility ensuring that students have a good experience. Many staff in the old universities will see research as their first priority."

That view is endorsed by many of his colleagues in the new universities. It is also echoed by Roger Brown, head of the old Higher Education Quality Council, and newly appointed director of Southampton Institute. "In general the former polys and many of the colleges provide students with a better- quality experience because they're more geared to their wants and needs," he says.

Scores in the teaching quality assessments, however, do not bear out that assertion. Some new universities fall down on learning resources - which means libraries and IT equipment. That is because they have been funded at a lower level than the old universities and have not built up the same superior libraries. They also do less well on teaching and learning - largely, it is thought, because students are taught in bigger groups. It is not uncommon to have classes of 20 to 25 in the new universities, whereas in the old they are often kept below eight. The quality assurance people have carried out comparisons between old and new universities over the past five years and on most criteria the old universities come out better.

One crumb of comfort for the new universities lies in the imperfect nature of the teaching quality assessments. According to Simeon Underwood, of the Professional Courses Unit at Lancaster University, who has looked in detail at how the assessments work, it is difficult to use the scores as a basis for comparing institutions. That is because of the wide variation in scores being achieved in different subjects. For example, it is much easier for universities to do well in some subjects than in others. In electrical engineering, one-third of institutions visited in the academic year 1996/97 scored a 21 (out of a maximum of 24) or more; but in history of art, 70 per cent scored 21 or more.

He has also found that, although some universities do fare badly, that happens rarely. Under the old methodology, which clobbered English at Exeter, Teesside and Chichester, only 12 departments were graded unsatisfactory out of 972 assessments. In the 1995/96 assessments only two grade ones (bottom grades) were awarded out of a possible 1,662. If one compares that with secondary schools, where the Chief Inspector, Chris Woodhead, talks about one in 10 needing substantial improvement, one can only conclude that higher education is a lot better than many claim - or there is a problem with the figures. You don't have to be very cynical to guess which view politicians take, says Mr Underwood.

Finally, he has found marked differences in the points awarded to institutions under the various headings. Many more fours (the top mark) are given in student support and guidance than for other aspects. Many more twos (a relatively low mark) are given for learning resources and quality assurance and enhancement. For all those reasons it is difficult for quality assessments to be used as a basis for comparison on teaching.

But, as one of the small army of experts helping universities to boost their scores in quality assessments, Mr Underwood has found noticeable differences in the performance of the old universities, the former polys and the colleges. He detected a difference of 1.5 points in the overall average score between new and old universities in the 1996/97 round of assessments. The old universities scored 21.5 on average; the new universities 19.5. At the bottom came the colleges, with an average of 18.7 points.

Nevertheless, Mr Underwood found that some universities were doing well regardless of their provenance (see table above). He has found Manchester Metropolitan, a new university, doing better than Manchester. Among the colleges, he found Southampton Institute, an institution which has been hauled over the coals for its management, doing particularly well.

Like Exeter, many universities believe the quality assessments have done them the world of good. Certainly, they have made old universities much less cavalier about the process of ensuring quality.

Simeon Underwood thinks the system is "undoubtedly a force for good". He adds: "It has focused attention on teaching and learning and the student experience at a time when many universities were becoming obsessed by research and the research assessment exercise."

But not everyone is a fan. Sue Reid, deputy director of the school of law, humanities and international studies at Teesside University, complains that the assessors who marked down Teesside's English department had difficulty in understanding the modular system, which gave students a great deal of choice in what they could study.

"We had to restructure the English department in the light of the report, and this we have successfully done," she says. "And, yes, this has been along more old-fashioned lines."

Individual academics complain that the bureaucracy accompanying quality assessment militates against good teaching in which the interested academic tries to convey a passion for ideas. Susan Blackmore, a columnist for The Independent, who teaches psychology at the University of the West of England, says she is bombarded with instructions about how to teach, and as a result has got out of teaching where she can. Nowadays she teaches only third-year students because she can pass on what she loves: the psychology of consciousness and parapsychology.

"They don't ask if you are a leader in your field or whether you are putting over the latest ideas," she explains. "It's as if they don't care about that."

Teaching assessment ratings (out of 24)

'Old' Former Higher ed universities polytechnics colleges

1995/96 20.1 19.4 19.1

1996/97 21.5 19.5 18.7

n Among the old universities performing outstandingly are Oxford, Cambridge, York, Essex and Warwick.

n Among the new universities performing outstandingly are the University of the West of England in Bristol (a 23 out of 24 for sociology), Northumbria (23 for modern languages) and Oxford Brookes (thought to be the first new university to be awarded 24 out of 24 - for planning).