Dr Hilary Radnor, one of the report's authors, said yesterday that some markers "just did not know the stories" of the three Shakespeare plays included.
The highly critical report from Exeter University backs up English teachers' claims that the test marking unfairly penalised bright pupils.
Thousands of pupils had their marks changed after teachers appealed against the results, but the report - commissioned by the School Curriculum and Assessment Authority - suggests that many more may have suffered because of poor marking.
The report, based on 2,000 pupils, found that some markers were confused about who were Capulets and who were Montagues in Romeo and Juliet. One ticked an answer that said Juliet wanted to marry Paris and another marked as correct the statement that, in A Midsummer Night's Dream, Bottom was a carpenter.
The report says that some markers knew so little about English teaching that they gave pupils marks for naming figures of speech when it was clear they had no idea what they meant.
One marker ticked one pupil's script when she wrote that Romeo uses "Simples and Metfords" instead of similies and metaphors.
Most of the 2,500 markers were practising English teachers or recently retired, according to the authority. Dr Radnor said bright pupils who did not give stock answers had been penalised because teachers did not know the three Shakespeare plays well enough.
Pupils were given a list of "bullet points" to include in their answers, but those who referred to material outside the points had sometimes not been given credit for their greater knowledge. Dr Radnor said: "If markers did not know the context very well, anything that wasn't on the list didn't get a mark."
The report says: "The script scrutiny exercise has thrown up markers who ticked as correct misunderstandings about the plays in the pupils' response."
The three Shakespeare plays chosen were Romeo and Juliet, Julius Caesar and A Midsummer Night's Dream.
Dr Radnor recommends that Shakespeare should not be assessed by a single timed written test, but by coursework marked by children's own teachers.
A spokesman for the assessment authority said that this year they would ensure that teachers used as markers had done their homework on Shakespeare. "It isn't that they don't know the plays, but that they have not necessarily taught them recently," he said.
Markers will receive more training and their own work will be marked before they are accepted. Recruitment of English teachers for this year's test among practising and recently retired teachers was going well, the spokesman added.
But Anne Barnes, general secretary of the National Association for the Teaching of English, said: "There is no evidence that we are going to get better markers this year than last.
"It requires people with knowledge of teaching children of that age. The authority needs to address the problem of attracting English teachers into marking."
The authority is piloting different ways of testing Shakespeare, including the use of class assessments at a time chosen by teachers, but ministers have so far resisted testing through coursework.
Leading article, page 14Reuse content