David Blunkett, Labour's education spokesman, said the results of last year's national tests were "very worrying indeed. We must do all we can to raise standards".
Results of tests for 14-year-olds, which were introduced two years ago, have not been published before because of the teachers' boycott.
Last year's test results are still incomplete. Only about one-fifth of secondary schools and half of primary schools reported results because of the boycott.
They cannot be compared with previous years because the tests have changed.
In reading and arithmetic, 20 per cent of seven-year-olds failed to reach the target of Level 2 or above on a scale of one (bottom) to 10 (top).
In spelling and writing, the figure was 30 per cent.
Overall, more than three quarters reached or surpassed Level 2 in English, maths and science.
Among 14-year-olds, 58 per cent reached the target of Level 5 or above in English, 60 per cent in maths and 64 per cent in science.
About one-third reached Level 6. In English, 1 per cent reached Level 8, the equivalent of a good GCSE pass and in maths just 2 per cent.
Standards at Levels 5 and 6 are set to challenge a typical 14-year-old.
Teachers' assessments of pupils tended to be slightly higher than test results.
Mr Blunkett said: "We must do all we can to raise these standards.
"Coming a week after a report showing that one in six adults has difficulty mastering basic literacy and numeracy, these are figures which illustrate more than ever the need to concentrate on raising standards from as early an age as possible."
Doug McAvoy, general secretary of the National Union of Teachers, said the proportion of children doing worse than average was roughly the same as the proportion doing better.
"If you set a test to gauge the average performance, the laws of mathematics dictate that you will get a certain proportion above and a certain proportion below.
"It was Kenneth Clarke [the former Secretary of State for Education] who first started this ludicrous nonsense about children having failed. He just showed he didn't understand the basic laws of mathematics."
However, Peter Smith, general secretary of the Association of Teachers and Lecturers, said: "The concerns have to be taken seriously. Teachers can't ignore the cross-party consensus and dismiss it as political infighting."
Gillian Shephard, the Secretary of State for Education, said: "I am publishing these results so that information is publicly available about the standards being achieved across the country, and schools themselves have clear national benchmarks against which to compare their own performance."