Pioneering Painters: The Glasgow Boys 1880-1900, Royal Academy, London

For the final two decades of the 19th century, a rag-tag-and-bobtail group of artists hinted at a new, modern mood in painting. Sadly, it wasn't to last

If Edinburgh wanted Roman heroes, Glasgow wanted grit: work, industry, invention, a heady commission for a Victorian artist.

Like some unrelentingly industrious Glasgow engineer, James Guthrie paints himself on a French beach, painting. Not for him trellissed terraces or the Empress Eugenie: this one of The Glasgow Boys is crouched behind his umbrella, his face less important than his activity. The picture's surface looks as though it has been laid on with a trowel. Perhaps it was. To drive the point home, Guthrie calls the work Hard at It.

The Glasgow Boys were a rag-tag-and-bobtail group – "loose-knit" is hardly the word – whose best-known members are Guthrie, Arthur Melville and Sir John Lavery, and the Royal Academy show takes an orthodox line in defining The Boys as anti-Edinburgh, the snooty patrons of that city preferring their art classical.

To be a Glasgow Boy, you didn't have to be from Glasgow, merely broadly anti-academic, popular with Glaswegians, and hanging out with other Glasgow Boys. It may have been a lack of a manifesto that allowed William Stott and the others to paint so freely, in little groups in Kirkcudbrightshire or Grez-sur-Loing in the last 20 years of the 19th century.

When Stott painted the ferry at Grez in 1882, he too was being wilder than his solid name and his comforting subject suggest. For one thing, the young son of a Lancashire mill owner – Stott, then 25, proudly signs his name "of Oldham" – was going against his master, the Academic painter, Jean-Léon Gérôme. Gérôme had made his name in sword-and-sandal histories: the gladiatorial Thumbs Down, happily lost to a private collection, is typical. Stott, by contrast, paints the here and now and the rough and ready.

The subject of The Ferry is less the skiff of its title than the two young girls who watch it – farmers' daughters, probably, thick-shod and seen back view. You might reach for the word "Realism", although Stott's image is too pretty for that, too poetic. In any case, Courbet, from whom the mood of the picture distantly derives, was old hat by 1882, and The Ferry has none of the urbs in rure tension of Monet or Manet. Where the work's real rebellion lies is not in its subject but in its composition.

Screw up your eyes and The Ferry quickly resolves itself into four horizontal bands – light, dark, light, dark, reading down from the top – of sky, cottages and their reflection, river and river bank. It doesn't take much to lose the girls and punt altogether, to push the colour-bands flat up against the picture plane, reduce the houses to rectangles; to make The Ferry abstract.

Further south, in Aix, Cézanne was busy flattening Mont Sainte-Victoire into the planes that would lead to Cubism. Can Stott have known his work? In a sense, it doesn't matter. Coincidentally or not, The Ferry hints at a new mood in painting, that tipping-point where paint becomes more important than the thing painted.

The Glasgow Boys' free approach is shown in the direct gaze, and cabbages, of Guthrie's A Hind's Daughter. In fixing the artist with precisely the appraising eye with which he fixes her, the labourer's child bridges the gulf that would normally lie between them. Both she and Guthrie are at work, and so his paint is workmanlike – a scaffolded composition of horizontals and verticals, scuffed on in muddy greens and browns.

As with Stott's ferry, the image is both backward-looking, Courbet-ish, and strongly formal and modern. And it's this curious valuing of freedom and ingenuity that allowed Lavery and the rest to make work that, on occasion, wowed artists in Munich and Vienna.

And yet, the story of the Glasgow Boys does not end happily. That freedom didn't last. The same Lavery who painted Glasgow folk milling around the 1888 International Exhibition died a knight, best known for his society portraits. Guthrie gave up his gritty surfaces and grids and went back to painting the varnished nymphs and Venuses Gérôme had taught him. George Henry and E A Hornel travelled to Japan and returned with geishas and lotuses. Worse, they went all mystical and Celtic: the man must have a heart of stone who would not shriek with laughter at the pair's jointly painted Druids. As so often with British painting, as with Ben Nicholson and John Piper 40 years later, modernity ran out of steam. A shame, really.

Royal Academy, London W1 (020- 7300 8000) to 23 Jan

Next Week:

Charles Darwent courts immortality in the Ancient Egyptian Book of the Dead at the British Museum