Memorabilia from the Wenlock Olympian Society inside Much Wenlock Town Museum

Take your time in the Shropshire town where the modern games began, says Mark Rowe

How has the Olympic flame been for you? Aside from the torchbearers, did you find the Olympian ideal burning brightly in the fleet of corporate advertising that trundled through town and countryside? Thought not. But while the world's attention may be poised to turn to a fortnight of sporting activity in London next month, there is one place that is home to what you could genuinely characterise as the original Olympic spirit.

Much Wenlock – the Shropshire town, rather than the hypnotic one-eyed Games mascot – is where the modern Games began, and this walk (not a sprint) around the town and countryside provides the chance to peer a little more closely at an oddly moving story.

Much Wenlock was the home town of William Penny Brookes, a quintessential Victorian do-gooder. Brookes was a classicist, philanthropist, and doctor, who worried tremendously about the self-inflicted misery – from the plague of drink to the lack of education – of the town's inhabitants.

Devoting himself to dragging the town's civic pride out of the gutter, Brookes went about improving bodies and minds, first setting up a public library and then, drawing on the long defunct Greek games, founding the Wenlock Olympian Games in 1850. This annual sporting event sought to improve the prospects, morals and behaviour of the town's working folk. Word spread, and meetings and correspondence followed with Baron Pierre de Coubertin. While de Coubertin ultimately hogged the credit, Brookes's legacy was assured – at least in Shropshire.

Striking out of the town centre along Gaskell Fields, the legacy looks alive and well. Every July, the Olympian Games take place here, as they did in Brookes's day. The events have changed a little though. While the Victorians pursued pole vaulting and tilting (threading a lance through a ring while hurdling on horseback), they also favoured blindfold wheelbarrow races and penny-farthing cycling sprints. Women's competitions were limited to handicrafts and choral singing.

A steady climb away from the village soon deposited me on Wenlock Edge, the graceful curving limestone ridge that pushes upwards from the town to Craven Arms, 18 miles (30km) to the south-west. The ridge is topped by pollarded woods that tumble down steep scarps to the plains to the north, while arable and grazing fields drift steadily west towards the town. Limestone and aggregate quarries pock-mark the landscape.

Tempting as it was to keep going – Craven Arms claims to be the only town in Britain named after its pub, and so deserves a visit – the charms of Much Wenlock won. Forget the Olympic tag; this town deserves a visit at any time. Most of it evolved as part of the Wenlock estate, which kept a steady hand on any development. The consequence is that Much Wenlock's medieval heart was protected from the advances of post-war planners and presents an architectural dash through the centuries, from a 12th-century priory to Tudor and Elizabethan flourishes and grander Georgian townhouses.

The High Street is full of creaking, crumpled timbered buildings as well as the Corn Exchange, an Italian-style arcaded loggia. Look out for narrow alleyways, known as "shuts", passageways that lead through to rear yards. The cafés, delis, art galleries and bookshops encourage you to linger and explore.

The Brookes story is relayed in full at Much Wenlock's recently renovated museum, at the bottom of the High Street. Stuffed with interest, this is one of the world's great little museums. The Olympian story is retold in some detail, but the museum also branches out, and helpfully explains the geology of the countryside you've just walked in. Wenlock Edge began life millions of years ago as a coral sea somewhere south of the Equator, and it's not that uncommon to pick up a piece of coral from the soil along the Edge. Over the road, original Elizabethan black and white beams stitch together the Guildhall and old court house, which houses an impressive set of original stocks, along with Tudor wood panelling. Just along from the Guildhall, at 4 Wilmore Street stands Brookes's former house; in the Norman church opposite a plaque commends his "development of the manliness of the human race"; he lies in a family tomb in the churchyard.

The home straight, so to speak, of this walk is the town's priory. Serenely set among leafy grounds with fetching topiary shaped as squirrels, bears and other animals, the priory has a haunting beauty. Founded by the Saxon saint Milburga, the site is an utter enchantment. Sit here, and the cacophony around the London Olympics will be reassuringly distant.