A century and a half later the Derby is still there, but in recent years there has been no need for anyone on the downs to jostle, surge or scramble. Just a decade ago, crowds of up to half a million continued to overrun the hills to watch the world's most famous race for nothing, while the course's infield was an impenetrable mass of bodies, every last one of them, apparently, queueing for the bar.
Yet even then the crowds had already started to trickle away. By last year the downs were all but empty, and on the infield there was more space than people. Gone with the deserters was the riotous party atmosphere which had always given Derby day an addictive and almost dangerous edge. Spectators who had been noting the decline for several years suddenly began to use a new word in their discussions. Irreversible.
Such was the depressing trough in which Edward Gillespie found the Derby 11 months ago. He had just been appointed managing director of United Racecourses, the company which owns Sandown and Kempton in addition to Epsom, after several innovative and highly successful years as the general manager at Cheltenham.
The National Hunt Festival had prospered so grandly under Gillespie's stewardship, in fact, that some critics awarded him the ultimate accolade, describing it as "too successful".
If the Derby is ever to regain a similar level of popularity, Gillespie is probably the only person who can make it happen, and since last July he has set about the task with typical zest. Quite simply, he gets things done. For years there had been talk of shifting Derby day from Wednesday to Saturday. Within months of Gillespie's arrival, vague aspirations had become fact.
There had been justifiable criticism of the overall standard of racing at the four-day Derby meeting, and on the big-race undercard in particular. This year, the meeting has been trimmed to three days, with the Group One Coronation Cup to be run two races before the Derby on Saturday.
Next, he addressed the long-standing problem of the paddock, situated half a mile away from the stands down a narrow path. It was solved so simply - by installing a new paddock behind the grandstand - that you can only wonder why no one considered it before.
It is typical of Gillespie that as he looks forward to the race in which he has invested so much effort, he is more concerned with what remains to be done. "I tend to be frustrated by what we haven't achieved," he says, "but by God we have achieved a lot in the last few months, and it feels like something is going on."
Typical too that he describes the likely absence on Saturday of Celtic Swing as "a blessing, as it's made people realise that they really care about the race."
The essence of Gillespie's approach is "to make it more accessible. I think Epsom has suffered slightly from being an event which by the nature of the sort of people who train and own horses nowadays has become a little distant from people's lives.
"It may still be impossible for them to identify with some owners and trainers, but at least because we've moved it to a Saturday and various other things it will get them closer to the event.
"The trick with some courses is that people actually care what happens and leave a little bit of themselves there. It becomes part of their life, and what we are doing is all about bringing Epsom closer into people's lives."
What no one will know until Saturday is how many people are still prepared to let the Derby into their life. All great races and meetings need a solid, fertile core of support. The Irish, for example, will never stop travelling to Cheltenham, and money, both old and new, will forever want to show itself off at Ascot. The Derby, by contrast, was always the workers' day at the races, but the sort of workers who cherished it for generations have all but vanished. It is a daunting problem, as Gillespie acknowledges.
"I think this year the Derby has got to find a slightly new identity," he says. "It should of course be the workers' day out, but you can't rely on them coming and spending money any more because they're actually going shopping or down to the garden centre.
"So we've got to make it something which is a bit of a party at all levels. The great thing about the Derby is that it has got so many different levels, from the Queen's Stand through to the hill. It reflects society, and the daring bit is to allow them all to mix up, which I think the old system didn't."
Unlike some other racecourse managers, who decide for themselves what the public wants, Gillespie will be mingling with the crowd throughout the enclosures this weekend, digesting both praise and criticism.
"There's no point having a meal in the committee room. I'm very much part of the front-of-house team, and if you were running a theatre you would be out there listening to people. You can certainly detect whether people are having a good time or not."
There will also be soundings taken from Epsom's residents, who have never accepted their local sporting institution in the manner of those other south London suburbs, Wimbledon and Twickenham.
"We're trying to do things with them, to create a race which reflects well on the town rather than just being the event which goes on up on the hill," Gillespie says. His ideal Epsom resident, perhaps, would be the local who observed in the early 19th century that "if the weather be fine, there are seldom less than 60,000 assembled when the Derby be run, and of these the vicious and unprincipled form a tolerable proportion."
Just how many will assemble on Saturday is not something which Gillespie would care to predict, although "it would be sad if we stood there with 10,000 people on the downs and realised we'd made the biggest mistake of all time."
That, surely, is not going to happen. However, the disturbing thought for Gillespie, as he tries to revitalise the Derby, is that there is just as little chance that the Times will report, as it did in 1829, that "the whole world was at Epsom yesterday."Reuse content