How do we know that is not how the real world runs - apart from the taxi driver's unrealistic generosity? In fact, how do we know that time has a direction at all?
At atomic level, it has long seemed that the laws of fundamental physics, as discovered by Newton and Einstein, would allow time to run forwards or backwards. Physicists say the equations are "time-symmetric": their accuracy is not affected by the direction of time. The fact that those laws do not seem to ban time travel has delighted science fiction writers for years.
But earlier this month 100 scientists from nine countries published the results of a three-year collaborative project. It demonstrated, for the first time, that in our universe at least, time moves in only one direction.
The experiment, called CP-LEAR (Charge Parity experiment in the Low Energy Antiproton Ring), was carried out to study the differences between matter and antimatter, the "converse" of matter. Antimatter particles have the same mass but opposite charge (and other characteristics) to their matter counterparts; in theory, every matter particle has an antiparticle. The electron's counterpart is the positively charged positron, for example.
When a particle and its antiparticle meet, the two annihilate each other in a burst of light energy. What physicists therefore find strange about antimatter is its general absence in the universe. Theory suggests that the Big Bang should have created equal amounts of matter and antimatter. Why didn't they eliminate each other at the universe's birth?
"That is the big mystery," says Professor Frank Close, from the Rutherford Appleton Laboratory in Didcot. He is presently on secondment to Cern, the European Laboratory for Particle Physics in Geneva, Switzerland, which led the CP-LEAR work. Antimatter has not been found "free" in the wider universe, despite careful searches.
One suggestion is that time affects particles and antiparticles differently. Early quantum physics assumed that, like other laws of physics, subatomic reactions would be the same no matter which way time flowed. If you started with a group of particles and antiparticles with known charges and "parities" (measurable quantities such as "spin" and "flavour"), then banged them together and measured the charge and parity of the resulting particles, the totals would be the same before and after. Physicists called this "CPT symmetry" - for charge parity time symmetry.
However, physicists always want to check such assumptions with the real world. They could not run time backwards, but they could experiment with antiparticles by pretending that antiparticles were just particles moving back in time.
Testing this idea experimentally meant evaluating the charge and parity of every particle produced in thousands of high-speed particle collisions in high-energy accelerators. In 1964 a Japanese team discovered that, in some reactions, the totals differed.
This effect, known as "charge parity violation", or CP violation, centres on an electrically neutral particle called the K meson, or kaon. In most reactions, it simply broke down into three pi mesons (pions). But in a fraction of cases, it decayed into only two pions - violating CP symmetry.
The experiment put a bomb underneath the idea that time could run in either direction. For 30 years CP violation bothered physicists; they needed more powerful particle accelerators to confirm what was happening.
Finally, in 1995, a set of new experiments set out to test this, using kaons and their antiparticles, antikaons. These are short-lived particles produced by the collision of antiprotons with hydrogen atoms. (Hence the use of the Low Energy Antiproton Ring for the work.) Kaons can turn into antikaons - and antikaons can turn into kaons - until they finally decay into an electron, a pion and a neutrino. By measuring the electron's exact charge, observers can determine whether the parent was a kaon or antikaon.
In a paper published last month in the journal Physics Letters, the international team working on the CP-LEAR experiment found that antikaons turned into kaons more often than kaons turned into antikaons. In other words, with time, antimatter is more likely to turn into matter - evidence of a clock running under the fabric of the universe.
Very possibly, this difference was one of the reasons our nascent universe turned into a matter-dominated place, instead of being snuffed out in a blast of gamma rays.
Of the CP-LEAR results, Professor Close says: "This is confirmation that everything we believe about the universe holds together."
So does that mean that time travel is impossible? Yes, according to Professor Close. "The way I describe it is that while you may not be able to tell which way a film is running when you see two billiard balls colliding, you'll certainly be able to tell if you see a white ball shooting towards a scattered group of balls on a table, after which they group together into a pyramid. You'd know it's crazy. You might be able to play tricks with time at the single-atom level, but not in the larger world."
The next step is to repeat the experiment using more massive, though also more elusive, subatomic particles. CERN and the American researchers now want to test CP violation using "bottom" quarks, one of the six varieties of quark (up, down, charm, strangeness, top and bottom). Quarks are the basic constituents of all particles with mass. Electrons consist of three quarks; mesons of a quark and an antiquark.
Time's arrow should be much more obvious with bottom quarks, but producing them calls for higher-energy collisions mimicking conditions in the early universe, when such quarks fleetingly roamed free.
The US is building an accelerator that, in about a year's time, will be able to produce bottom quarks. "That will give us an idea of what's happening," says Professor Close.
These experiments offer a justification for the cost of particle colliders, often derided by politicians looking for budgets to cut: they could tell us how the universe survived its birth.
"The idea of what time is at all and how at the atomic level we exist and pass through time - understanding that adds to the profundity of our understanding of the universe," says Professor Close. "Though on the other hand, when people ask me what time is, I sometimes tell them - well, it's the stuff that stops everything from happening at once."