How I wonder what you are!
Up above the world so high,
Like a diamond in the sky ...
From childhood, we are brought up with this romantic image of tiny points of light shimmering amid the black background of empty space. Unfortunately, the atmospheric turbulence that makes stars twinkle is the biggest obstacle to observing the night sky. Since the invention of the telescope almost 400 years ago, frustrated astronomers have been peering into the cosmos through a 100km blanket of air that bubbles and boils in the eyepiece.
Now, modern technology is on the verge of providing a breakthrough. For the first time, ground-based telescopes will have the potential to probe the universe in minute detail, a window of opportunity that previously has been open only to orbital observatories such as the Hubble Space Telescope.
The first step was to make the instruments as perfect as possible. Methods of manufacturing and polishing the large mirrors used in telescopes led to significant reductions in observational errors. Problems caused by air motion inside the protective dome were removed by an air- conditioning system. However, the most significant obstacle remained - atmospheric turbulence. It was this, more than anything, that prevented large instruments from separating close objects such as double stars any better than small amateur scopes.
Ironically, many of the new advances in astronomy, probably the science discipline with the least military potential, have been made possible with the aid of secret research carried out since the Seventies by the US Defense Department.
Spurred on by the Cold War, Pentagon scientists began to research how to destroy incoming enemy missiles by using laser beams. But ways had to be found of focusing such an intense beam of light on a distant target without it dissipating in the atmosphere.
Even more urgent was the need to spy on Communist bloc military satellites. In the Seventies and Eighties the US Defense Advanced Research Projects Agency looked for better ways to photograph and identify Soviet spacecraft. The answer it came up with became known as adaptive optics. Its purpose was to remove the distortion caused by random motion in Earth's thin blanket of air.
Although some work in adaptive optics was carried out in the astronomical community, the major advances were made under a cloak of secrecy by Defense Department scientists. Much of this work was carried out at the Starfire Optical Range, Kirtland Air Force Base, New Mexico, the most advanced adaptive optics facility in the world.
One of the key figures behind its success is the technical director, Robert Fugate. For 20 years he worked 80-hour weeks in the utmost secrecy, unable to explain to his wife and children why he was away from home every night. Then, in May 1991, the Starfire project was declassified. Dr Fugate recalls the startling turnaround. "Previously, we had been talking to such a small audience, and suddenly I was sharing our work with a group of 600 at an open meeting of the American Astronomical Society in Seattle.''
Dr Fugate and his colleagues now happily share their knowledge with the astronomical community and co-operation is flourishing. One of the most recent by-products is a $5m (£3.2m) grant to the University of Arizona from the Air Force Office of Scientific Research to fund a Centre for Adaptive Optics.
Limited viewing opportunities are now available with the 1.5-metre Starfire telescope. "We have recently concluded an agreement with the National Science Foundation to allow 24 nights for astronomical observations with this telescope," says Dr Fugate. "Half a dozen astronomers will be selected to use it. This is a unique situation for a military installation. We hope it will be the beginning of a dedicated science programme.''
The key to the success of their adaptive optics system is the availability of a bright guide star close to the heavenly object being studied. The sensor splits the wavefront of the incoming light from this star and sends the information on the amount of light distortion to the computer. This, in turn, is fed to the large, flexible primary mirror. The shape of the mirror is continually changed by mobile supports known as actuators. To cancel out the turbulence, each part of the mirror has to be independently adjusted by less than one hair's-breadth every millisecond.
Unfortunately, the adaptive system becomes increasingly complex as the mirror size increases. Ideally, very large mirrors should be equipped with thousands of actuators, but there is an inevitable compromise between the scientific requirements and the cost. Such balancing acts are becoming increasingly relevant as observatories around the world build 8- or 10- metre telescopes.
The other drawback is that adaptive optics are effective over only a tiny angle of sky. Over a wide viewing area, the turbulence varies so much that the sensor is unable to measure it in order to give a consistently clear image. This would not matter much if the sky were liberally scattered with bright reference stars, but this is not the case. Scientists such as Dr Fugate have thus decided to create their own guide stars.
The Starfire range solution is to use a beam of light generated by a 15-watt copper-vapour laser. An artificial star is then created by focusing the laser beam in the atmosphere and picking up the reflection. Light is backscattered either from a naturally occurring layer of sodium atoms about 90 kilometres above the ground, or from air molecules at altitudes of 10 to 40 kilometres. Dr Fugate says the sodium beacon will become the preferred method for telescopes larger than four metres. "The higher altitude means the error in the estimate of atmospheric correction is less - it more closely approximates to infinity.''
Safety remains a problem. Sentries are posted at the Starfire range to warn of nearby aircraft. The brilliant laser beacons are shut down if any aircraft accidentally strays into the illuminated area.
Furthermore, adaptive optics still have limitations. Fully compensated images have never been obtained across large fields of view. For an object the size of Jupiter, whose apparent diameter is about 40 arc seconds, there will be about 50 areas where the incoming light is affected in a different way by atmospheric distortion. One possible answer is to create numerous guide stars by deploying multiple lasers.
So what of the future? Dr Fugate is enthusiastic: "The next decade will see a revolution in optical astronomy. Systems will get cheaper and we will learn a lot more on how to operate them. Lasers will come into their own for very faint objects and the big observatories will use adaptive optics.''
Among the advances to be expected are the best-ever observations of distant galaxies and newly forming stars. Adaptive optics may even alter our perspective on extraterrestrial life if we detect Earth-like planets around nearby stars.