It is the most sobering fact in physics. Everything that has ever been observed, from elementary particles to clusters of galaxies hundreds of millions of light years across, makes up just 4% of the Universe. The remaining 96% is currently missing: dubbed dark matter (23% of the Universe) and dark energy (73%), their existence is inferred by the effect they have on ordinary matter, but their natures are unknown.

Astronomer Brian Schmidt, from the Australian National University's Research School of Astronomy and Astrophysics in Weston Creek, Australia, is partly to blame for revealing this unsettling situation. In 1998, his observations of exploding stars known as supernovae led to the conclusion that the Universe is imbued with something that is causing the expansion of space-time to accelerate. This wasn't what he expected to see: although it had been observed that the Universe was expanding, the prevailing idea was that this expansion was decelerating and that gravity would eventually stop the growth.

Death of a star. Type 1a supernovae are the 'standard candles' of cosmology. Credit: NASA, ESA, CXC, SAO, THE HUBBLE HERITAGE TEAM (STSCI/AURA), J. HUGHES (RUTGERS UNIVERSITY)

Schmidt recalls how he joked about possible outcomes before starting. “The worst thing that could happen to us is that we find that the Universe's expansion is accelerating because people would just say that we were crazy.” Yet, in 1998, this is exactly what they did see. What's more, he adds, the observations suggested this dark energy could be a form of antigravity, “having physical properties that essentially makes gravity repulsive instead of attractive — it's a pretty big deal.”

The discovery was made jointly by two groups who shared the 2011 Nobel Prize in Physics: the Supernova Cosmology Project, led by astrophysicist Saul Perlmutter of Lawrence Berkeley National Laboratory and the University of California, Berkeley (awarded half of the prize), and the High-Z Supernova Search Team led by Schmidt and Adam Riess of Johns Hopkins University in Baltimore, Maryland. Schmidt and Riess's half was split between them.

Together with ordinary matter and radiation, dark energy and dark matter are the basic ingredients of the standard model of cosmology that accounts for 13.7 billion years of cosmic evolution. Dark matter might one day be revealed by experiments — one candidate, 'supersymmetric' particles, are being searched for by physicists at CERN's Large Hadron Collider, for instance. Dark energy, on the other hand, presents a new kind of bafflement. “You have no intuition or guidance because the physics is so different to anything we have experienced,” says cosmologist Robert Caldwell of Dartmouth College in Hanover, New Hampshire.

Relative predictions

The framework for the standard model of cosmology is provided by Einstein's 1916 general theory of relativity, which regards gravity as a consequence of matter causing space-time to curve. The general theory predicted a dynamic Universe that could either expand or contract. Since scientists at the time believed the Universe to be static, Einstein inserted an extra term into his equation to enable his theory to fit with convention: a 'cosmological constant' called lambda. Lambda was generally abandoned a decade later, after astronomer Edwin Hubble and others showed that the Universe is in fact expanding and not constant. But perhaps Einstein was on the right track, even if he did not know it: general relativity is consistent with recent observations if the cosmological constant is reinstated with, as Schmidt puts it, a “ridiculously small value” of roughly 7 × 10−27 kg m−3. (By comparison, the density of the air around us is about 1.2 kg m−3 — a billion billion billion times greater.)

The Mount Stromlo Observatory, where Brian Schmidt (pictured) made his observations, was destroyed by bushfire in 2003. Credit: ANDREW CAMPBELL/FAIRFAX SYNDICATION

This tiny number has big consequences for cosmic evolution because, unlike matter or radiation (whose density decreases as the volume of space increases), lambda has the same value regardless of the size of the Universe. Consequently, if dark energy is equivalent to lambda, its contribution to the total density of 'stuff' in the Universe will be greater in a bigger universe, and vice versa. This is important because the relevant amount of stuff determines whether the expansion of the Universe will eventually grind to a halt or continue for eternity.

Data suggest that the Universe passed a tipping point about 6–7 billion years ago (see 'A universal lifecycle'). Prior to this point the Universe was dominated by matter — visible and dark — which was slowing the expansion. Then dark energy took over, driving an ever faster — perhaps exponential — expansion. So instead of the Universe gradually winding down under gravity, dark energy propels it to a far different fate, explains Caldwell. He says we could be headed towards a 'Big Chill' in which our local group of galaxies becomes an island in an eternally cold and lonely Universe, or possibly even a 'Big Rip' — a catastrophic demise that would “pull apart the Milky Way, the Earth, and ultimately atoms and nuclei”.

Figure 1
figure 1


The cosmological constant is so small, yet not quite zero, that to explain it some cosmologists are entertaining the idea that our Universe may be part of an infinite number of universes, each encoded by different fundamental laws and causally decoupled from one another. In this picture the 'unnaturally' small value of our cosmological constant is statistically plausible given the large number of other possible universes out there — and we just happen to be in this one. If its value were much different, galaxies, stars and people would not exist. Such anthropic reasoning is not to everyone's taste. “It could turn out to be the best way to explain dark energy,” says Perlmutter, “but it's a lot to take on board unless it makes some kind of prediction that we can test.”

Imagineering the universe

Cosmologists are in need of a new theoretical insight — a “big leap”, says Schmidt. Since 1998, more than 7,500 papers with “dark energy” in the abstract have been published — most of which tackle its theoretical roots. Within these theories, a cosmological constant is not the only solution proposed: one alternative invokes a new type of field called quintessence (from the classical name for the fifth element beyond earth, air, fire and water). Rather than remain constant, quintessence allows dark energy to vary over time and space. It is described as a scalar quantum field — an entity associated with an elementary particle that has no spin — defined by a simple value at all points in the Universe.

Scalar fields are not new ideas: they are also suspected to have caused an extremely short-lived period of expansion of the Universe called inflation, when space expanded by a factor of 1080 from approximately 10−35 to 10−32 seconds after the Big Bang. Inflation would have amplified the tiny events that are naturally part of space-time (quantum fluctuations) to macroscopic scales, generating regions of differing densities, which allowed gravity to kick in and cosmic structure to form. Until recently, scalar fields were the stuff of imagination. But CERN's announcement in July 2012 of a new particle suspected to be the Higgs boson (a scalar particle) suggests that space is filled with at least one scalar field — the Higgs field that gives elementary particles their mass.

“Cosmologists have been working with fundamental scalars for 30 years without knowing if they exist,” says astrophysicist Pedro Ferreira of the University of Oxford in the United Kingdom. “So if this new particle is a scalar then it's a big deal.” Ferreira adds, however, that the energy scales of the Higgs field, dark energy and inflation differ by so many orders of magnitude that it is difficult to build convincing theories linking the phenomena.

I don't think that we've learned anything significant about dark energy since 1998.

However, scalar fields are only one option. “There are hundreds if not thousands of possibilities,” says Ferreira. Dark energy might be a vector field — one that has both a magnitude and a direction at each point in space. Other, wilder theories postulate higher dimensions of space. But Ferreira laments the overall lack of progress. “I don't think that we've learned anything significant about dark energy since 1998.”

Some cosmologists don't even think dark energy is real: rather, it is a consequence of two assumptions that underlie the standard cosmological model. The first assumption is that Einstein's general relativity holds everywhere in the Universe; the second is that the Universe is homogenous and isotropic. So far, both assumptions have been supported by observations, yet at distant regions of space they might break down.

One doubter is theorist Subir Sarkar at the University of Oxford: “I don't dispute the measurements, but I think dark energy is an artefact of interpreting data in the simplest possible cosmological model — formulated when there were essentially no data available to test homogeneity.” Sarkar argues that the data show only that distant supernovae are fainter than expected for objects at that redshift (the change in colour of light from supernovae as space-time stretches). It is another assumption, however, to say that that region of the Universe is undergoing the same rate of expansion as here. “The expansion rate would vary in space — not just in time — in an inhomogenous Universe.”

It could be that general relativity needs some tweaking when it comes to describing the largest scales. Physicists have grappled for some time with modified-gravity theories as alternatives to invoking dark matter, but the accelerating expansion of the Universe adds another experimental observation for theories to accommodate. “We are in a mess with dark energy,” says Sarkar. “Astronomers like the simplest solutions of general relativity but there are other, more complicated solutions that we have to consider.”

Dark energy may not be an attractive proposition, admits Schmidt, but with it the standard model of cosmology correctly predicts the geometry of the Universe, temperature fluctuations in the cosmic microwave background radiation (CMBR) and the abundance of light elements forged in the Big Bang. “The only problem,” Schmidt notes, “is that it requires us to invent 96% of the Universe.”

Finding dark energy

Pending new theoretical insights, researchers are badly in need of more data. Upcoming experiments will refine measurements of the acceleration. Since 1998, the precision of the supernovae data has been improved tenfold, says Schmidt. But measurements are now at the level in which systematic errors, such as the ability of astrophysicists to measure how bright things are, dominate statistical errors caused by the finite number of objects observed. Knowing the brightness of supernovae is vital because these objects are considered 'standard candles' with which to infer cosmic distances.

The “big one” for dark energy, says Schmidt, is the European Space Agency's (ESA's) Euclid mission, which in June 2012 got the go-ahead for construction, to launch in 2019. This satellite will use visible-wavelength and near-infrared cameras to map the three-dimensional distribution of more than one billion galaxies stretching over distances of tens of billions of light years. Euclid will measure so-called baryon acoustic oscillations — periodic clumpiness in the distribution of matter caused by sound waves that propagated in the hot plasma of the young Universe. In this way, the mission aims to track how the expansion of the Universe has changed over time.

Euclid will also exploit a technique called weak gravitational lensing, the bending of light from faraway galaxies as it passes through the gravitational field of nearer matter. Like acoustic oscillations, gravitational lensing is sensitive to the structure of matter in the Universe over time. Weak gravitational lensing tells cosmologists how quickly structure formed and so allows detailed tests of general relativity on the largest scales. “Modified gravity used to be a fringe thing, but it has now become an integral part of missions such as Euclid or SKA [the Square Kilometre Array radio telescope],” says Ferreira.

NASA has proposed a similar probe to Euclid called the Wide-Field Infrared Survey Telescope (WFIRST), designed to give researchers a better handle on the brightness and spectral evolution of supernovae. Should the project get the go-ahead, WFIRST would measure the properties of more than 1,000 supernovae at various distances. Meanwhile, NASA's James Webb Space Telescope (JWST), the successor to the Hubble Space Telescope, is due to launch in 2018 (see 'A conversation about observation', page S5). In 2004, it was Hubble's high-redshift (or deep-field) observations of supernovae that enabled Riess and colleagues to pin down the time at which the cosmic acceleration kicked in; the JWST will take this further. Cosmologists are also looking forward to tighter constraints on the geometry of the Universe from recent measurements of the CMBR taken by ESA's Planck satellite.

Dark future

Shortly after the 1998 result, recalls Perlmutter, his group was discussing what it would take to obtain sufficient data for theorists to gain an understanding of dark energy. “We knew we had a decade of development work to do, and now that we have done that we are ready to build these projects,” he says. “If we can spend the next decade measuring things then I am optimistic that we will make progress in understanding dark energy. This is a fairly new field, so there is a reasonable chance of a breakthrough.”

First, Perlmutter and others will take Earth-based measurements. These include the Dark Energy Survey, a specially adapted camera fitted to the 4-metre Blanco telescope in Chile, which from this year will use supernovae, baryon acoustic oscillations, weak lensing and measurements of large-scale structure from hundreds of millions of distant galaxies. Weak lensing is also being exploited by an instrument fitted to the 8.2-metre Subaru telescope in Hawaii. Towards the end of the decade, an experiment funded by the US Department of Energy will study baryon acoustic oscillations using one of the National Optical Astronomical Observatory 4-metre telescopes — a project called BigBOSS is a leading contender.

Meanwhile, projects at telescopes in across the world — for example the Skymapper telescope in New South Wales — will give astronomers a better idea of the brightness calibration of supernovae in the nearby (low-redshift) Universe. Further in the future is a high-resolution optical spectrograph called Codex, which will aim to provide an absolute measure of the cosmic expansion rate (enough to convince even the hardest of acceleration sceptics). Codex will be installed on the recently approved European Extremely Large Telescope in Chile and will measure the change in the Hubble expansion rate — the 'redshift drift' — over a ten-year period.

The data will drive new theories, researchers hope. “In terms of understanding the rhyme or reason of how it works, there have been many papers on dark energy but no breakthrough ideas,” says Craig Hogan of the Fermi National Accelerator Laboratory in Batavia, Illinois. “As far as experimental prospects go, there are experiments that can test extensions of standard physics, if not dark energy directly, that may point towards some emergent theories of cosmic acceleration.”

They need to seize the moment: if the expansion of the Universe is accelerating, then in a few tens of billions of years everything will have disappeared over the cosmic horizon and be too far away for its light to ever reach us. “We are losing information by the second,” says Schmidt. “The number of atoms that are in our horizon is dropping rapidly and has been for about 6 billion years, so it's an interesting time that we live in.”