A gene with links to late-onset dementia is also suspected of boosting people's brains in their youth, according to a study.
People who inherit one copy of the gene variant, known as APOE e4, have up to four times the normal risk of developing Alzheimer's disease in later life.
Neuroscientists tested the cognitive skills of those with the gene variant, which is found in around a quarter of the population, against those without it.
They also looked at the brain structure and brain activities of both groups during the tasks.
The study, led by the University of Sussex, found that young people with the same variant performed better in attention tests, including episodic memory of words and spotting number sequences.
Experts suggested that while the e4 variant might help boost the brain in early life, it could also increase the possibility of "burnout" in old age.
Lead researcher Professor Jennifer Rusted said: "Earlier studies suggested that those with the e4 variant outperform those without it in tasks such as memory, speed of processing, mental arithmetic and verbal fluency.
"But it is also well-established that this gene is a risk factor for Alzheimer's disease.
"The suggestion is that while this confers cognitive advantages in early life, leading to higher achievement, it may also increase susceptibility to memory failure as we enter old age.
"Our study is the first to show that subtle differences in the structure and activation of the brain during cognitive tasks in APOE e4 carriers are linked to their cognitive performance.
"It is possible that the brain over-activations that we see in youth have negative effects over the longer term and contribute to a kind of 'burnout' in older adulthood."
The study - APOE e4 polymorphism in young adults is associated with improved attention and indexed by distinct neural signatures - was funded by the Biotechnology and Biological Sciences Research Council and is published in NeuroImage.