Data Leakage and Overfitting in Biological Age Prediction Models

Start with what the model predicts

A model trained to predict chronological age learns patterns associated with time since birth. Its accuracy does not, by itself, establish that it measures health, functional decline or an underlying rate of ageing. Models trained on mortality or other outcomes address different prediction targets.

The difference between predicted and chronological age can contain biological information, measurement error and statistical bias. It should not automatically be interpreted as years of life gained or lost. Human methylation research illustrates why improving chronological-age prediction and establishing biological relevance are separate tasks. Read the primary study.

Leakage and overfitting are different problems

Data leakage occurs when information from evaluation data improperly influences model development. Selecting age-associated molecular features using all participants before splitting the dataset is one example: the eventual test participants have already helped determine the model’s inputs.

Overfitting occurs when a model or selection procedure adapts too closely to the available data, including noise. It can occur without leakage. Trying many algorithms and reporting the one with the best cross-validation score can overfit the selection criterion itself. An evaluation used repeatedly to choose models no longer provides an independent assessment of the chosen model. These distinctions draw on leakage research and model-selection experiments.

Where an ageing-model pipeline can leak information

The following are illustrative applications of general leakage principles, not allegations about any particular ageing clock:

Feature selection before splitting: choosing methylation sites or proteins using age labels from the full dataset.
Shared participants: placing different visits, tissues or technical replicates from one person in both training and testing when the claim concerns new people.
Preprocessing across the split: estimating imputation, scaling or dimension-reduction parameters using evaluation samples instead of fitting them within training data.
Future information: using measurements collected after the intended prediction time to predict an earlier outcome.
Overlapping cohorts: calling a dataset external without checking whether its participants appeared in development data.

The split must match the intended use. Predicting a new participant, a future visit or a new laboratory setting requires different independence checks. See the methodological analysis of leakage.

What stronger validation looks like

Separate model selection from performance estimation. In nested cross-validation, inner folds select features and tuning settings; outer folds evaluate the entire selection procedure on withheld data. Learned preprocessing must also remain inside the appropriate training folds.

After development, evaluate the frozen pipeline on an untouched test set. If that set guides further revisions, another independent evaluation is needed. Regularisation can reduce model flexibility, but it cannot repair a contaminated evaluation.

These safeguards address selection bias; they do not guarantee transfer to populations, tissues or measurement platforms absent from development. The distinction between fitting, selection and final evaluation is supported by Cawley and Talbot’s methodological study.

What the evidence establishes

Human data: A study of human brain-connectivity datasets experimentally compared leaky and non-leaky prediction pipelines, including age prediction. Feature-selection leakage and repeated participants could inflate apparent performance, while effects varied by outcome and leakage mechanism. Some corrections produced different directions of distortion. This supports auditing the pipeline rather than assuming that every leakage error has the same effect. Read the study.

Animal and laboratory evidence: This article does not present animal lifespan experiments or cell-culture rejuvenation results. The human neuroimaging findings do not establish how much leakage affects every methylation, protein or animal ageing model. Examples involving molecular features above are methodological illustrations.

Leakage-free prediction is necessary for credible evaluation, but it does not demonstrate that changing a clock score causes better health.

Questions to ask when reading a clock study

Check whether participants were separated before feature selection, whether repeated samples stayed together, and whether evaluation data influenced tuning. Look for a clear distinction between chronological-age accuracy and associations with health outcomes. Ask whether the reported findings concern humans, animals or cultured cells, and whether the proposed application matches the validation setting.

An unexplained validation step is a reason to limit confidence, not proof that a model is invalid. Explore the broader Research Literacy collection.