Evidlake
Menu

Walk-forward Validation

Also known as: rolling-origin evaluation, time series cross-validation, expanding window.

Evaluating a forecasting method by training on data up to a date, testing on the period just after, then moving the date forward and repeating.

Updated Last reviewed

Synthetic dataExamples on this page use synthetic data created for illustration.

Not adviceThis is educational material. It is not investment, business or other professional advice.

Definition

Walk-forward validation evaluates a forecasting method by repeatedly training on data up to a cut-off date, testing on the period just after it, then moving the cut-off forward. Every test uses only the past, exactly as the method would in real use.

In plain terms

It is cross-validation adapted to time. The training window can expand (everything up to the cut-off) or slide (only the latest stretch). Each step produces an honest out-of-sample score, and the sequence of scores shows whether performance is stable or deteriorating.

When a label takes time to become known, for example “did the price rise over the next 30 days?”, the last training examples overlap the test period; a gap or Purging and Embargo removes the overlap.

Example

Monthly data for a year, one-month-ahead forecasts, expanding window:

FoldTrained onTested onError (MAE)
1Jan–JunJul4.1
2Jan–JulAug3.8
3Jan–AugSep5.2
4Jan–SepOct4.4
5Jan–OctNov4.0

The average error is 4.3, and the spread across folds (3.8 to 5.2) is part of the answer. Shuffled folds would let the model see August when predicting July, which no real forecaster can do.

Common misunderstandings

  • “Shuffled k-fold is fine for forecasting.” It lets the future inform the past.
  • “The last fold tells the whole story.” One period can be unusually easy or hard; look at all of them.
  • “The window length does not matter.” Expanding and sliding windows answer different questions about how fast conditions change.
  • Cross-validation

    Estimating how well a method will do on new data by repeatedly leaving part of the data out, training on the rest, and rotating.

  • Backtesting

    Evaluating a rule or model by replaying it on historical data as if it had been used at the time. Informative only if the replay is honest.

  • Purging and Embargo

    Removing training observations whose time span overlaps the test period, and skipping a short stretch after it, so no information leaks between the two.

References

  1. Rob J. Hyndman, George Athanasopoulos. Forecasting: Principles and Practice, section 5.10: Time series cross-validation . OTexts, 2021.
  2. Christoph Bergmeir, José M. Benítez. On the Use of Cross-validation for Time Series Predictor Evaluation . Information Sciences, 2012. DOI: 10.1016/j.ins.2011.12.028