Evidlake
Menu

Feature Engineering

Also known as: feature construction, features, derived fields.

Turning raw data into inputs a model or rule can use, such as ratios, rates of change and counts over windows. Each must be computable from what was known at the time.

Updated Last reviewed

Synthetic dataExamples on this page use synthetic data created for illustration.

Not adviceThis is educational material. It is not investment, business or other professional advice.

Definition

Feature engineering is turning raw data into inputs a model or rule can use, such as ratios, rates of change, counts over windows or categories. The choices decide what the model can see, and each feature must be computed only from what was known at the time of the prediction.

In plain terms

Models rarely work well on raw columns. A change in a rating count over 30 days, a price relative to the category median, or the number of reviews per month can say more than the raw numbers. Such derived inputs are features.

Two disciplines matter. Every feature needs an exact definition (window, units, handling of gaps), kept like a versioned metric definition. And every feature must be calculated as of the prediction date; computing it from later data is Look-ahead Bias in disguise.

Example

An item’s rating count was 100 on day 0 and 130 on day 30. Predictions are made on day 25.

FeatureFormulaValueKnowable on day 25?
Absolute change over 30 days130 − 10030No (needs day 30)
Change in the first 25 dayscount(day 25) − 100knownYes
Relative change30 / 10030%No
Rate per day so farchange(day 25) / 25knownYes

The first and third features, although natural to compute from the finished data, would leak the future into any prediction made on day 25.

Common misunderstandings

  • “More features always help.” Redundant or unstable features add noise and invite overfitting.
  • “Any column can be a feature.” Only values knowable at prediction time qualify.
  • “Feature definitions are an implementation detail.” A change in a window or unit changes the results, so definitions deserve versions and documentation.
  • Data Leakage

    Information that would not exist at prediction time finds its way into the data used to build or evaluate a model, so results look better than they can ever be in use.

  • Point-in-Time

    Reading data as it was known at a past moment — not as it looks today after later corrections and additions.

  • Look-ahead Bias

    A study error where information that was not yet available at the time of a decision is used to evaluate that decision.

References

  1. Alice Zheng, Amanda Casari. Feature Engineering for Machine Learning: Principles and Techniques for Data Scientists . O'Reilly Media, 2018. ISBN 978-1491953242.
  2. Max Kuhn, Kjell Johnson. Feature Engineering and Selection: A Practical Approach for Predictive Models . CRC Press (online edition), 2019.