Data Contract

A written, versioned agreement between the producer of a dataset and the people who use it, covering structure, meaning, timing and quality.

Synthetic dataExamples on this page use synthetic data created for illustration.

Definition

A data contract is an explicit, versioned agreement between whoever produces a dataset and whoever uses it. It states the fields, their types and meaning, the time semantics, the quality expectations and how changes will be announced, so that consumers are not surprised.

In plain terms

Without a contract, a dataset is whatever it happens to look like today. A producer renames a column or changes a unit, and a report downstream quietly breaks or, worse, quietly goes wrong. A contract moves that agreement out of people’s heads and into a file that can be reviewed, versioned and checked automatically.

A contract is wider than a schema. A schema says a field is a number; a contract also says in which unit, what a missing value means, how fresh the data is promised to be and who to ask.

Example

A small, invented contract for a daily price file:

dataset: daily_prices
version: 2
owner: data-team
fields:
  - name: item_id
    type: string
    meaning: Stable identifier of the item; never reused.
  - name: price_minor
    type: integer
    meaning: Price in minor units (cents). Empty means not observed, never zero.
  - name: observed_at
    type: timestamp (UTC)
    meaning: When the price was seen, not when the file was written.
freshness: new file by 06:00 UTC every day
changes: breaking changes need a new major version and 30 days' notice

A consumer can now test each delivery against the file, and the producer knows which changes are safe.

Common misunderstandings

  • “A schema is a contract.” A schema covers structure only. Units, meaning, freshness and change rules are what usually go wrong.
  • “Contracts freeze the data.” They define how change happens (versions, notice periods), not that it never does.
  • “Only large organisations need them.” Any two people who share a dataset have an implicit contract; writing it down is cheap.

References

  1. Open Data Contract Standard . Bitol.
  2. Martin Kleppmann. Designing Data-Intensive Applications . O'Reilly Media, 2017. ISBN 978-1449373320.