Evidlake
Menu

Data Snapshot

Also known as: snapshot, periodic snapshot.

A complete record of a list or table as it stood at one moment, kept as a unit with its time, scope and completeness, so later versions can be compared against it.

Updated Last reviewed

Synthetic dataExamples on this page use synthetic data created for illustration.

Definition

A data snapshot is a complete record of a list or table as it stood at one moment, kept as a unit together with its time and scope. Comparing snapshots shows what changed, and a snapshot that also records its own completeness is never mistaken for a full one when it is partial.

In plain terms

A table that is updated in place only knows the present. A ranking that changes every hour, a price list, a set of members: if you keep only the latest, you cannot answer “who was on it last Tuesday?”. Taking a snapshot on a schedule, with the same conditions each time, preserves the membership and the order for every moment.

A snapshot should say what it covers (which list, under which conditions), when it was taken and whether it is complete, partial or failed. Without the completeness flag, a half-captured list looks like a list that shrank.

Example

A “top 5” list, captured daily:

DayMembers, in orderStatusWhat can be concluded
MonA, B, C, D, ECompleteBaseline
TueA, C, B, F, GCompleteD and E left the list; F and G entered
WedA, C, B (second page failed)PartialNothing about places 4 and 5; F and G may still be on it

Treating Wednesday as complete would wrongly report that F and G left the list.

Common misunderstandings

  • “A snapshot is just a backup.” A backup restores a system; a snapshot is an analysable record of a state at a time.
  • “Missing from this snapshot means gone.” Only if the snapshot is complete, and absence from a list is not the same as being discontinued.
  • “Changing the capture conditions is harmless.” A different time, region or language can change the list itself; start a new scope version instead of comparing across the change.
  • Pagination

    Splitting a long list into pages fetched one at a time. How you page decides whether items can be skipped or repeated when the list changes.

  • Sampling Frame

    The list or source a sample is actually drawn from. Whatever the frame leaves out cannot appear in the sample, however large the sample is.

  • Survivorship Bias

    A selection error from studying only what survived a process and ignoring what dropped out along the way.

  • Bitemporal Data

    Data that records two times for every fact: when it was true in the world, and when the system learned it.

References

  1. Ralph Kimball, Margy Ross. The Data Warehouse Toolkit . Wiley, 2013. ISBN 978-1118530801.
  2. Martin Kleppmann. Designing Data-Intensive Applications . O'Reilly Media, 2017. ISBN 978-1449373320.