A Dataset Is a Map of Where You Looked

A dataset with ten thousand rows looks like progress. Sample size is the number most often quoted, and it is the easiest one to make large: run the collector longer, poll more often, merge in another source. None of that guarantees the dataset knows more about the system than it did with a tenth as many rows.

What a sample size does not report is coverage — which conditions were actually visited, how often, and under what circumstances. Ten thousand observations of the same regime, at the same time of day, through the same instrument, describe one condition ten thousand times. A hundred observations spread across ten distinct conditions describe more of the system than either version of the larger file.

Statistical intervals assume the observations feeding them are independent draws. Rows collected back to back, by the same instrument, under conditions that have not changed, are not independent in the way the arithmetic assumes — they are closer to one observation measured many times. Treating them as separate evidence understates the uncertainty and overstates how much has actually been learned. The fix is not to distrust repetition; it is to be explicit about what a run of similar rows is evidence of, which is usually stability under one condition rather than confirmation across many.

A single observation is a number attached to almost nothing, unless the record around it survives with it: when it was taken, under what conditions, by which version of which instrument, and what was happening around the collection that might have shaped it. Provenance is not paperwork. It is the difference between being able to ask, later, whether two rows are actually comparable, and having to assume they are.

The harder discipline is recording what the collection did not reach. A sampling process has a shape, and the shape has edges — conditions it under-visits, times it is rarely running, populations it structurally excludes. Those absences do not show up in a table of what was collected; they only show up if someone writes them down on purpose. A dataset that states its blind spots is more trustworthy than one that simply does not mention them, not less.

The more honest measure of progress is a map: the conditions worth distinguishing, laid out against the times, instruments, and contexts observation could occur in, with each cell marked by how much has actually landed there. Growth in row count means little until it is placed on that map — a dataset can double in size while adding rows almost entirely to cells that were already full, in which case the useful information in it barely moved.

Keeping that map in view supports two disciplines that are easy to skip otherwise:

1. Set a stopping rule before collection begins — a stated point at which the map counts as full enough for the question at hand — rather than discovering one once the pattern starts to feel familiar. A rule chosen after the data already look finished is not a rule; it is the data flattering itself.
2. Treat an unfilled cell as a fact about the dataset, not a gap to quietly interpolate past. An absence that is recorded can be reasoned about; an absence that is smoothed over cannot.

None of this produces a finding. A coverage matrix is a report on how well a question has been asked, not an answer to it. We keep the distinction deliberately sharp: collecting evidence is a claim about a process, and claiming a finding is a claim about a system. Confusing the two is how a well-instrumented dataset ends up supporting a conclusion it never actually covered.