Goodness of Measurements
Broadly speaking, there are two criteria used to evaluate a quantitative measurement. First is accuracy (how close the measured value is to the true) and second is the variability (how much will this measurement vary if performed again under the same conditions). These definitions (accuracy = closeness to true; variability = spread or dispersion) are informal but well understood and sufficient for this overview. Both criteria are associated with a measurement, but they are not mutually dependent. It is possible to have an accurate measurement with high variability or an inaccurate measurement with low variability. The range (estimated measurement uncertainty) associated with a quantitative measurement is often small, but there is a range.
To illustrate the concepts of interest here, think about a dartboard and throwing darts. The center of the dartboard represents the true value of some measured quantity such as a blood alcohol concentration. If a person hits the middle of the bull’s-eye, the measurement is as accurate as possible; the measured value is the same as the true value. Even an expert dart thrower will not hit this bull’s-eye every time a dart is thrown although all would be expected to cluster relatively close to the bull’s-eye. Here, throwing multiple darts is analogous to making the same measurement under the same conditions. There is always a spread of darts—very small for a good experienced player, and big for a novice—but there is always a spread. Estimating the spread of the darts represents to an uncertainty estimation.
In laboratory terms, learning to throw darts is analogous to method development and validation process. The person throwing the darts needs to be trained and learn to follow a standard method of throwing before it makes any sense to rely on this person being acceptably accurate with minimal variation. Once the process is standardized and the person is trained, a set of historical data could be collected to establish increasingly better estimates of the variability. At some point, it will be possible to define a region around the center of the target to which a probability of landing a dart can be stated probabilistically. If a person is well-trained and has demonstrated prowess over time, it may be possible to state that there is a 95% probability that any dart thrown under the exact same conditions will land within У2 of an inch of the bull’s-eye. This also means that there is a 5% chance that it will fall outside this У2 inch range. This range (measurement uncertainty estimate) is quantitative and relies on tangible data; the same is true for an uncertainty estimate that accompanies a forensic measurement. While it is natural to have variation and uncertainty, it is something that scientists must be prepared to address to assure those outside of the profession that it is normal and not a matter of misconduct.