Are Data Quality Measures Meaningful?

Dec 16, 2024

Crime statistics are in the eye of the beholder.  Politicians love to quote them.  We the people ignore them.  Why?  Because crime numbers are highly situation specific.  We care about specific crimes by particular individuals in our neighborhoods, and are unmoved by stale averages across counties, countries or centuries.  Data quality is like that. Data users could care less about mean accuracy, completeness or timeliness reported periodically at an enterprise level. We want to know the quality of our data-sometimes for every record and variable-in real-time as it arrives for our use.  This is not an outlandish expectation.  If we are expected to deliver powerful and reliable insights from our analysis, accurate data quality measures should be part of our bill of rights.  This may require a sea-change in prevailing paradigms for managing data quality.  But for intelligence to move beyond the artificial, the change is essential.