High quality data are accurate, timely, meaningful and complete.
Fitness-for-purpose is achieved if the stated or implied needs, or expectations, of the end-user of the data are met.
The design of a quality management system starts with specification of end-user needs and expectations. These expectations are inter-dependent. Consider the situational irony of the sign stating: “Our data are timely, affordable and accurate – pick any two out of three” that one of my managers in a previous job kept on his door during implementation of GOES telemetry. This sentiment is memorable even decades after the fact. Improvements in accuracy or timeliness usually come at some cost. Affordability may not be a data quality in itself but impacts on the compromises needed in data qualities to ensure data collection programs are sustainable.
The expectations of end-users are collectively described as the ‘standards’ against which data are evaluated. A product standard can be expressed as a performance result (e.g. +/- 5%). A process standard, typically expressed as ‘standard operating procedure’ (SOP), specifies requirements to meet or exceed a product standard. The connection between the SOP and the performance expectation may not be explicitly stated but is inherent in the development of the standard. As expectations change (e.g. for quality controlled data in real-time) the SOPs have to change accordingly.
Hydrometric monitoring programs either develop their own standards documents (e.g. USGS) or adopt international standards (e.g. ISO).
It is one thing to say what standard you follow but it is another thing to demonstrate compliance. There are even standards for conformance to standard. This is typically an internal review-and-approval process that may have additional credibility provided by audited international compliance monitoring standards such as ISO 9001. A key component of compliance monitoring is how non-compliances are dealt with. ISO 9001 uses a Correction and Preventive Action Report (CPAR) to initiate a root cause analysis investigation. Information loss is mitigated even when there has been some fault in the data collection process (Correction). Lessons are learned from non-compliances and are applied to improving the process to avoid future exceptions (Preventive Action).
Deviation from expectation can usually be remedied with improved training. Training addresses: procedural errors (i.e. the intention is correct but the execution is wrong); communication errors (e.g. inadequate field notes); proficiency errors (i.e. lack of knowledge or skill); and operational decision errors (i.e. discretionary decisions that unnecessarily increase the likelihood of a mistake). Compliance monitoring will also uncover intentional non-compliances that result from complacency and disregard for rules; or SOPs that are poorly designed or inappropriate.
My observation of training in the hydrology industry is that some schools or colleges offer introductory courses for hydrometric field and office procedures. Equipment vendors will often supply limited, device-specific, training. There are also a number of on-line hydrometric training resources (e.g. USGS). Readily available hydrometric training is adequate for basic proficiency but is generally inadequate to meet the ISO 9001 intent for adaptive preventive action. It seems that many hydrometric operations lack the resources to self-monitor for adaptation and continuous improvement. Knowledge and experience sharing, through weblogs such as this one, may be the best path forward for these organizations to make the most of their data investment.
What do you think about standards, training and continuous improvement in our industry?