Why is Data Validation Important in Maintaining Accuracy

Alberta Air Quality Monitoring Directive notes

It also outlines data validation best practices, how to use validation codes, keep logs, check for completeness and gives you best practices to improve the accuracy and reliability of your data.

You'll find out about software systems that support graphical and tabular displays.  You might also make sure the tools and systems you use align with these recommendations.

Learn how to keep a transparent and defensible audit trail for data validation, including logs and data completeness criteria.  Also, how to identify and fix quality issues such as rapid changes in data, zero-drift in your instruments, and relationships between parameters.  Use this to troubleshoot problems.

Read the entire Chapter 6 of the AMD here.

The basics and a collection of parameters

4.3.5 Adjustments to the baseline

It's possible for continuous air quality instruments to experience zero drift, which changes baseline concentrations over time. Performance checks can confirm this drift. Data affected by analyzer drift can be corrected by subtracting a verified drift value from the data. You can make adjustments automatically or manually.

Maintaining calibration specifications will minimize the need for zero adjustments. A review process should be in place when applying data adjustments based on zero check data. Adjustments between multipoint calibrations shouldn't be based on span check results. Only full calibration adjustments based on reference standards should be upscaled. Reporting ambient concentration values shouldn't be negatively affected by zero baseline adjustments. There's a risk of erroneous zero adjustments with automated zero adjustments.

4.3.6 Relationships between parameters
To maintain the relationship between measured and derived parameters, adjustments should be applied uniformly to all continuous ambient air parameters during validation procedures. For example, if you adjust NO, you should also adjust NOX. It's especially relevant for parameters like NO/NO2/NOX. Commercial NOX analyzers measure NO, and NO2 is derived from NOX minus NO. The precision of relationships between parameters may be affected by rounding or signal noise when they're reported directly from an analyzer. Table 1 shows specific examples of parameter relationships and considerations of why is data validation important.

Table 1 - Here are some validation considerations for specific parameter relationships:
- NO/NO2/NOX: NO and NO2 should add up to NOX. Data adjustments must preserve the relationship between these parameters.
- CH4/NMHC/THC: Methane (CH4) and non-methane hydrocarbons (NMHC) should equal total hydrocarbons (THC). It's important to preserve the relationship between these parameters when making data adjustments.
- If PM10 flow is split for PM2.5 measurement, PM10 is calculated as the sum of PM2.5 and coarse particles.
- Vector wind speed (VWS), vector wind direction (VWD), and standard deviation of wind direction (SDWD) should all be invalid during the same time period. It should also invalidate VWS, VWD, and SDWD if scalar parameters like scalar wind speed (SWS) or scalar wind direction (SWD) are invalidated.
- VWS and SWS: SWS can be equal to or greater than VWS.
- The delta temperature is the difference between two temperatures. Delta temperature is invalid if either level is invalid.

4.3.7 Adjustments below zero
The person responsible must adjust continuous ambient hourly averages for all valid negative gas and particulate concentrations to zero before reporting them (DQ 4-J). A valid -1 ppb ozone reading should be reported as 0 ppb ozone. Instrument precision and zero noise limitations require this adjustment.

DQ 4-K: Don't apply zero adjustments to sub-hourly intervals before aggregating them into 1-hour averages. Negative values should be adjusted to zero after baseline adjustments. From the Canada-wide National Air Pollution Surveillance (NAPS), Table 2 defines the lower acceptable limits for 1-hour PM2.5 data.

Table 2 - Here are the zero adjustment criteria for different aggregation levels:
- Before aggregating hourly averages for intervals less than one hour and all parameters, all negative values that are valid should remain negative.
- If PM2.5 falls between -3 and 0, it should be adjusted to 0. PM2.5 less than -3 is invalid. Under-zero values should be adjusted to zero for all gases.
- By addressing negative values before aggregation and reporting, these criteria help maintain data quality. PM2.5 criteria are from National Air Pollution Surveillance (NAPS).

4.4 Level 2 - Final Validation
Level 2 - Final Validation involves a thorough review of data to identify any anomalies. Data validity is determined by checking for relationships between dependent and independent data. Here's what you need to know:
- Analyze and investigate anomalous data or outliers collected from validated continuous ambient air data.
- Logs should be kept for decisions regarding anomalous data or outliers, along with justifications.
- Unless there's compelling evidence to the contrary, inconsistent data should be considered valid.
- An investigation of suspect data might reveal instrument malfunctions or other issues.
- Logs should be used to record and justify data validity decisions.

Plotting time series, especially when multiple parameters are displayed together, can reveal relationships that are hard to see in tables. Check these examples:
- Dependent data relationships include anti-correlation between O3 and NO, O3 levels increasing with UV and temperature, and pollutant events affecting multiple parameters.
- By showing consistency across large geographic areas, independent data sets can be used to validate meteorological data.
- By evaluating both dependent and independent data relationships and conducting thorough investigations when anomalies or outliers are detected, Level 2 validation ensures the quality and consistency of data.

4.5 Level 3 - Independent Assessment
Independent Assessment is a final review of validated data by someone who isn't involved in field operations or primary data validation. This review is to make sure data gets an independent quality assurance review before submission. The key points are:
- Submitted data should be reviewed by an independent person who isn't involved in field operations or primary data validation.
- As per the AMD Reporting Chapter, the person responsible should record and report certification statements from the independent reviewer.
- To evaluate data based on expected and historical behavior, the independent reviewer should have some knowledge of pollutant and meteorological behavior.
- Time series plots should be used for manual review of validated data reduced to hourly averages.
- Any suspect data can be communicated to the data validator for investigation, data validation modifications, or justification such as why is data validation important.

All reports (monthly/annual) and data must be certified, as described in the AMD's Reporting Chapter. Validation at Level 3 ensures the data undergoes a final, independent quality check to ensure data reliability.

4.6 Procedures for Post-Final Validation - Annual Validation
Annual Validation - This is a step to re-evaluate data after the initial validation to catch any errors or omissions. Here's what you need to know:
- Data validation may contain errors or omissions despite our best efforts.
- Annual data reviews help identify issues or patterns that may not have been obvious on a monthly basis.
- Before submitting an annual report to the regulator, the person responsible should review all validated data for the previous calendar year.

Review includes annual charts and basic statistics, including comparisons to historical mean, maximum, and minimum values. Independent review certification (DQ 4-P) should indicate who conducted this annual validation review.

As outlined in the Reporting Chapter of the AMD, annual reports will include a report certification form to confirm this review has been completed. Before submitting annual reports to the regulator, the validation review serves as an additional quality assurance step.