Aquarius HydroCorrect FAQ: Answered by Aquarius Product Manager
In 2023, Aquatic Informatics™ announced the launch of HydroCorrect, a new automated data validation tool that can power proactive monitoring and management of groundwater, flooding, and water quality in the Aquarius™ platform.
Trusted Automation, Estimation, and Correction
HydroCorrect enables organizations to easily transition to automation by providing support and giving operators full control. Trusted automation provides flexibility through user approved actions until operators are ready to fully automate any action as well as visibility with the audit trail and easy to understand rule creation and anomaly detection. Estimations and corrections use high-accuracy, in-situ, field-visit data to adjust the continuous time series data, creating full confidence in the abilities of the analysis.
Reduce Time and Resources Spent on QA/QC with HydroCorrect
Organizations invest a significant amount of time in the QA/QC process, with one customer estimating “approximately 210 hours” — or 26 business days — spent each year, on average, on data validation alone. HydroCorrect reduces mundane work and enables skilled workers to perform higher-value and more engaging tasks that can have an impact on making better data driven decisions. Using machine-learning technology validates water data from numerous sources effortlessly, and the suggestion-based approach means the user always stays in control while saving hours of time on manual QA/QC processes. Existing Aquarius users can implement HydroCorrect to see the benefits of automating their QA/QC processes.
HydroCorrect transforms data quality assurance and quality control (QA/QC) processes using machine-learning to automate and standardize workflows, detect data anomalies, ensure regulatory compliance, and enforce data standards organization-wide, leading to an exponential reduction in time and resources spent on the QA/QC process. The tool provides immediate recommendations on how to respond to environmental events and influence long-term decisions while ensuring the data that drives those suggestions is defensible, reliable, and timely.
Ask the Expert: Symone Kumar
We’re excited to announce a new content series: Ask the Expert. We’re collected the most Frequently Asked Questions about our products and data management in general, and our inviting our team to provide answers, starting with Aquarius module, HydroCorrect. This blog is a transcript of an FAQ video that’s available publicly. If you’re rather watch the video, click on the image to the left.
1. Is HydroCorrect a supplemental product or is it included with the base Aquarius license?
Now, if you don't know, HydroCorrect is our QA QC Automation product, which incorporates some predictive analytics capabilities to expedite the QA/QC processes for your environmental/water data.
And this product is actually a supplemental product, so not everyone has those high-level QA/QC needs.
It's something that you should reach out to your account executive that you would be working with to understand how you can get that added into the suite of Aquarius products that you're using today.
2. Are corrections automatically applied in HydroCorrect?
This is a common question when it comes to HydroCorrect, and the answer is that there are options for both scenarios.
Automatic Corrections: HydroCorrect provides a feature known as automatic corrections. This functionality allows the software to automatically apply corrections based on patterns it recognizes in the data. For instance, if you become familiar with certain types of data that consistently exhibit similar issues, such as gap fills or flat lines, you can opt for an automated process. HydroCorrect will handle the acceptance of these corrections on your behalf.
Approval Process: Alternatively, you have the option to go through an approval step. In this case, HydroCorrect will conduct a preliminary assessment and correction of the data, creating a backlog of screened data. You will receive a notification, alerting you to this queue. To finalize the corrections and make them live, you need to manually review and approve each one. This approach provides a more hands-on, controlled method of ensuring the accuracy of your data.
Considerations:
- Pattern Recognition: Automatic corrections are effective when dealing with data patterns that are easily identifiable.
- Controlled Review: The approval process allows for a manual review, ensuring a more controlled and deliberate correction of data issues.
In conclusion, HydroCorrect offers flexibility in its correction processes. Whether you prefer the efficiency of automatic corrections or the precision of a manual approval step, the choice is yours. Understanding these options empowers you to tailor your data correction workflow according to your specific needs.
Remember, the key is to strike a balance between automation and control, ensuring the integrity of your data in every step of the correction process.
3. How do you view multiple time series and apply qualifiers to all of them at the same time?
This individual, working in water quality, doesn't use ratings but aims to qualify data for multiple parameters simultaneously at the same locations. Currently, when working on a location-by-location basis, users can configure parameters needed for measurement. The ongoing roadmap includes an expansion to select and add parameters for multiple locations simultaneously, particularly when dealing with similar data types so everything can be done in bulk.
This enhancement aims to streamline processes during location onboarding, allowing users to set up everything in bulk. A significant focus for the upcoming year is to implement more efficient updates to the product. This will enable users to make backend changes to data input and analysis seamlessly, whether for a single location, one time series, or multiple locations simultaneously.
4. Is HydroCorrect designed for water quality data, pH, dissolved oxygen, etc.?
Yes, you can use HydroCorrect for water quality data. In fact, we love that because water quality data has so many parameters, and a lot of them can be really complex. We built HydroCorrect with that complexity in mind. So absolutely bring on the water quality data parameters.
To paraphrase, the question is, “If there are gaps in the data due to following of the instrument or if there's a shift in the data due to calibration, does HydroCorrect recognize that in a way that makes sense?”
A person who is correcting data would say, "Oh, I know that's because that sensor has been covered up by debris," or "I know that calibration check, that's the reason why."
That's actually a part of the functionality of HydroCorrect - to be able to make those similar types of assessments and adjustments for situations like following and calibration, as opposed to other types of reasons for having data that's going to be atypical. It's also taking that into account.
Continue Learning
5. How can we customize HydroCorrect to work for custom types of variables?
Right now, we have a certain number of features and rules that can be applied to HydroCorrect. There are certain conditions that the correction tool works for, but we're always expanding that because it's a product that we're actively building on.
If you have a variable that's not part of what I'm about to explain in the upcoming section, please reach out to me or to the engineering team if you have a contact there, and we can help learn more about what your needs are. We're always open to understanding that, so if that's a need for a lot of customers, we're happy to build that. As far as the features and functionality for the rules that can be applied in HydroCorrect:
- Level Threshold: Detecting values above or below a defined max or min.
- Rate of Change Threshold: Detects values that are increasing or decreasing faster or slower per second than a defined max or min.
- Flat Line Rules: Detects when data stops changing over time.
- Sensor Reset: Applies drift correction to correct time series data between two field visits due to a fouled sensor left in situ.
- Auto Encoder: A machine learning-based approach with data trained on surface water temperature time series.
- Field Visit Corrections: Related to calibration and field visit aspects.
These are the core rules that can be applied with HydroCorrect today. If you want to learn more about other rules, you can reach out to us to get more information.
6. What are some good examples of corrected time series data that HydroCorrect can help solve?
I think what's important to note is that not only can you apply rules to solve specific problems, but we're also trying to address the broader issue of identifying errors in the data due to issues unrelated to water events.
This includes technical malfunctions that make the data unreliable and untrustworthy. Additionally, HydroCorrect is designed to identify real events, such as rare but sudden spikes in turbidity associated with unexpected high-flow events.
These are problems that can be equally important when making resource management decisions that affect both human and environmental conditions. So those are some of the types of time series data that HydroCorrect can help solve.
In addition to that, it's essential to understand the data processing behind HydroCorrect. The raw data of a time series undergoes processing on a set schedule multiple times per day in the HydroCorrect system. This means you don't have to wait a full day for corrections. The script behind the engine of HydroCorrect runs periodically, allowing you to check and refresh data throughout the day, providing real-time insights into your time series.
In summary, HydroCorrect is not only solving specific issues with time series data but also ensuring that the processing is efficient and provides real-time updates for a more comprehensive understanding of your data.
7. What are some good examples of corrected time series-data that HydroCorrect can help solve?
I think what's important to note is that not only can you apply those rules, but we're trying to solve the problem of identifying errors in the data due to issues unrelated to water events, such as technical malfunctions that make the data unreliable and untrustworthy.
We're also focusing on identifying real events, including things like rare but sudden spikes in turbidity associated with sudden high flow events.
These are problems that can be equally important when making resource management decisions that affect human welfare and environmental conditions. So those are some of the types of time series data that HydroCorrect would help to solve.
In addition to that, something to know about the data processing behind HydroCorrect: the raw data of a time series will be processed on a set schedule several times per day in the HydroCorrect system.
This means you don't have to wait a full day. We're running a script behind the engine that is HydroCorrect on a basis where you can check, and you'll have refreshed data throughout the day so that you can have real-time understanding of what's going on with your time series, and then suggested corrections.
Find Your Partner in Change Management
As utilities progress on their digital transformation journey, one of the main challenges they face is resistance to change.
Technology partners can help organizations guide their change management strategies to ensure the success of any new technology projects. While every water organization is unique, many of the challenges are similar: legacy systems, data silos, lack of resources, etc.
Experienced technology partners like Aquatic Informatics have seen it all before. We can help you achieve streamlined quality control for your environmental data. From source water through to the receiving environment, our interconnected data management platforms can help you drive the efficient management of water information across the water cycle with minimal disruption to your operations.
Get in touch to discuss your water data projects, check out our upcoming product demo, or schedule a personalized demo of our platforms.
