We’re excited to share updates to our data quality flagging system, driven by improvements to the underlying algorithm. These updates have resulted in minor adjustments to quality flags, primarily affecting plateau flags (D10).

As part of this update, the entire database has been re-flagged using the new algorithm. Additionally, GLDAS flags (D03, D05) have been added for the period from September 2020 to August 2024 in our near-realtime networks: FMI, WEGENERNET, USCRN, SCAN, SNOTEL, ARM, RSMN, TERENO, SOILSCAPE, and CW3E. Updates have also been applied to 13 regular networks: COSMOS, FR_Aqui, HOAL, iRON, LABFLUX, LAB-net, OZNET, REMEDHUS, Ru_CFR, STEMS, TAHMO, VDS, and WATERLINE.

The updated flagging algorithm is based on the publicly available flagit algorithm, which allowed us to take advantage of improvements to underlying Python packages such as numpy and pandas. Importantly, the new algorithm uses the same core logic as the previous one, but during testing and comparison, we identified three minor bugs in the old implementation (partially raised due to outdated packages). After careful evaluation, we concluded that the new algorithm is more accurate and correct than its predecessor.

We believe these updates will enhance the reliability of our data quality flags. Should you have any questions or require further clarification, please do not hesitate to contact us at ismn@bafg.de.

A special thanks goes to Daniel Aberer and his colleagues at TU Wien that has kindly provided us with the updated algorithm which was developed in the FRM4SM project.