As anyone working in the water world knows all too well, water(sheds) have little regard for political boundaries. Water(sheds) and the myriad of issues that pervade them rarely restrict themselves to just one jurisdiction. Navigating transboundary water resources and implementing effective management strategies have been a long-standing thorn in the side of water advocates, scientists, legislators, regulators, and the general public. At The Commons, we recognize that water monitoring data operates similarly to the water(sheds) from which our users collect data: software systems cannot restrict data to just one use or one platform.
In the first-of-its-kind integration, water quality monitoring data stored in Water Reporter by Buffalo Niagara Waterkeeper (BNW), an environmental non-profit located in New York state, has been pushed to the Gordon Foundation’s Great Lakes Data Stream, a web mapping application targeting Canadian water basins, via the Water Reporter’s application programming interface (API). The path to get to this epic integration involved a long series of strategic software development choices, commitments from data aggregation system operators, and a data collector partner (BNW) willing to let us test our API integration capabilities with her data.
Integrating with Gordon Foundation’s Great Lakes Data Stream
DataStream is an online platform for accessing, sharing, and visualizing water quality data. It was developed by The Gordon Foundation and is delivering data in collaboration with regional monitoring networks across Canada. The first DataStream was launched in the MacKenzie Basin in 2016. Gordon Foundation now operates independent applications for four major drainage basins in Canada with another basin’s application in development at the time of this writing. By sharing data on DataStream, monitoring and research groups broaden the audience and impact of their work.
The DataStream development team values, much like our own shop, building approachable solutions for data access, storage, and interoperability. Their rigorous attention to the detail of data models and commitment to open access paved the way for the integration opportunity with Water Reporter.
To test the capabilities of Water Reporter’s beta API, The Commons team provided DataStream with an access token to BNW’s Water Reporter data source. With this information, and referencing the Water Reporter API documentation, the DataStream team was able to complete the integration between Water Reporter and their existing web-mapping application.
The integration started with some setup configuration. First, the DataStream team identified and created nodes based on the BNW parameters in Water Reporter. This started to build the digital bridge so that station and data information managed in Water Reporter could be shared and visualized on the Data Stream web-map application without requiring BNW to do any continuous intermediary data reformatting. BNW also created an account in the DataStream platform so that they could manage and approve information being fed into the application. Next, BNW entered meta-data information into their new DataStream account. This provided the details for Data Stream's information sharing interface with static information not available via the Water Reporter API. BNW staff completed this step with about 30 minutes of effort. Together, everyone then convened to finalize some of the data field mappings during a quick phone call. After completing all of these preparatory steps, DataStream established the integration and flow of data from Water Reporter to their application. Lastly, before any information was displayed on the DataStream map for the first time, BNW reviewed the intended outputs and approved the uploaded data.
The data set managed in Water Reporter by BNW is now fully integrated into DataStream. To the best of our knowledge, this integration of BNW Riverwatch monitoring data into the Canadian Great Lakes DataStream is the first transboundary sharing of community science monitoring data on a publicly accessible platform. We see many opportunities to expand the integration opportunity to more participants of the LEBAF program as we transition the Water Reporter beyond the beta testing phase.
Going forward, BNW will continue uploading their monitoring data into Water Reporter. From Water Reporter the data will be visualized in not only the Water Reporter hosted map on the BNW website but also shared to the regional LEBAF partnership application and DataStream. To assist the BNW team in further marketing of their data, they can share a link to their metadata page in DataStream.
How we got here
Step 1: Build smart software systems.
Water Reporter’s API supports cross-platform data sharing
Small watershed organizations are the driving force in collecting baseline and discrete water quality monitoring data that detail how our actions affect the nation’s waterways. Volunteer and nontraditional monitoring data often provide insight at a high resolution; however, localized protocols and the intent to quantify the unique challenges of each water body often lead to siloed use cases, differing data standards, and investment in technological solutions that are designed, bought, and built with barriers to expanded use. Furthermore, the collection of monitoring data is undertaken in different ways, with non-standard and non-digital entry formats, resulting in repositories that are not easily findable, accessible, interoperable, or reusable (FAIR).
The Water Reporter data model trains users to streamline their monitoring data into processes that are digitally acceptable and formats that are machine-readable. Our monitoring groups can focus energy on collecting data while Water Reporter does the heavy lifting of managing that data through FAIR data principles. Once data is managed in Water Reporter, the platform creates easy ways to leverage data such as powering embeddable maps, porting data to third-party efforts such as state and federal repositories, and powering audience-engagement applications.
“Good data management is not a goal in itself, but rather is the key conduit leading to knowledge discovery and innovation, and to subsequent data and knowledge integration and reuse by the community after the data publication process” - FAIR data principles
Water Reporter has adhered to an internal integration strategy that supports data collectors in meeting the technical requirements to map to these available and to-be-designed software solutions and third-party systems. When you have the technical requirements in place, FAIR data management facilitates adoption and infrastructure expansion for our users with existing third-party data aggregation platforms.
The strategy has manifested itself in multiple ways. First, Water Reporter creates a system that easily channels data collected streamside into structured, machine-readable formats. We’re fast closing the book on the days of paper collection sheets left to yellow in folders! Second, Water Reporter offers a public (beta) API that encourages users to conveniently share data out to new and existing applications without having to manage multiple versions of the same dataset, expend excessive hours on re-formatting data sets, or remaining beholden to sharing data to a single third party aggregator.
Step 2: Create strong partnerships with like-minded stakeholders
The Commons has supported regional efforts, such as the LEBAF Champions, with data management
We found an excellent and willing partner in planning out multi-use data opportunities: Cleveland Water Alliance and their Lake Erie Baseline Assessment Framework (LEBAF) effort. LEBAF has produced, a cohesive set of standards for operating procedures, data gathering, and necessary technology which elevates the credibility and capacity of volunteer science.
Since 2020, Lake Erie Baseline Assessment Framework (LEBAF) Champions have used Water Reporter to structure their monitoring data in machine-readable formats. Of the twelve inaugural champion groups, Buffalo Niagara Waterkeeper had a leg up in the effort because their RiverWatch chemical monitoring program had been managing and visualizing data with Water Reporter since 2018.
As each Champion on-boarded their own monitoring program into LEBAF, they’ve been able to showcase their monitoring data both on individual Water Reporter generated monitoring maps and a web-mapping application created by The Commons specifically for Cleveland Water Alliance. LEBAF successfully uses Water Reporter to support the data management of disparate monitoring programs. The partners wondered, where else could we share this data? We were primed to continue to the next critical step in the future of data sharing: cross-platform integrations.
Cleveland Water Alliance shared this vision of getting data that they supported in collecting into as many data portals and visualizations as possible. These efforts take time and resources and CWA has been instrumental in funding the development and documentation of the Water Reporter public API.
The Great Lakes One Water, original supporters and funders of Lake Erie Basin Assessment Framework, was integral in seeing the vision of taking data of known quality collected from the Champions and, through an API, sharing it with additional data aggregators. They identified the opportunity with Great Lakes DataStream and made the critical introductions necessary to get the effort moving forward. BNW, a veteran Water Reporter user and LEBAF participant allowed their monitoring data to be used to test the integration.
Step 3. Build public APIs and clear documentation
A connected network of federated databases has been envisioned by stakeholders from the water data community for years because it expands opportunities for community monitoring data to be incorporated into both decision-making and information-sharing opportunities. The LEBAF pilot project demonstrated that participants could manage their own siloed programs while also contributing to the larger regional effort. However, no partner wanted participation in the LEBAF project to preclude additional data sharing or increase resources spent on inadequate data management. As previously stated, good data management is not an end goal but rather a gateway to amplify data use. Getting the public API to work pushing data through different nodes was the next step in actualizing this goal.
Step 4. Expand Use
We intend to continue opening up data opportunities through the Water Reporter API. Our first step will be to expand the partnership between Gordon Foundation's Great Lakes DataStream and the LEBAF Champions. For each group that wants to participate, the DataStream team will walk through the data source connection and the point of contact will complete the meta-data input exercise. The information will then appear on the DataStream map, the Cleveland Water Alliance LEBAF Map, and in any Water Reporter hosted visualizations like maps or station cards.
APIs can facilitate participation in multiple applications. In the water sector, APIs let datasets build bridges and amplify impact of data. In other words, APIs can break down barriers to use and assist in multi-stakeholder decision-making through efficient use of data. Since we cannot remove political boundaries that complicate water resource management, we might as well create software systems that meet data collectors streamside and facilitate data sharing.
