Local organizations and volunteer groups spread out across the Great Lakes region shoulder the responsibility of monitoring water quality in rivers, creeks, and streams. These community science groups historically arose in response to hyper-local management needs to collect data to inform their localized questions and concerns. For years, these groups have primarily shared their results with local stakeholders to meet local management needs. While these local activities can have a lasting impact on regional water quality, lack of standardized methods and protocols between groups results in a fragmented data landscape unable to provide a regional understanding of the state of water quality. However with the right digital and partnership infrastructure in place, regional efforts can overcome the limitations of operating on a watershed-by-watershed basis to create stronger monitoring programs that lead to improved water quality. Cleveland Water Alliance, a non-profit economic development organization focused on water technology innovation, began the Smart Citizen Science Initiative to unlock this regional potential. The result is a network of local volunteer programs that support and accelerate the collection of Lake Erie Basin data and serves as a platform for the development of data standards and ongoing community science innovation.
“The Smart Citizen Science Initiative is helping Lake Erie Basin communities benefit from our region’s remarkable natural resources and water technology assets. CWA and our collaborators are using technical innovation, grassroots capacity building, and regional standards development to accelerate the collection and use of community water data across the region,” explained Max Herzog, Program Manager at Cleveland Water Alliance.
In Spring 2020, the Cleveland Water Alliance and The Commons launched phase one of a three year partnership made possible by the support of Great Lakes One Water, a collaboration of community foundations committed to solving Great Lakes water challenges including Ralph C. Wilson Jr. Foundation, Community Foundation for Greater Buffalo, Cleveland Foundation, Greater Toledo Community Foundation, Community Foundation for Southeast Michigan, and Great Lakes Protection Fund. The Smart Citizen Science Initiative (SCSI) coordinates a core network of water quality monitoring groups (dubbed “Local Champions”) across the Lake Erie basin. This collaborative has committed to testing new community monitoring technologies and adopting data management strategies intended to streamline data collection and storage while simultaneously democratizing data through regional visualizations. Faced with a demand for a common data platform tailored to volunteer monitoring, SCSI teamed up with The Commons to integrate Water Reporter into existing monitoring programs with an end-to-end data management system that results in exportable, portable data as well as interactive mapping applications.
The original twelve Champions hailed from three states with waterways flowing into Lake Erie: Michigan, Ohio, and New York, and represented non-profit organizations as well as academic and government institutions. While all of the participating Champions already managed robust water quality monitoring programs, no standard methods for monitoring connected these groups. Nonetheless, the initiative identified critical areas were standardization could unite these organizations and collectively support broader dissemination of water quality data.
CWA and The Commons set out clear expectations for the first year that intended to meet the Champions where they were rather than forcing new protocols and data management procedures on these programs that already had established protocols and procedures that fit their localized needs. The commonalities across the programs that SCSI could establish included:
- Where to store data
- How to structure data
- Completion data for cleaning up the annual data
By addressing some data management principles prior to tackling monitoring protocol updates, CWA could begin to demonstrate water health on a regional level through an application that visualized all of the data collected by the Champions’ monitoring programs despite variability in current monitoring protocols. Ultimately, The Commons team used the collected data set from Champions to build a widget to visualize community water quality data at the regional scale.
Building this widget for the Smart Citizen Science Initiative took an investment in strategic training, on-boarding, and structuring of data for each Champion program and across the Initiative.
Applying Principles of Data Management
Before any data exchanged hands or volunteers recorded readings from 2020 water samples, The Commons sat down with each Champion for an in-depth analysis on each program’s current data management practices and hosted a training to key data management principles they can incorporate into everyday practices and procedures. Along with talking in generalities about the data management goals and principles to guide their programs, The Commons introduced Water Reporter to the Champions and set up their subscription accounts. Water Reporter’s built-in data management features help keep groups grounded in best management practices throughout the life cycle of their programs.
At the end of each training, the Champions shared their historical data with The Commons for restructuring. While no Champion had to undertake requisite changes to their data management workflows, Champions are required to store all of their water quality sampling data in Water Reporter. Not only does Water Reporter help standardize approaches to data collection and storage but it also streamlines collaborative regional analysis without sacrificing the localized distribution of data results.
Standardizing Data Via One System
All Champions committed to storing their water quality monitoring data within Water Reporter. Each Champion was given a subscription account to Water Reporter. Within this subscription the Champions agreed to upload their monitoring data from the 2020 season. Champions were invited to use the additional features if they so choose to collect observational data or support additional monitoring programs. Furthermore, in order to take advantage of all of the data already in existence, the Water Reporter team worked with each Champion to restructure and upload all of their historical monitoring data. Finally, digital collection forms as well as .csv upload templates were provided so that each Champion could easily port data from the current monitoring season into Water Reporter.
Supplying Thresholds and Indicators
Along with different protocols used and parameters collected, Champions currently use different systems for indicating water quality health. The Water Reporter Team worked with each Champion to enter in parameter-specific ranges of indicators so that the language matched the Champion’s code. By tying these thresholds and indicators to specific parameters, Champions can build a quick data visualization map to then embed in their website that communicates their water quality health to their local audiences.
Unifying Data Across Champions
Ultimately, CWA wants to elevate Champions both individually and collectively to achieve a regional view of water quality and open up the window to explore new opportunities to strategically invest in tackling sources of water quality stress. To achieve this regional view, The Commons team collected all of the Champion data managed within Water Reporter and built an application that aggregated all monitoring data into one regional water quality monitoring dashboard. The dashboard application collected all of the data contributed from Champions and displayed the location and monitoring data through an interactive map. Via station cards, viewers can explore data trends or select a range of data to export and analyze outside of the application. Truly, the end results offer a completely accessible, democratized data repository that can fit the needs of a wide-range of stakeholders and their long term water quality goals.
The Commons’s Widget Building Process
The Commons’s data analyst and front end software engineer built and designed the Smart Citizen Science Initiative water quality monitoring results application on the popular R-Shiny framework. This framework makes it relatively easy for our team to build and deploy interactive web applications straight from R-hosted databases. In particular, R-Shiny combines the computational power of R, a language and environment for statistical computing and graphics, with the interactivity of a website. Our team uses R because this environment allows us to more efficiently deliver the end application desired by a client without having to invest in building new widgets and systems from scratch each time.
The Smart Citizen Science Initiative widget marks the first time that The Commons has aggregated water quality monitoring data collected from multiple separate groups on one interactive map. Collectively, audiences visiting the widget can see a ten thousand foot view of where monitoring takes place and zoom in to the data collected by a particular organization. Users can compare water quality parameters, such as dissolved oxygen or nitrate, from different groups and stations, allowing for interesting comparison opportunities typically not feasible in Water Reporter. Not only does this approach allow users to look at the existing data but it allows stakeholders with the means to expand or begin new monitoring programs to highlight geographic gaps in collection.
In order to create a collaborative widget, The Commons embarked on aligning expectations, developed the widgets, and delivered iterative updates to the CWA team for review. In particular, having all Champions organize and store their data in Water Reporter significantly shortened the requisite time to clean up and restructure data.
As a first step, The Commons always scopes out the project goals with our client. Before any development begins, communicating goals and aligning on expectations sets the project up for success.
Second, we review the data delivered by the client to include in the application. In this project, we had most data already on hand thanks to the long-term collaboration between The Commons and CWA through the SCSI program. Thanks to the previous investment in time with each Champion, our team could more quickly port the data to the R-Shiny platform. At the current time, we export the data from Water Reporter and host it on a web-based spreadsheet. In order to more conveniently link the web application and the data collected from streams, CWA and The Commons are working to open the API — described further below.
Third, our team built the back end of the R-Shiny application. This meticulous process critically aligns all of the data, application components, and front-end features intended for display in the final widget. In other words, our team takes all of the initial requests made by the client and turns those ideas and visions into an online application. Iteratively, we demonstrated progress to CWA to confirm progress met their expectations.
Finally, with the greenlight from CWA, we entered the user interface build phase of the project. Here, our front end developer gets to showcase his UI skills by taking all of the features and functionality built in R-Shiny to build a branded, user-friendly wrapper. We optimize the wrapper to behave well for both mobile and desktop devices.
One important shortcoming of the R shiny applications compared to Water Reporter’s out of the box maps, is the ability to dynamically connect to user submitted data. Currently, in order to ingest Water Reporter data into R Shiny applications, the data needs to be downloaded from each group, cleaned, and then stored with the application. In practice, this means that adding new data to the application can only happen periodically, say once a month, or once a season. Ideally, the application would communicate directly with Water Reporter ensuring the most up to date information is available for the public. To address this issue, the Commons is working diligently to make the Water Reporter API publicly accessible to further improve the functionality and usefulness of our R Shiny applications. With a public API, the data that feeds the application will refresh every time it loads. Additionally, new information like additional parameters, stations, or even entire groups can be quickly added with minimal to no code changes. This not only dramatically improves performance and scalability, but also opens the door to new application
Connecting Water Reporter to R-Shiny
The R-Shiny platform has been built to accept data from third-party programs through their own application programming interface. Typical examples are systems like Amazon Web Services (AWS), or Google Cloud. These allow for dynamic data connections but present extra complexity. Instead, CWA and The Commons are investing in creating our own nodes in Water Reporter that enable the data sources built by Champions to be directly connected to the R-Shiny application, creating a real-time pulse on water quality monitoring across the Lake Erie basin. The work necessary to fully open the Water Reporter API is underway at the time of this writing and is expected to near completion by the end of the third quarter of 2021. Once completed, the Smart Citizen Science Initiative and similar regional efforts can collect and store their data in Water Reporter while preparing third party uses for the data such as R-Shiny applications or porting data to data ingestors like EPA’s WQX or state data portals. Until the API is complete, all users can export their machine-readable data from Water Reporter and restructure it to meet any data aggregator requirements.
In summary, a goal of Water Reporter is to democratize data by helping community science data become machine readable and readily available for collaborative use. Discoverable, digestible water quality data can help deliver improved water quality over time. Water Reporter’s use by regional efforts like Smart Citizen Science Initiative is a critical step in aligning data management across multiple organizations with complementary but diverse data uses. The Commons has demonstrated that applications like R-Shiny offer a great bridge for turning raw data into community-driven visualizations to analyze and share data. As a next step, Water Reporter will open its API to facilitate the sharing of data managed in Water Reporter with other data aggregators or to build custom visualizations to meet concrete needs.
