The Commons has spent the past few months developing a new soon-to-be public API or ‘Application Programming Interface’ for our Water Reporter platform. With release coming later this year we thought it would be useful to try and demystify APIs and provide some examples of why they are incredibly valuable to community led water quality improvement efforts.
What is an API?
An API is a framework for transferring data between websites, databases, applications, and anything else that might live on the internet. Think of an API as a type of conversation, it facilitates the passing of data between two different web properties much like one person requesting directions from another.
Request: “Where’s the pizza place?”
Response: “Turn right at the circle, drive straight for half a mile, the pizza place is on the left”.
It’s not the information itself that’s important, but how it’s clearly and concisely communicated. We’ve all found ourselves lost, drowning in ‘rights’ and ‘lefts’ from overly complicated directions. API’s solve this problem by having a mutually agreed upon style that typically reveals small bits of data rather than entire databases. In other words, an API allows you to take the direct route to your data, requesting specific elements, like simple directions to the pizza parlor, not the tour down every side street between you and lunch.
What Does an API Do?
APIs are a fundamental element of the modern internet. They’re everywhere, sharing information, and most of the time we’re none the wiser. Most websites, applications, and web services all require APIs to function. For example, Water Reporter uses Mapbox’s API, which allows us to leverage their extensive mapping capabilities in our maps.
Consider how you buy stuff online. Countless ecommerce websites use PayPal or Stripe to process transactions but the money magically deposits into their bank account. Curious about the ubiquity of the conversations APIs drive? Here’s a list of Google’s APIs, ranging from the Google Calendar API to an API for discovering APIs! Ok, now that we understand the extensiveness of APIs in our everyday online experience, let’s explore how they actually work.
How does an API work?
When two applications are talking — what they choose to say is the heart of how an API works. As developers, we get to dictate the conversation. Among many, there are two basic verbs found in most APIs, GET and POST. As the word implies, a GET request asks for a piece of information from one system and transfers it to another system, like when Water Reporter uses USGS station’s flow rate data. Water Reporter sends a GET request for a particular station and parameter, and USGS returns the data. Conversely, a POST request sends a piece of information to a server, like if you were to add a new sample to EPA’s Water Quality Exchange Portal through their API. There are other verbs that could be part of the conversation like PATCH, DELETE, and CONNECT, but let’s not worry about those for now.
As a first step, a developer needs to create nodes that are willing to talk, think of these as access hatches for particular elements of an application. Once the developer has created the nodes, the conversations can start flowing via an API request. To construct an API request, a developer sends some information, and typically an access token — a special key that gives you permission to access certain information based on your credentials.
Google’s Geocoding API is a great example of a simple API; It allows users to send a street address and a token, then the API returns the latitude longitude coordinate pair and additional metadata. Check out the URL request, and subsequent API response.
https://maps.googleapis.com/maps/api/geocode/json?address=1600+Amphitheatre+Parkway,+Mountain+View,+CA&key=YOUR_API_KEY
"formatted_address": "1600 Amphitheatre Pkwy, Mountain View, CA 94043,USA",
"geometry": {
"location": {
"lat": 37.4224428,
"lng": -122.0842467
},
"location_type": "ROOFTOP",
"viewport": {
"northeast": {
"lat": 37.4239627802915,
"lng": -122.0829089197085
},
"southwest": {
"lat": 37.4212648197085,
"lng": -122.0856068802915
}
}
},
Notice the format returned by the API, it’s called JSON. APIs can return quite a few different data formats like XML, plaintext, or CSVs, but JSON is the most common because it is both machine readable, and human readable! Because the returned JSON will be the same format every time, you could use this workflow programmatically. A possible use case could be locating crowd sourced addresses of illegal dumping for targeted cleanups.
Lets try it out ourselves with a QR code API from GoQR.me! Copy and paste this link into your browser and the free QR code API will honor your GET request. You can swap out ‘URL’ at the end of the request to encode the content of your choice.
https://api.qrserver.com/v1/create-qr-code/?size=150x150&data=URL
You can use your phone’s QR code scanner to try it out. You’ve just performed an API request! You sent the base URL (api.qrserver.com/v1/create-qr-code), along with some parameters like the QR code ‘size’ and your link ‘data’. Note the use of ‘?’ following the base URL to indicate the start of your request and ‘&’ to separate the parameters, known as key value pairs. Take a look at the documentation to play with other key value pairs like color and border styles!
We can play fun games thanks to APIs all day, but this technology also has big impacts in our line of work to help power broader and stronger connections of information in our platforms. In FieldDoc, the API allows us to pull critical nutrient and sediment load reduction estimation models into the system to calculate the effect of environmental improvement practices on the greater ecosystem. For example, we POST required best management practice (BMP) data such as location, BMP Type, and other necessary attributes, and the API returns nutrient and sediment load reductions which are then rendered in FieldDoc’s web application. We are brainstorming how APIs can dismantle existing technological barriers by opening up data sharing as well as modeling pathways for community science water quality monitoring.
Impacts of API for Water Data
Across the globe, countless small watershed organizations are conducting independent water quality monitoring programs. Some collect basic chemical information like water temperature, pH, and dissolved oxygen. Other groups record benthic invertebrate counts while still others test for heavy metals or E. Coli. These monitoring programs provide incredible resolution of watershed health at the local scale but also can inform and mobilize regional and federal efforts to address water quality issues if the data is sharable. It’s useful for comparisons, historical trends, assessing improvement efforts, and identifying irregularities.
Historically, a lot of data collected by community scientists did not meet the rigorous requirements of data management and or meet a high enough quality for ingestion into broader data aggregator efforts. Groups like the Water Data Collaborative, Cleveland Water Alliance’s Smart Citizen Science Initiative, and Long Island Sound Unified Waters Study have invested heavily in converting community groups from paper data sheets to robust digital data collection that facilitates the cross-platform sharing of data.
APIs catalyze these efforts by enabling water data collectors and users to speak the same language, sharing and transferring data seamlessly. Even with the broad efforts to standardize monitoring programs, many groups currently store their data in local spreadsheets, databases, and disparate formats. Some may send their data to a county, state, or federal portal like the Water Quality Exchange Portal.
Unfortunately, more often than not, most collected data never sees the light of day. We hear about groups that have years, sometimes decades of data that they have collected with volunteers that no one has ever seen or analyzed. The data lives on a hard drive or an isolated database system lacking a usable API framework. However, there is hope for the water monitoring community to keep a pulse on their waterway and share their results without losing authenticity or expending more time into data management. That hope comes in the form of APIs.The Commons isn’t the first organization to build an API dedicated to water data and we won’t be the last. The USGS for example has a host of different APIs for revealing monitoring sites, getting instantaneous readings, or even historical groundwater data.
A good API for water quality data establishes an agnostic data structure that can accept different data formats and reveal that data through simple and logical requests. Think of the opportunities! With APIs allowing conversations between different data owners, data aggregators, analysts, and broad audiences, the number of opportunities to effectively use the data skyrocket. Consider this, with an API people can build applications and run frequent analysis without worrying about inconsistent data through time. Data owners can easily share their data without having to reconstruct their files to meet the individual specifications of each system they are asked to contribute to. With time and the broadening of networks that communicate with each other, a group monitoring water quality in Wisconsin at the headwaters of the Mississippi can communicate that data to folks near the mouth in Louisiana and Arkansas with the click of a button or just managing their data, conducting business as usual.
As we have built Water Reporter and expanded our Data Manager feature and functionality, we have been thoughtfully discussing with our stakeholders how we can take these ideas into reality to spark conversations amongst our users’ datasets as well as amongst monitoring program administrators themselves. Before long, we anticipate opening up some of the system’s APIs so that more developers and watershed organizations can lean into the data democratization effort.
Where the Water Reporter API is different is that it’s geared directly towards community science water quality monitoring and daylighting the types of data often gathered by small watershed groups. We are also thinking about highly innovative ways to leverage these data beyond just sending data from one platform to another. The Water Reporter API is structured to easily reveal sample data, station pictures, and even threshold dependent information like station status and color. Although you’ll likely require some development background to leverage the API, we hope that folks with moderate statistical programming background in R, Python, or Stata will find it accessible. Over the coming months, we’ll be demonstrating some example applications and data analysis that wouldn’t be possible without the API, so stay tuned for more! For any questions about our development efforts reach out via support@waterreporter.org or DM @ourcommoncode on Twitter.
