Participatory (Volunteer) Scientists are Monitoring Waters and Sharing Data via EPA's Water Quality eXchange (WQX)
TADA Team
2025-08-21
Source:vignettes/Participatory-Science-Water-Projects-in-WQX.Rmd
Participatory-Science-Water-Projects-in-WQX.RmdInstall and load packages
First, install and load the remotes package specifying the repo. This is needed before installing EPATADA because it is only available on GitHub (not CRAN).
install.packages("remotes",
repos = "http://cran.us.r-project.org"
)## Installing package into '/home/runner/work/_temp/Library'
## (as 'lib' is unspecified)Next, install and load EPATADA using the remotes package. USGS’s dataRetrieval and other TADA R Package dependencies will also be downloaded automatically from CRAN with the TADA install. If desired, the development version of dataRetrieval can be downloaded directly from GitHub (un-comment).
remotes::install_github("USEPA/EPATADA",
ref = "develop",
dependencies = TRUE
)
# remotes::install_github("USGS-R/dataRetrieval", dependencies=TRUE)Finally, use the library() function to load the TADA R Package into your R session.
Find volunteer data in WQX
Let’s explore participatory
science water projects using the Water Quality eXchange (WQX), Water
Quality Portal (WQP), and the EPATADA R Package. To start, let’s find
volunteer monitoring organizations who have submitted data to EPA’s
Water Quality eXchange (WQX) by reviewing the organization
domain table available here.
# Get the WQX organizations domain
organizations <- read.csv(url("https://cdx.epa.gov/wqx/download/DomainValues/Organization.CSV"))
# Subset to include only "Volunteer" organizations and exclude WQX test/training orgs
volunteer_orgs <- subset(organizations,
Type == "Volunteer" &
!grepl("training",
Name,
ignore.case = TRUE) &
!grepl("test",
Name,
ignore.case = TRUE) &
!grepl("\\*",
Name,
ignore.case = TRUE))
unique(volunteer_orgs$Name)## [1] "Bronx River Alliance (Volunteer)"
## [2] "HOWARD T ODUM FLORIDA SPRINGS INSTITUTE (Volunteer)"
## [3] "SUNCOAST WATER KEEPERS (Volunteer)"
## [4] "501CTHREE (Volunteer)"
## [5] "Alliance for Aquatic Resource Monitoring (Volunteer)"
## [6] "American Littoral Society (Volunteer)"
## [7] "Association to Preserve Cape Cod (Volunteer)"
## [8] "Applegate Partnership & Watershed Council (Volunteer)"
## [9] "Adams Rib Ranch (Volunteer)"
## [10] "Animas River Stakeholders Group (Colorado) (Volunteer)"
## [11] "Alamosa River Watershed Restoration Foundation (ARWRF) (Volunteer)"
## [12] "Ash Creek Conservation Association (Volunteer)"
## [13] "Bantam Lake Protective Organization (Volunteer)"
## [14] "Buzzards Bay Coalition (Volunteer)"
## [15] "Barnegat Bay Partnership (Volunteer)"
## [16] "Big Dry Creek Watershed Association (Volunteer)"
## [17] "Billion Oyster Project (Volunteer)"
## [18] "Barr Lake Milton Reservoir Watershed (Colorado) (Volunteer)"
## [19] "Blackstone River Coalition (Volunteer)"
## [20] "Broadwater Conservation District (Volunteer)"
## [21] "Blue Ridge Watershed Coalition (Volunteer)"
## [22] "Big Thompson Watershed Forum (CO) (Volunteer)"
## [23] "Buzzards Bay National Estuary Program (Volunteer)"
## [24] "The Cadmus Group (Massachusetts) (Volunteer)"
## [25] "Catawba Riverkeeper Foundation (CRF) (Volunteer)"
## [26] "Cape Cod Commission (Volunteer)"
## [27] "Cape Cod Cooperative Extension/Woods Hole Sea Grant (Volunteer)"
## [28] "Center for Coastal Studies (Massachusett) (Volunteer)"
## [29] "Coal Creek Watershed Coalition (Colorado) (Volunteer)"
## [30] "Clear Creek Watershed Foundation (CCWF) (Volunteer)"
## [31] "Chesapeake Bay National Estuarine Research Reserve (Volunteer)"
## [32] "Chesapeake Monitoring Cooperative (Volunteer)"
## [33] "Clackamas Soil and Water Conservation District (Volunteer)"
## [34] "Clinch Valley Soil & Water Conservation District (Volunteer)"
## [35] "Calapooia Watershed Council (Volunteer)"
## [36] "Coastal America Foundation"
## [37] "Columbia Soil and Water Conservation District"
## [38] "Connecticut River Conservancy (Volunteer)"
## [39] "Coos Watershed Association (Volunteer)"
## [40] "Coquille Watershed Association"
## [41] "Colorado River Watch (Volunteer)"
## [42] "Coalition for the Poudre River Watershed (Volunteer)"
## [43] "Clackamas River Basin Council (Volunteer)"
## [44] "Columbia Riverkeeper (Volunteer)"
## [45] "Coalition to Save Hempstead Harbor (Volunteer)"
## [46] "Colorado State University Agricultural Water Quality Program (Volunteer)"
## [47] "CT DEEP Volunteer Water Monitoring Program (Volunteer)"
## [48] "Connecticut National Estuarine Research Reserve (Volunteer)"
## [49] "Curry Watershed Partnerships (Volunteer)"
## [50] "Colorado Water Science Center (Volunteer)"
## [51] "Delaware Center for the Inland Bays (Volunteer)"
## [52] "Colorado Division of Reclamation, Mining and Safety (DRMS) (Volunteer)"
## [53] "Ducks Unlimited (Volunteer)"
## [54] "ElephantFish, LLC (Volunteer)"
## [55] "Engineering and Land Planning Associates (Volunteer)"
## [56] "Environmental Science Associates (Volunteer)"
## [57] "Eagle River Watershed Council (ERWC) (Volunteer)"
## [58] "Eightmile River Wild and Scenic Watershed in Connecticut (Volunteer)"
## [59] "Experience Learning"
## [60] "Friends of the Bay (Volunteer)"
## [61] "Friends of Blackwater (WV) (Volunteer)"
## [62] "Friends of the Cheat (Volunteer)"
## [63] "FODC ~ Friend of Deckers Creek (West Virginia)"
## [64] "Megan Murray (Volunteer)"
## [65] "Friends of the Rouge (Volunteer)"
## [66] "Farmington River Watershed Association (Volunteer)"
## [67] "Greater Arkansas River Nature Association (GARNA) (Volunteer)"
## [68] "Grand County Water Information Network (Volunteer)"
## [69] "GEI Consultants"
## [70] "Glenn and Gibson Creek Watershed Council (Volunteer)"
## [71] "Gilliam Soil and Water Conservation District (Volunteer)"
## [72] "Great Lakes Environmental Center (Volunteer)"
## [73] "GLEON Lake Observer (Volunteer)"
## [74] "Great Lakes Indian Fish and Wildlife Commission (Volunteer)"
## [75] "Grand Environmental Services (Volunteer)"
## [76] "Hedin Environmental (Volunteer)"
## [77] "Hudson River Foundation (Volunteer)"
## [78] "Housatonic Valley Association - Housatonic River Watershed (Volunteer)"
## [79] "Hyla Woods (Volunteer)"
## [80] "Illinois RiverWatch Network (Volunteer)"
## [81] "Ipswich River Watershed Association (Volunteer)"
## [82] "Kansas Alliance for Wetlands and Streams (Volunteer)"
## [83] "Kenai Watershed Forum (Volunteer)"
## [84] "Kerber Creek watershed for BLM Natural Resource (Volunteer)"
## [85] "Lawrence Brook Watershed Watch (Volunteer)"
## [86] "Lake Erie Volunteer Science Network (Volunteer)"
## [87] "Lake Fork Watershed Stakeholders (Colorado) (Volunteer)"
## [88] "Long Tom Watershed Council"
## [89] "Pontchartrain Conservancy (Volunteer)"
## [90] "Lake Quinsigamond Watershed Association (Volunteer)"
## [91] "Little Snake Field Office, BLM (Volunteer)"
## [92] "Lincoln Soil and Water Conservation District (Volunteer)"
## [93] "Malheur Watershed Council (Volunteer)"
## [94] "Massachusetts Bays National Estuary Partnership (Volunteer)"
## [95] "Mill Creek Alliance (Volunteer)"
## [96] "Mill Creek Watershed Council of Communities (Volunteer)"
## [97] "Middle Deschutes Watershed Council (Volunteer)"
## [98] "Moore Institute for Plastic Pollution Research (Volunteer)"
## [99] "Montana Volunteer Water Quality Monitoring"
## [100] "Musconetcong Watershed Association (Volunteer)"
## [101] "North American Lake Management Society"
## [102] "Nanticoke Watershed Alliance (Volunteer)"
## [103] "North Carolina Aquatic Data Hub (Volunteer)"
## [104] "Northern Colorado Water Conservancy District (Colorado) (Volunteer)"
## [105] "Groundwork NRG (Volunteer)"
## [106] "New River Conservancy (Volunteer)"
## [107] "North Fork River Improvement Association (NFRIA) - Colorado (Volunteer)"
## [108] "New Jersey Water Ambassadors (Volunteer)"
## [109] "NY/NJ Baykeeper (Volunteer)"
## [110] "North and South Rivers Watershed Association (Volunteer)"
## [111] "OARS - For the Assabet Sudbury & Concord Rivers (Volunteer)"
## [112] "Ohio Citizen Science Network (Volunteer)"
## [113] "Ohio Lake Management Society (Volunteer)"
## [114] "Potomac Appalachian Trail Club Volunteer Monitoring - VA,MD (Volunteer)"
## [115] "Parker River Clean Water Association (Volunteer)"
## [116] "Piscataqua Region Estuaries Partnership (Volunteer)"
## [117] "Port Townsend Marine Science Center (Volunteer)"
## [118] "Partnership for the Umpqua Rivers (Volunteer)"
## [119] "Rutgers Cooparative Extension Water Resource Program (Volunteer)"
## [120] "Rio Grande Headwaters Restoration Project (RGHRP) (Volunteer)"
## [121] "Raritan Headwaters Association (Volunteer)"
## [122] "Riverkeeper (Volunteer)"
## [123] "Rivers Unlimited (Volunteer)"
## [124] "Rockbridge Conservation (Volunteer)"
## [125] "Rogue River Watershed Council (Volunteer)"
## [126] "Stony Brook-Millstone Watershed Association (Volunteer)"
## [127] "South Branch Watershed Association (Volunteer)"
## [128] "Sourland Conservancy"
## [129] "South Carolina Congaree RiverKeeper (Volunteer)"
## [130] "Siuslaw Watershed Council (Volunteer)"
## [131] "South Santiam Watershed Council (Volunteer)"
## [132] "Sparkill Creek Watershed Alliance (Volunteer)"
## [133] "Solano Resource Conservation District (Volunteer)"
## [134] "Spokane River Regional Toxics Task Force (Volunteer)"
## [135] "Salem Sound Coastwatch (Volunteer)"
## [136] "Siuslaw Soil and Water Conservation District (Volunteer)"
## [137] "Standley Lake Watershed Group (Volunteer)"
## [138] "Gunnison Basin & Grand Valley Selenium Task Force (STF) (Volunteer)"
## [139] "St. Johns Riverkeeper (Volunteer)"
## [140] "Save the Sound (Volunteer)"
## [141] "Takshanuk Watershed Council (Volunteer)"
## [142] "Tillamook Estuaries Partnership"
## [143] "Tenmile Lakes Basin Partnership (Volunteer)"
## [144] "The Last Green Valley Inc (Volunteer)"
## [145] "The Watershed Institute (Volunteer)"
## [146] "UPPER MERRIMACK RIVER LOCAL ADVISORY COMMITTEE (Volunteer)"
## [147] "Uncompahgre Watershed Partnership (Volunteer)"
## [148] "Salmon Drift Creek Watershed Council (Volunteer)"
## [149] "Volunteers trained in 2009 (Volunteer)"
## [150] "Volunteer Water Information Network (TN) (Volunteer)"
## [151] "Walla Walla Basin Watershed Council"
## [152] "Waterkeeper Alliance (Volunteer)"
## [153] "Waquoit Bay National Estuarine Research Reserve (Volunteer)"
## [154] "Wasatch Front Water Quality Council Utah (Volunteer)"
## [155] "Whatcom Conservation District (Volunteer)"
## [156] "EPA Great Lakes National Program"
## [157] "North American Lake Management Society (Volunteer)"
## [158] "Poarch Band of Creeks (Tribal)"
## [159] "WaterReporterDataManagement18 (Volunteer)"
## [160] "White River Field Office - BLM (Volunteer)"
## [161] "Virginia - World Water Monitoring Day (Volunteer)"Generate a list of 5 random organization IDs:
random_volunteer_orgIDs <- sample(volunteer_orgs$ID, size = 5)Prepare list for use in TADA_DataRetrieval:
unlist(random_volunteer_orgIDs)Query the Water Quality Portal (WQP) using
TADA_DataRetrieval and the 5 random volunteer organizations
IDs. We will look for any data available from 2015 to present.
volunteer_data <- TADA_DataRetrieval(
startDate = "2015-01-01",
organization = random_volunteer_orgIDs,
ask = FALSE,
applyautoclean = TRUE
)Retrieve data from WQP
Alternatively, choose 5 volunteer monitoring organizations and query WQP. We will move forward with this example of volunteer organizations in CT:
selected_orgs <-
c("CONNRIVERCONSERVANCY",
"CT_NERR",
"BANTAMLAKE_WQX",
"CTVOLMON",
"CT_NERR")
volunteer_data <- TADA_DataRetrieval(
# startDate = "2022-01-01",
organization = selected_orgs,
ask = FALSE,
applyautoclean = TRUE
)## Checking what data is available. This may take a moment.## The number of sites and/or records matched by the query terms is large, so the download may take some time.## [1] "Downloading data from sites with fewer than 350000 results by grouping them together."
## | | | 0% | |== | 2% | |====== | 9% | |============== | 20% | |============================ | 39% | |======================================================================| 100%
## [1] "Data successfully downloaded. Running TADA_AutoClean function."
## [1] "TADA_Autoclean: creating TADA-specific columns."
## [1] "TADA_Autoclean: harmonizing dissolved oxygen characterisic name to DISSOLVED OXYGEN SATURATION if unit is % or % SATURATN."
## [1] "TADA_Autoclean: handling special characters and coverting TADA.ResultMeasureValue and TADA.DetectionQuantitationLimitMeasure.MeasureValue value fields to numeric."
## [1] "TADA_Autoclean: converting TADA.LatitudeMeasure and TADA.LongitudeMeasure fields to numeric."
## [1] "TADA_Autoclean: harmonizing synonymous unit names (m and meters) to m."
## [1] "TADA_Autoclean: updating deprecated (i.e. retired) characteristic names."
## [1] "No deprecated characteristic names found in dataset."
## [1] "TADA_Autoclean: harmonizing result and depth units."
## [1] "TADA_Autoclean: creating TADA.ComparableDataIdentifier field for use when generating visualizations and analyses."
## [1] "NOTE: This version of the TADA package is designed to work with numeric data with media name: 'WATER'. TADA_AutoClean does not currently remove (filter) data with non-water media types. If desired, the user must make this specification on their own outside of package functions. Example: dplyr::filter(.data, TADA.ActivityMediaName == 'WATER')"Explore and refine results
Review volunteer monitoring projects in WQX:
unique(volunteer_data$ProjectName)## [1] "Riffle Bioassessment by Volunteers Program"
## [2] "Volunteer Stream Temperature Monitoring Network"
## [3] "CRC and Affiliate Monitoring"
## [4] "Connecticut Lake Watch"
## [5] "CRC 2019 Bacteria Monitoring"
## [6] "Chicopee Four Rivers Watershed Council 2019"
## [7] "Deerfield River Watershed Association 2019"
## [8] "Connecticut River Conservancy/Connecticut River Watershed Council 2012-2018"
## [9] "2019 Anguilla Brook Watershed Bacteria Source Trackdown"
## [10] "2012 Flat Brook Trackdown Survey"
## [11] "Connecticut River Conservancy 2020"
## [12] "Fort River Watershed Asssociation 2020"
## [13] "Deerfield River Watershed Association 2020"
## [14] "Chicopee 4River Watershed Council 2020"
## [15] "Pomperaug River Watershed Based Plan Implementation Groundwork: Additional Water Quality Monitoring, Agricultural Outreach, BMP Implementation Design and Landowner Agreements to Address Bacteria Impairments. EPA RFA No. 21059"
## [16] "Still River Watershed Pollution Trackdown Survey. CTDEEP Contract No. 17-06"
## [17] "Connecticut River Conservancy 2021"
## [18] "Chicopee 4 Rivers Watershed Council 2021"
## [19] "Deerfield River Watershed Association 2021"
## [20] "Connecticut River Conservancy 2022"
## [21] "Deerfield River Watershed Association 2022"
## [22] "Chicopee 4 Rivers Watershed Council 2022"
## [23] "Clean Up Sound & Harbors (CUSH) volunteer water monitoring program"
## [24] "CT Harbor Watch water monitoring program"Generate pie chart:
TADA_FieldValuesPie(volunteer_data, field = "MonitoringLocationTypeName")
Review monitoring locations:
TADA_FlaggedSitesMap(volunteer_data)Review and remove sites if coordinates are imprecise or outside US:
volunteer_data <- TADA_FlagCoordinates(volunteer_data,
clean_outsideUSA = "remove",
clean_imprecise = TRUE,
flaggedonly = FALSE)Use TADA_OverviewMap to generate a map:
TADA_OverviewMap(volunteer_data)Review and remove duplicate results if present:
volunteer_data <- TADA_FindPotentialDuplicatesSingleOrg(volunteer_data)## [1] "TADA_FindPotentialDuplicatesSingleOrg: 344 groups of potentially duplicated results found in dataset. These have been placed into duplicate groups in the TADA.SingleOrgDupGroupID column and the function randomly selected one result from each group to represent a single, unduplicated value. Selected values are indicated in the TADA.SingleOrgDup.Flag as 'Unique', while duplicates are flagged as 'Duplicate' for easy filtering."
volunteer_data <- dplyr::filter(volunteer_data, TADA.SingleOrgDup.Flag == "Unique")Prepare censored (nondetects and overdetects) results for analysis:
volunteer_data <- TADA_SimpleCensoredMethods(
volunteer_data,
nd_method = "multiplier",
nd_multiplier = 0.5,
od_method = "as-is",
od_multiplier = "null"
)## [1] "TADA_IDCensoredData: 85 records in supplied dataset have conflicting detection condition and detection limit type information. These records will not be included in detection limit handling calculations."Run key TADA quality control flagging functions and remove suspect results:
volunteer_data <- TADA_RunKeyFlagFunctions(
volunteer_data,
clean = TRUE
)## [1] "TADA_FindQCActivities: Quality control samples have been removed or were not present in the input dataframe. Returning dataframe with TADA.ActivityType.Flag column for tracking."Flag results above and below thresholds. Review carefully and consider removing.
volunteer_data <- TADA_FlagAboveThreshold(volunteer_data,
clean = FALSE,
flaggedonly = FALSE)
volunteer_data <- TADA_FlagBelowThreshold(volunteer_data,
clean = FALSE,
flaggedonly = FALSE)## [1] "No data below the WQX Lower Threshold were found in your dataframe. Returning the input dataframe with TADA.ResultValueBelowLowerThreshold.Flag column for tracking."Harmonize synonyms if found:
volunteer_data <- TADA_HarmonizeSynonyms(volunteer_data)Generate table:
TADA_FieldValuesTable(volunteer_data, field = "ActivityTypeCode")## Value Count
## 1 Sample-Routine 33605
## 2 Field Msr/Obs 3553Generate pie chart:
TADA_FieldValuesPie(volunteer_data, field = "OrganizationFormalName")
Generate pie chart:
TADA_FieldValuesPie(volunteer_data, field = "TADA.MonitoringLocationName")
Remove non-numeric results:
volunteer_data <- TADA_ConvertSpecialChars(
volunteer_data,
col = "TADA.ResultMeasureValue",
clean = TRUE)Review the number of sites and records for each characteristic:
TADA_SummarizeColumn(volunteer_data)## # A tibble: 22 × 3
## TADA.CharacteristicName$TADA.CharacteristicName n_sites n_records
## <chr> <int> <int>
## 1 AMMONIA 15 244
## 2 CONDUCTANCE 15 149
## 3 COUNT 798 22161
## 4 DEPTH, SECCHI DISK DEPTH 41 474
## 5 DISSOLVED OXYGEN (DO) 15 589
## 6 ENTEROCOCCUS 15 117
## 7 ESCHERICHIA COLI 411 8641
## 8 FECAL COLIFORM 16 129
## 9 INORGANIC NITROGEN (NO2, NO3, & NH3) 15 269
## 10 NITRATE 30 303
## # ℹ 12 more rowsFilter data to review a single characteristic:
ecoli <- dplyr::filter(
volunteer_data,
TADA.ComparableDataIdentifier %in% c(
"ESCHERICHIA COLI_NA_NA_CFU/100ML"
)
)Generate scatter plot for E. coli:
TADA_GroupedScatterplot(ecoli)## [1] "TADA_GroupedScatterplot: No 'groups' selected for MonitoringLocationName. There are 236 MonitoringLocationNames in the TADA dataframe. The top four MonitoringLocationNames by number of results will be plotted: Sunderland Boat Ramp; CT River at Barton Cove Boat Ramp (now MA-CTR_122.5); DCR/UMASS boat dock and Oxbow/Easthampton Boat Ramp."Filter to a single site and continue exploring E. coli:
ecoli <- dplyr::filter(
ecoli,
TADA.MonitoringLocationIdentifier %in% c(
"CONNRIVERCONSERVANCY-OXE1"
)
)Let’s check if any results are above the EPA 304A recommended maximum criteria magnitude (see: 2012 Recreational Water Quality Criteria Fact Sheet).
If interested, you can find other state, tribal, and EPA 304A criteria in EPA’s Criteria Search Tool.
Let’s check if any individual results exceed 320 CFU/100mL (the magnitude component of the EPA recommendation 2 criteria for ESCHERICHIA COLI).
# add column with comparison to criteria mag (excursions)
ecoli <- ecoli %>%
dplyr::mutate(meets_criteria_mag = ifelse(TADA.ResultMeasureValue <= 320, "Yes", "No"))
# review subset
ecoli_subset_review <- ecoli %>%
dplyr::select(
TADA.MonitoringLocationIdentifier, TADA.MonitoringLocationName, OrganizationFormalName, ActivityStartDate, TADA.ResultMeasureValue,
meets_criteria_mag
)
TADA_TableExport(ecoli_subset_review)Generate stats table. Review percentiles. Less than 5% of results fall above ~19 CFU/100mL and over 98% of results fall below ~2185 CFU/100m.
TADA_TableExport(TADA_Stats(ecoli))Generate a scatterplot.
TADA_Scatterplot(ecoli, id_cols = "TADA.ComparableDataIdentifier") %>%
plotly::add_lines(
y = 320,
x = c(min(ecoli$ActivityStartDate), max(ecoli$ActivityStartDate)),
inherit = FALSE,
showlegend = FALSE,
line = list(color = "red"),
hoverinfo = "none"
)Generate a histogram.
TADA_Histogram(ecoli, id_cols = "TADA.ComparableDataIdentifier")TADA_Boxplot can be useful for identifying skewness and
percentiles.
TADA_Boxplot(ecoli, id_cols = "TADA.ComparableDataIdentifier")Check out other example R workflows designed to work with WQP data under the Articles tab on the EPATADA package website.