Overview

fluxtools is an R package that provides an interactive Shiny‐based QA/QC environment for data in the AmeriFlux BASE format. In just a few clicks, you can:

1. Upload eddy covariance data in a .csv format (AmeriFlux standard naming and timestamp conventions)
2. Visualize any two numeric columns against time (or each other)
3. Highlight statistical outliers (±σ from a linear fit) and add them to your point-removal R code
4. Manually select and remove data points via a lasso or box. Selecting these adds to the accumulated removal code
5. Copy and paste the generated code into your own R script for reproducible QA/QC
6. Download a “cleaned” CSV with excluded values (using “apply removals”) set to NA and an R script for reproducibility

This vignette shows you how to install, launch, and use the main Shiny app—run_flux_qaqc()—and walks through a typical workflow.

Installation

You can install fluxtools from CRAN, or directly from GitHub:

# Install from CRAN 
install.packages("fluxtools")

# Install from GitHub
library(devtools) 
devtools::install_github("kesondrakey/fluxtools")

Launching the Shiny App

Load fluxtools and launch the QA/QC application:

library(fluxtools)

# Run the app
run_fluxtools()

Example workflow

1. Upload: Select your AmeriFlux-style CSV (e.g., US_VT1_HH_202401010000_202501010000.csv). Files can be up to 500MB (larger file sizes might be harder on the Shiny interface)

2. Choose Year(s): By default “all” is selected, but you can subset to specific years

3. Choose variables: TIMESTAMP_START is on the x-axis by default. Change the y-axis to your variable of interest (e.g., FC_1_1_1). The generated R code focuses on removing the y-axis variable

4. Select data: Use the box or lasso to select points. This populates the “Current” code box with something like:

   df <- df %>%
     mutate(
       FC_1_1_1 = case_when(
         TIMESTAMP_START == '202401261830' ~ NA_real_,
         TIMESTAMP_START == '202401270530' ~ NA_real_,
         …
         TRUE ~ FC_1_1_1
       )
     )

5. Flag data and Accumulate code: With points still selected, click “Flag data.” Selected points turn orange, and code is appended to the “Accumulated” box, allowing multiple selections per session.

6. Unflag data: Use the box or lasso to de-select points and remove from the Accumulated code box.

7. Clear Selection: To reset all selections from the current y-variable, click “Clear Selection” to reset the current view.

8. Switch variables: Change y to any other variable (e.g., SWC_1_1_1) and select more points. Click “Flag data” Code for both variables to appear:

  df <- df %>%
    mutate(
      FC_1_1_1 = case_when(
        TIMESTAMP_START == '202401261830' ~ NA_real_,
        TIMESTAMP_START == '202401270530' ~ NA_real_,
        …
        TRUE ~ FC_1_1_1
      )
    )

  df <- df %>%
    mutate(
      SWC_1_1_1 = case_when(
        TIMESTAMP_START == '202403261130' ~ NA_real_,
        TIMESTAMP_START == '202403270800' ~ NA_real_,
        …
        TRUE ~ SWC_1_1_1
      )
    )

9. Compare variables: Change to variables you would like to compare (e.g., change y to TA_1_1_1 and x to T_SONIC_1_1_1). The app computes an R² via simple linear regression. The top R² is based on points before removals, and once data is selected, a second R² will pop up - calculating the linear regression assuming the selected points have been removed

10. Highlight outliers: Use the slider to select ±σ residuals. Click “Select all ±σ outliers” to append them to the Accumulated code. Click “Clear ±σ outliers” to deselect and remove from the code box

11. Copy all: Click the Copy Icon to the right of the current or accumulated code box and paste into your own R script for documentation

12. Apply Removals: Click “Apply Removals” to remove each selected data points, from the current y-variable, to replace points with NA in a new .csv (raw data is unaffected), available using ‘export cleaned data’ and remove these values from view

13. Reload original data: Make a mistake or want a fresh start? Click Reload original data to reload the .csv from above to start over

14. Export cleaned data: Download the cleaned .csv reflecting your confirmed removals. This button will download a zip file containing your .csv, reflecting changes from using the “apply removals” button, and includes a compiled R script with the R code for those removals.

Physical Boundary Module (PRM) function:

The Physical Range Module (PRM) removes out-of-range values to NA based on similar variables using patterns like ^SWC($|_) or ^P($|_).
Columns containing "QC" are skipped by default. No columns are removed.

Source of ranges: AmeriFlux Technical Documents, Table A1 (Physical Range Module).

# tiny demo dataset with a few out-of-range values
set.seed(1)
df <- tibble::tibble(
  TIMESTAMP_START = seq.POSIXt(as.POSIXct("2024-01-01", tz = "UTC"),
                               length.out = 10, by = "30 min"),
  SWC_1_1_1 = c(10, 20, 150, NA, 0.5, 99, 101, 50, 80, -3),  # bad: 150, 101, -3; 0.5 triggers SWC unit note
  P         = c(0, 10, 60, NA, 51, 3, 0, 5, 100, -1),        # bad: 60, 51, 100, -1
  RH_1_1_1  = c(10, 110, 50, NA, 0, 100, -5, 101, 75, 30),   # bad: 110, -5, 101
  SWC_QC    = sample(0:2, 10, replace = TRUE)                # QC col should be ignored
)

# To see the Physical Boundary Module (PRM) rules:
get_prm_rules()

## # A tibble: 113 × 6
##    family           variable     min   max description                     units
##    <chr>            <chr>      <dbl> <dbl> <chr>                           <chr>
##  1 ^COND_WATER($|_) COND_WATER     0 10000 Conductivity of water           uS c…
##  2 ^DO($|_)         DO             0    NA Dissolved oxygen in water       umol…
##  3 ^PCH4($|_)       PCH4           0    NA Dissolved methane (CH4) in wat… nmol…
##  4 ^PCO2($|_)       PCO2           0 10000 Dissolved carbon dioxide (CO2)… umol…
##  5 ^PN2O($|_)       PN2O           0    NA Dissolved nitrous oxide (N2O) … nmol…
##  6 ^PPFD_UW_IN($|_) PPFD_UW_IN     0  2400 Photosynthetic photon flux den… umol…
##  7 ^TW($|_)         TW           -20    50 Water temperature               deg C
##  8 ^DBH($|_)        DBH            0   500 Tree diameter at breast height  cm   
##  9 ^LEAF_WET($|_)   LEAF_WET       0   100 Leaf wetness (0-100)            %    
## 10 ^SAP_DT($|_)     SAP_DT       -10    10 Sapflow probe temperature diff… deg C
## # ℹ 103 more rows

#Apply filter to all relevant variables
res <- apply_prm(df)

## PRM summary:
## * P
##   expected units: mm, PRM range: 0 to 50
##   4 values set to NA (44.4% of data)
## * RH_1_1_1
##   expected units: %, PRM range: 0 to 100
##   3 values set to NA (33.3% of data)
## * SWC_1_1_1
##   expected units: %, PRM range: 0 to 100
##   3 values set to NA (33.3% of data)

# PRM summary (counts and % replaced per column)
res$summary

## # A tibble: 3 × 7
##   column    family   min   max n_non_na_before n_replaced pct_replaced
##   <chr>     <chr>  <dbl> <dbl>           <int>      <int>        <dbl>
## 1 P         P          0    50               9          4         44.4
## 2 RH_1_1_1  RH         0   100               9          3         33.3
## 3 SWC_1_1_1 SWC        0   100               9          3         33.3

# Only set range for SWC 
df_filtered_swc <- apply_prm(df, include = "SWC")

## PRM summary:
## * SWC_1_1_1
##   expected units: %, PRM range: 0 to 100
##   3 values set to NA (33.3% of data)

# Only set range for SWC + P 
df_filtered_swc_P <- apply_prm(df, include = c("SWC", "P"))

## PRM summary:
## * P
##   expected units: mm, PRM range: 0 to 50
##   4 values set to NA (44.4% of data)
## * SWC_1_1_1
##   expected units: %, PRM range: 0 to 100
##   3 values set to NA (33.3% of data)