Details:
Published: June 30, 2020
This tutorial demonstrates how to use R to connect to the AppEEARS API The Application for Extracting and Exploring Analysis Ready Samples (AppEEARS) offers a simple and efficient way to access and transform geospatial data from a variety of federal data archives in an easy-to-use web application interface. AppEEARS enables users to subset geospatial data spatially, temporally, and by band/layer for point and area samples. AppEEARS returns not only the requested data, but also the associated quality values, and offers interactive visualizations with summary statistics in the web interface. The AppEEARS API offers users programmatic access to all features available in AppEEARS, with the exception of visualizations. The API features are demonstrated in this tutorial.
Example: Submit a point request with multiple points in U.S. National Parks for extracting vegetation and land surface temperature data
In this tutorial, Connecting to the AppEEARS API, querying the list of available products, submitting a point sample request, downloading the request, working with the AppEEARS Quality API, and loading the results into R for visualization are covered. AppEEARS point requests allow users to subset their desired data using latitude/longitude geographic coordinate pairs (points) for a time period of interest, and for specific data layers within data products. AppEEARS returns the valid data from the parameters defined within the sample request.
Data Used in the Example:
- Data layers:
- Combined MODIS Leaf Area Index (LAI)
- MCD15A3H.006, 500m, 4 day: 'Lai_500m'
- Terra MODIS Land Surface Temperature
- MOD11A2.061, 1000m, 8 day: 'LST_Day_1km', 'LST_Night_1km'
- Combined MODIS Leaf Area Index (LAI)
Topics Covered in this tutorial:
- Getting Started
1a. Load Packages 1b. Set Up the Output Directory
1c. Login - Query Available Products
2a. Search and Explore Available Products
2b. Search and Explore Available Layers - Submit a Point Request
3a. Compile a JSON Object 3b. Submit a Task Request 3c. Retrieve Task Status - Download a Request
4a. Explore Files in Request Output
4b. Download Files in a Request (Automation) - Explore AppEEARS Quality API
5a. List Quality Layers
5b. Show Quality Values
5c. Decode Quality Values - BONUS: Load Request Output and Visualize
6a. Load CSV
6b. Plot Results (Line/Scatter Plots)
Prerequisites:
-
A NASA Earthdata Login account is required to login to the AppEEARS API and submit a request . You can create an account at the link provided.
-
Install R and RStudio. These tutorials have been tested on Windows and MAc systems using R Version 4.0.0, RStudio version 1.1.463, and the specifications listed below.
-
Required packages:
getPasshttrjsonlitewarnings
- To read and visualize the tabular data:
dplyrtidyrreadrggplot2
Procedures:
Getting Started:
-
Clone/download AppEEARS API Getting Started in R Repository from the LP DAAC Data User Resources Repository.
-
Open the
AppEEARS_API_R.Rprojfile to directly open the project. Next, select theAppEEARS_API_Point_R.Rmdfrom the files list and open it.
Environment Setup:
1. Check the version of R using version and RStudio using RStudio.Version() and update them if needed.
- Windows
- Install and load installr:
Install.packages("installr");library(installr)- Copy/Update the existing packages to the new R installation:
UpdateR()
-
Open RStudio, go to Help > Check for Updates to install newer version of RStudio (if available).
-
Mac
- Go to https://cloud.r-project.org/bin/macosx/.
- Download the latest release (R-4.0.1.pkg) and finish the installation.
- Open RStudio, go to Help > Check for Updates to install newer version of RStudio (if available).
- To update packages, go to Tools > Check for Package Updates. If updates are available, select All, and click Install Updates.
2. Download any required packages that you are missing using either options below.
-
Use
install.packages('package name')command in the console. To download multiple packages useinstall.packages(c('package #1', 'package #2',...)). -
Run
AppEEARS_API_install.Rto find the missing packages and install them.
source("../Scripts/AppEEARS_API_Point_Install.R")
All packages already installed
AppEEARS Information:
To access AppEEARS, visit: https://appeears.earthdatacloud.nasa.gov/.
For comprehensive documentation of the full functionality of the AppEEARS API, please see the AppEEARS API Documentation.
1. Getting Started
1a. Load Packages
First, load the R packages necessary to run the tutorial.
# Load necessary packages into R
library(getPass) # A micro-package for reading passwords
library(httr) # To send a request to the server/receive a response from the server
library(jsonlite) # Implements a bidirectional mapping between JSON data and the most important R data types
library(ggplot2) # Functions for graphing and mapping
library(tidyr) # Function for working with tabular data
library(dplyr) # Function for working with tabular data
library(readr) # Read rectangular data like CSV
1b. Set Up the Output Directory
Set your input directory, and create an output directory for the results.
outDir <- file.path("R_output") # Create an output directory if it doesn't exist
suppressWarnings(dir.create(outDir))
1c. Login
To submit a request, you must first login to the AppEEARS API. Use the getpass package to enter your NASA Earthdata login Username and Password. When prompted after executing the code block below, enter your username followed by your password.
user <- getPass(msg = "Enter NASA Earthdata Login Username: ") # Enter NASA Earthdata Login Username
password <- getPass(msg = "Enter NASA Earthdata Login Password: ") # Enter NASA Earthdata Login Password
Decode the username and password to be used to post login request.
secret <- jsonlite::base64_enc(paste(user, password, sep = ":")) # Encode the string of username and password
Next, assign the AppEEARS API URL to a static variable.
API_URL = 'https://appeears.earthdatacloud.nasa.gov/api/' # Set the AppEEARS API to a variable
Use the httr package to post your username and password. A successful login will provide you with a token to be used later in this tutorial to submit a request. For more information or if you are experiencing difficulties, please see the API Documentation.
# Insert API URL, call login service, set the component of HTTP header, and post the request to the server
response <- httr::POST(paste0(API_URL,"login"),
add_headers("Authorization" = paste("Basic", gsub("\n", "", secret)),
"Content-Type" ="application/x-www-form-urlencoded;charset=UTF-8"),
body = "grant_type=client_credentials")
response_content <- content(response) # Retrieve the content of the request
token_response <- toJSON(response_content, auto_unbox = TRUE) # Convert the response to the JSON object
remove(user, password, secret, response) # Remove the variables that are not needed anymore
prettify(token_response) # Print the prettified response
{
"token_type": "Bearer",
"token": "EXIW5BKOfm6GyO99WnD3Acwexo-JN5cNdlksZcdRB34C9x9kmCVxI34hwyjYKyO3hfXfmHk32ZdrcAg59QVb2Q",
"expiration": "2022-04-06T21:08:05Z"
}
Above, you should see a Bearer token. Notice that this token will expire approximately 48 hours after being acquired.
2. Query Available Products
The product API provides details about all of the products and layers available in AppEEARS. For more information, please see the API Documentation.
Below, call the product API to list all of the products available in AppEEARS.
prods_req <- GET(paste0(API_URL, "product")) # Request the info of all products from product service
prods_content <- content(prods_req) # Retrieve the content of request
all_Prods <- toJSON(prods_content, auto_unbox = TRUE) # Convert the info to JSON object
remove(prods_req, prods_content) # Remove the variables that are not needed anymore
# prettify(all_Prods) # Print the prettified product response
2a. Search and Explore Available Products
Create a list indexed by product name to make it easier to query a specific product.
# Divides information from each product.
divided_products <- split(fromJSON(all_Prods), seq(nrow(fromJSON(all_Prods))))
# Create a list indexed by the product name and version
products <- setNames(divided_products,fromJSON(all_Prods)$ProductAndVersion)
# Print no. products available in AppEEARS
sprintf("AppEEARS currently supports %i products." ,length(products))
[1] "AppEEARS currently supports 155 products."
Next, look at the product's names and descriptions. Below, the 'ProductAndVersion' and 'Description' are printed for all products.
# Loop through the products in the list and print the product name and description
for (p in products){
print(paste0(p$ProductAndVersion," is ",p$Description," from ",p$Source))
}
## [1] "GPW_DataQualityInd.004 is Quality of Input Data for Population Count and Density Grids from SEDAC"
## [1] "GPW_UN_Adj_PopCount.004 is UN-adjusted Population Count from SEDAC"
## [1] "GPW_UN_Adj_PopDensity.004 is UN-adjusted Population Density from SEDAC"
## [1] "MCD12Q1.006 is Land Cover Type from LP DAAC"
## [1] "MCD12Q2.006 is Land Cover Dynamics from LP DAAC"
## [1] "MCD15A2H.006 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "MCD15A3H.006 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "MCD43A1.006 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD43A1.061 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD43A2.061 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD43A3.006 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD43A3.061 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD43A4.006 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD43A4.061 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD64A1.006 is Burned Area (fire) from LP DAAC"
## [1] "MCD64A1.061 is Burned Area (fire) from LP DAAC"
## [1] "MOD09A1.006 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MOD09A1.061 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MOD09GA.006 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MOD09GA.061 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MOD09GQ.006 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MOD09GQ.061 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MOD09Q1.006 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MOD09Q1.061 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MOD10A1.005 is Snow Cover from NSIDC DAAC"
## [1] "MOD10A1.006 is Snow Cover (NDSI) from NSIDC DAAC"
## [1] "MOD10A2.005 is Snow Cover from NSIDC DAAC"
## [1] "MOD10A2.006 is Snow Cover from NSIDC DAAC"
## [1] "MOD11A1.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MOD11A1.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MOD11A2.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MOD11A2.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MOD13A1.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13A1.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13A2.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13A2.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13A3.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13A3.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13Q1.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13Q1.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD14A2.006 is Thermal Anomalies and Fire from LP DAAC"
## [1] "MOD14A2.061 is Thermal Anomalies and Fire from LP DAAC"
## [1] "MOD15A2H.006 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "MOD15A2H.061 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "MOD16A2.006 is Evapotranspiration (ET & LE) from LP DAAC"
## [1] "MOD16A2.061 is Evapotranspiration (ET & LE) from LP DAAC"
## [1] "MOD16A2GF.006 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MOD16A2GF.061 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MOD16A3GF.006 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MOD16A3GF.061 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MOD17A2H.006 is Gross Primary Productivity (GPP) from LP DAAC"
## [1] "MOD17A2H.061 is Gross Primary Productivity (GPP) from LP DAAC"
## [1] "MOD17A2HGF.006 is Gross Primary Productivity (GPP) from LP DAAC"
## [1] "MOD17A2HGF.061 is Gross Primary Productivity (GPP) from LP DAAC"
## [1] "MOD17A3HGF.006 is Net Primary Production (NPP) Gap-Filled from LP DAAC"
## [1] "MOD17A3HGF.061 is Net Primary Production (NPP) Gap-Filled from LP DAAC"
## [1] "MOD21A1D.061 is Temperature and Emissivity (LST&E) from LP DAAC"
## [1] "MOD21A1N.061 is Temperature and Emissivity (LST&E) from LP DAAC"
## [1] "MOD21A2.061 is Temperature and Emissivity (LST&E) from LP DAAC"
## [1] "MOD44B.006 is Vegetation Continuous Fields (VCF) from LP DAAC"
## [1] "MOD44W.006 is Land/Water Mask from LP DAAC"
## [1] "MODOCGA.006 is Ocean Reflectance Bands 8-16 from LP DAAC"
## [1] "MODTBGA.006 is Thermal Bands and Albedo from LP DAAC"
## [1] "MYD09A1.006 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MYD09A1.061 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MYD09GA.006 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MYD09GA.061 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MYD09GQ.006 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MYD09GQ.061 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MYD09Q1.006 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MYD09Q1.061 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MYD10A1.005 is Snow Cover from NSIDC DAAC"
## [1] "MYD10A1.006 is Snow Cover (NDSI) from NSIDC DAAC"
## [1] "MYD10A2.005 is Snow Cover from NSIDC DAAC"
## [1] "MYD10A2.006 is Snow Cover from NSIDC DAAC"
## [1] "MYD11A1.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD11A1.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD11A2.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD11A2.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD13A1.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13A1.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13A2.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13A2.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13A3.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13A3.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13Q1.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13Q1.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD14A2.006 is Thermal Anomalies and Fire from LP DAAC"
## [1] "MYD14A2.061 is Thermal Anomalies and Fire from LP DAAC"
## [1] "MYD15A2H.006 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "MYD15A2H.061 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "MYD16A2.006 is Evapotranspiration (ET & LE) from LP DAAC"
## [1] "MYD16A2.061 is Evapotranspiration (ET & LE) from LP DAAC"
## [1] "MYD16A2GF.006 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MYD16A2GF.061 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MYD16A3GF.006 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MYD16A3GF.061 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MYD17A2H.006 is Gross Primary Productivity (GPP) from LP DAAC"
## [1] "MYD17A2H.061 is Gross Primary Productivity (GPP) from LP DAAC"
## [1] "MYD17A2HGF.006 is Gross Primary Productivity (GPP) Gap-Filled from LP DAAC"
## [1] "MYD17A2HGF.061 is Gross Primary Productivity (GPP) Gap-Filled from LP DAAC"
## [1] "MYD17A3HGF.006 is Net Primary Production (NPP) Gap-Filled from LP DAAC"
## [1] "MYD17A3HGF.061 is Net Primary Production (NPP) Gap-Filled from LP DAAC"
## [1] "MYD21A1D.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD21A1D.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD21A1N.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD21A1N.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD21A2.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD21A2.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYDOCGA.006 is Ocean Reflectance Bands 8-16 from LP DAAC"
## [1] "MYDTBGA.006 is Thermal Bands and Albedo from LP DAAC"
## [1] "NASADEM_NC.001 is Elevation from LP DAAC"
## [1] "NASADEM_NUMNC.001 is Source from LP DAAC"
## [1] "SPL3SMP_E.004 is Enhanced L3 Radiometer Soil Moisture from NSIDC DAAC"
## [1] "SPL3SMP.007 is Soil Moisture from NSIDC DAAC"
## [1] "SPL3FTP.003 is Freeze/Thaw State from NSIDC DAAC"
## [1] "SRTMGL1_NC.003 is Elevation (DEM) from LP DAAC"
## [1] "SRTMGL1_NUMNC.003 is Source (DEM) from LP DAAC"
## [1] "SRTMGL3_NC.003 is Elevation (DEM) from LP DAAC"
## [1] "SRTMGL3_NUMNC.003 is Source (DEM) from LP DAAC"
## [1] "ASTGTM_NC.003 is Elevation from LP DAAC"
## [1] "ASTGTM_NUMNC.003 is Source from LP DAAC"
## [1] "ASTWBD_ATTNC.001 is Water Bodies Database Attributes from LP DAAC"
## [1] "ASTWBD_NC.001 is Water Bodies Database Elevation from LP DAAC"
## [1] "VNP09H1.001 is Surface Reflectance from LP DAAC"
## [1] "VNP09A1.001 is Surface Reflectance from LP DAAC"
## [1] "VNP09GA.001 is Surface Reflectance from LP DAAC"
## [1] "VNP13A1.001 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "VNP13A2.001 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "VNP13A3.001 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "VNP14A1.001 is Thermal Anomalies/Fire from LP DAAC"
## [1] "VNP15A2H.001 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "VNP21A1D.001 is Land Surface Temperature & Emissivity Day (LST&E) from LP DAAC"
## [1] "VNP21A1N.001 is Land Surface Temperature & Emissivity Night (LST&E) from LP DAAC"
## [1] "VNP21A2.001 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "VNP22Q2.001 is Global Land Surface Phenology (GLSP) from LP DAAC"
## [1] "VNP43IA1.001 is BRDF-Albedo Model Parameters from LP DAAC"
## [1] "VNP43IA2.001 is BRDF-Albedo Quality from LP DAAC"
## [1] "VNP43IA3.001 is Albedo (BRDF) from LP DAAC"
## [1] "VNP43IA4.001 is Nadir BRDF-Adjusted Reflectance from LP DAAC"
## [1] "VNP43MA1.001 is BRDF-Albedo Model Parameters from LP DAAC"
## [1] "VNP43MA2.001 is BRDF-Albedo Quality from LP DAAC"
## [1] "VNP43MA3.001 is Albedo (BRDF) from LP DAAC"
## [1] "VNP43MA4.001 is Nadir BRDF-Adjusted Reflectance from LP DAAC"
## [1] "DAYMET.004 is Daily Surface Weather Data for North America from ORNL"
## [1] "ECO2LSTE.001 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "ECO2CLD.001 is Cloud Mask from LP DAAC"
## [1] "ECO3ETPTJPL.001 is Evapotranspiration PT-JPL from LP DAAC"
## [1] "ECO3ANCQA.001 is L3/L4 Ancillary Data Quality Assurance (QA) Flags from LP DAAC"
## [1] "ECO4ESIPTJPL.001 is Evaporative Stress Index PT-JPL from LP DAAC"
## [1] "ECO4WUE.001 is Water Use Efficiency from LP DAAC"
## [1] "ECO1BGEO.001 is Geolocation from LP DAAC"
## [1] "ECO1BMAPRAD.001 is Resampled Radiance from LP DAAC"
## [1] "ECO3ETALEXI.001 is Evapotranspiration dis-ALEXI from LP DAAC"
## [1] "ECO4ESIALEXI.001 is Evaporative Stress Index dis-ALEXI from LP DAAC"
The product service provides many useful details, including if a product is currently available in AppEEARS, a description, and information on the spatial and temporal resolution. Below, the product details are retrieved using 'ProductAndVersion'.
# Convert the MCD15A3H.006 info to JSON object and print the prettified info
prettify(toJSON(products$"MCD15A3H.006"))
## [
## {
## "Product": "MCD15A3H",
## "Platform": "Combined MODIS",
## "Description": "Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR)",
## "RasterType": "Tile",
## "Resolution": "500m",
## "TemporalGranularity": "4 day",
## "Version": "006",
## "Available": true,
## "DocLink": "https://doi.org/10.5067/MODIS/MCD15A3H.006",
## "Source": "LP DAAC",
## "TemporalExtentStart": "2002-07-04",
## "TemporalExtentEnd": "Present",
## "Deleted": false,
## "DOI": "10.5067/MODIS/MCD15A3H.006",
## "ProductAndVersion": "MCD15A3H.006"
## }
## ]
##
Also, the products can be searched using their description. Below, search for products containing Leaf Area Index in their description and make a list of their productAndVersion.
LAI_Products <- list() # Create an empty list
for (p in products){ # Loop through the product list
if (grepl('Leaf Area Index', p$Description )){ # Look through the product description for a keyword
LAI_Products <- append(LAI_Products, p$ProductAndVersion) # Append the LAI products to the list
}
}
LAI_Products
## [[1]]
## [1] "MCD15A2H.006"
##
## [[2]]
## [1] "MCD15A3H.006"
##
## [[3]]
## [1] "MOD15A2H.006"
##
## [[4]]
## [1] "MOD15A2H.061"
##
## [[5]]
## [1] "MYD15A2H.006"
##
## [[6]]
## [1] "MYD15A2H.061"
##
## [[7]]
## [1] "VNP15A2H.001
Using the info above, Create a list of desired products.
desired_products <- c('MCD15A3H.006','MOD11A2.061') # Create a vector of desired products
desired_products
"MCD15A3H.006" "MOD11A2.061"
2b. Search and Explore Available Layers
This API call will list all of the layers available for a given product. Each product is referenced by its ProductAndVersion property which is also referred to as the product_id. First, request the layers for the MCD15A3H.006 product.
# Request layers for the 1st product in the list: MCD15A3H.006
MCD15A3H_req <- GET(paste0(API_URL,"product/", desired_products[1])) # Request the info of a product from product URL
MCD15A3H_content <- content(MCD15A3H_req) # Retrieve content of the request
MCD15A3H_response <- toJSON(MCD15A3H_content, auto_unbox = TRUE) # Convert the content to JSON object
remove(MCD15A3H_req, MCD15A3H_content) # Remove the variables that are not needed anymore
#prettify(MCD15A3H_response) # Print the prettified response
names(fromJSON(MCD15A3H_response)) # print the layer's names
[1] "FparExtra_QC" "FparLai_QC" "FparStdDev_500m" "Fpar_500m"
[5] "LaiStdDev_500m" "Lai_500m"
Next, request the layers for the MOD11A2.061 product.
MOD11_req <- GET(paste0(API_URL,"product/", desired_products[2])) # Request the info of a product from product URL
MOD11_content <- content(MOD11_req) # Retrieve content of the request
MOD11_response <- toJSON(MOD11_content, auto_unbox = TRUE) # Convert the content to JSON object
remove(MOD11_req, MOD11_content) # Remove the variables that are not needed anymore
names(fromJSON(MOD11_response)) # print the layer names
Lastly, select the desired layers and pertinent products and make a data frame using this information. This data frame will be inserted into the nested data frame that will be used to create a JSON object to submit a request in Section 3.
[1] "Clear_sky_days" "Clear_sky_nights" "Day_view_angl" "Day_view_time"
[5] "Emis_31" "Emis_32" "LST_Day_1km" "LST_Night_1km"
[9] "Night_view_angl" "Night_view_time" "QC_Day" "QC_Night"
desired_layers <- c("LST_Day_1km","LST_Night_1km","Lai_500m") # Create a vector of desired layers
desired_prods <- c("MOD11A2.061","MOD11A2.061","MCD15A3H.006") # Create a vector of products including the desired layers
# Create a data frame including the desired data products and layers
layers <- data.frame(product = desired_prods, layer = desired_layers)
3. Submit a Point Request
The Submit task API call provides a way to submit a new request to be processed. It can accept data via JSON or query string. In the example below, create a JSON object and submit a request. Tasks in AppEEARS correspond to each request associated with your user account. Therefore, each of the calls to this service requires an authentication token (see Section 1c.).
3a. Compile a JSON Object
In this section, begin by setting up the information needed for a nested data frame that will be later converted to a JSON object for submitting an AppEEARS point request. For detailed information on required JSON parameters, see the API Documentation.
For point requests, beside the date range and desired layers information, the coordinates property must also be inside the task object. Optionally, set id and category properties to further identify your selected coordinates.
startDate <- "01-01-2018" # Start of the date range for which to extract data: MM-DD-YYYY
endDate <- "12-31-2018" # End of the date range for which to extract data: MM-DD-YYYY
# If you are interested in submitting a request using a recurring time in a year, set the time variables as below:
# startDate <- '05-01' # change start/end date to MM-DD
# endDate <- '06-30'
# recurring <- TRUE # Specify True for a recurring date range
# fromYear <- 2018
# toYear <- 2020
lat <- c(36.206228, 37.289327) # Latitude of the point sites
lon <- c(-112.127134, -112.973760) # Longitude of the point sites
id <- c("0","1") # ID for the point sites
category <- c("Grand Canyon", "Zion") # Category for point sites
taskName <- 'NPS Vegetation' # Enter name of the task, 'NPS Vegetation' used here
taskType <- 'point' # Specify the task type, it can be either "area" or "point"
To be able to successfully submit a task, the JSON object should be structured in a certain way. The code chunk below uses the information from the previous chunk to create a nested data frame. This nested data frame will be converted to JSON object that can be used to complete the request.
# Create a data frame including the date range for the request
date <- data.frame(startDate = startDate, endDate = endDate)
# If you set the recurring to TRUE
# date <- data.frame(startDate = startDate, endDate = endDate , recurring = recurring)
# date$yearRange <- list(c(fromYear,toYear))
# Create a data frame including lat and long coordinates. ID and category name is optional.
coordinates <- data.frame(id = id, longitude = lon, latitude = lat, category = category)
task_info <- list(date,layers, coordinates) # Create a list of data frames
names(task_info) <- c("dates", "layers", "coordinates") # Assign names
task <- list(task_info, taskName, taskType) # Create a nested list
names(task) <- c("params", "task_name", "task_type") # Assign names
remove(date, layers, coordinates, task_info) # Remove the variables that are not needed anymore
toJSON function from jsonlite package converts the type of data frame to a string that can be recognized as a JSON object to be submitted as a point request.
task_json <- toJSON(task,auto_unbox = TRUE) # Convert to JSON object
3b. Submit a Task Request
Token information is needed to submit a request. Below the login token is assigned to a variable.
token <- paste("Bearer", fromJSON(token_response)$token) # Save login token to a variable
Below, post a call to the API task service, using the task_json created above.
# Post the point request to the API task service
response <- POST(paste0(API_URL, "task"),
body = task_json ,
encode = "json",
add_headers(Authorization = token, "Content-Type" = "application/json"))
task_content <- content(response) # Retrieve content of the request
task_response <- prettify(toJSON(task_content, auto_unbox = TRUE))# Convert the content to JSON object
remove(response, task_content) # Remove the variables that are not needed anymore
task_response # Print the prettified task response
{
"task_id": "0dd57143-d242-46f7-b608-e31e783b7070",
"status": "pending"
}
3c. Retrieve Task Status
This API call will list all of the requests associated with your user account, automatically sorted by date descending with the most recent requests listed first. The AppEEARS API contains some helpful formatting resources. Below, limit the API response to 2 entries for the last 2 requests and set pretty to True to format the response as an organized JSON object to make it easier to read. Additional information on AppEEARS API retrieve task, pagination, and formatting can be found in the API documentation.
params <- list(limit = 2, pretty = TRUE) # Set up query parameters
# Request the task status of last 2 requests from task URL
response_req <- GET(paste0(API_URL,"task"), query = params, add_headers(Authorization = token))
response_content <- content(response_req) # Retrieve content of the request
status_response <- toJSON(response_content, auto_unbox = TRUE) # Convert the content to JSON object
remove(response_req, response_content) # Remove the variables that are not needed anymore
prettify(status_response) # Print the prettified response
## [
## {
## "params": {
## "dates": [
## {
## "endDate": "12-31-2018",
## "startDate": "01-01-2018"
## }
## ],
## "layers": [
## {
## "layer": "LST_Day_1km",
## "product": "MOD11A2.061"
## },
## {
## "layer": "LST_Night_1km",
## "product": "MOD11A2.061"
## },
## {
## "layer": "Lai_500m",
## "product": "MCD15A3H.006"
## }
## ]
## },
## "status": "pending",
## "created": "2022-04-06T03:47:54.947711",
## "task_id": "0dd57143-d242-46f7-b608-e31e783b7070",
## "updated": "2022-04-06T03:47:55.124910",
## "user_id": "user@email.com",
## "estimate": {
## "request_size": 272
## },
## "has_swath": false,
## "task_name": "NPS Vegetation",
## "task_type": "point",
## "api_version": "v1",
## "svc_version": "3.1",
## "web_version": {
##
## },
## "has_nsidc_daac": false,
## "expires_on": "2022-05-06T03:47:55.124910"
## },
## {
## "params": {
## "dates": [
## {
## "endDate": "12-31-2019",
## "startDate": "01-01-2018"
## }
## ],
## "layers": [
## {
## "layer": "LST_Day_1km",
## "product": "MOD11A2.061"
## },
## {
## "layer": "LST_Night_1km",
## "product": "MOD11A2.061"
## },
## {
## "layer": "Lai_500m",
## "product": "MCD15A3H.006"
## },
## {
## "layer": "SRTMGL1_DEM",
## "product": "SRTMGL1_NC.003"
## }
## ],
## "output": {
## "format": {
## "type": "geotiff"
## },
## "projection": "geographic"
## }
## },
## "status": "pending",
## "created": "2022-04-06T03:13:04.825063",
## "task_id": "a359d60a-f877-4355-9f47-ace74656a9f0",
## "updated": "2022-04-06T03:13:14.813119",
## "user_id": "user@email.com",
## "estimate": {
## "request_size": 368451761.6404
## },
## "has_swath": false,
## "task_name": "Grand Canyon",
## "task_type": "area",
## "api_version": "v1",
## "svc_version": "3.1",
## "web_version": {
##
## },
## "has_nsidc_daac": false,
## "expires_on": "2022-05-06T03:13:14.813119"
## }
## ]
##
The task_id that was generated when submitting your request can also be used to retrieve a task status.
task_id <- fromJSON(task_response)$task_id # Extract the task_id of submitted point request
# Request the task status of a task with the provided task_id from task URL
status_req <- GET(paste0(API_URL,"task/", task_id), add_headers(Authorization = token))
status_content <- content(status_req) # Retrieve content of the request
statusResponse <-toJSON(status_content, auto_unbox = TRUE) # Convert the content to JSON object
stat <- fromJSON(statusResponse)$status # Assign the task status to a variable
remove(status_req, status_content) # Remove the variables that are not needed anymore
prettify(statusResponse) # Print the prettified response
## {
## "params": {
## "dates": [
## {
## "endDate": "12-31-2018",
## "startDate": "01-01-2018"
## }
## ],
## "layers": [
## {
## "layer": "LST_Day_1km",
## "product": "MOD11A2.061"
## },
## {
## "layer": "LST_Night_1km",
## "product": "MOD11A2.061"
## },
## {
## "layer": "Lai_500m",
## "product": "MCD15A3H.006"
## }
## ],
## "coordinates": [
## {
## "id": "0",
## "category": "Grand Canyon",
## "latitude": 36.2062,
## "longitude": -112.1271
## },
## {
## "id": "1",
## "category": "Zion",
## "latitude": 37.2893,
## "longitude": -112.9738
## }
## ]
## },
## "status": "pending",
## "created": "2022-04-06T03:47:54.947711",
## "task_id": "0dd57143-d242-46f7-b608-e31e783b7070",
## "updated": "2022-04-06T03:47:55.124910",
## "user_id": "use@email.com",
## "estimate": {
## "request_size": 272
## },
## "has_swath": false,
## "task_name": "NPS Vegetation",
## "task_type": "point",
## "api_version": "v1",
## "svc_version": "3.1",
## "web_version": {
##
## },
## "has_nsidc_daac": false,
## "expires_on": "2022-05-06T03:47:55.124910"
## }
##
Retrieve the task status every 5 seconds. The task status should be done to be able to download the output.
while (stat != 'done') {
Sys.sleep(5)
# Request the task status and retrieve content of request from task URL
stat_content <- content(GET(paste0(API_URL,"task/", task_id), add_headers(Authorization = token)))
stat <-fromJSON(toJSON(stat_content, auto_unbox = TRUE))$status # Get the status
remove(stat_content)
print(stat)
}
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "done"
4. Download a Request
4a. Explore Files in Request Output
Before downloading the request output, examine the files contained in the request output.
# Request the task bundle info from API bundle URL
response <- GET(paste0(API_URL, "bundle/", task_id), add_headers(Authorization = token))
response_content <- content(response) # Retrieve content of the request
bundle_response <- toJSON(response_content, auto_unbox = TRUE) # Convert the content to JSON object
prettify(bundle_response) # Print the prettified response
## {
## "files": [
## {
## "sha256": "4704c2b1563780ce3a6b9d5a0532e850865bf7df0266158fc8abe2ab9f0536d7",
## "file_id": "15815f7e-110b-45c0-924e-06b1c5e43b8c",
## "file_name": "NPS-Vegetation-MCD15A3H-006-results.csv",
## "file_size": 104038,
## "file_type": "csv"
## },
## {
## "sha256": "2a6825d3ca77b8c00c9a33c5dbd8956f7fa33e3397b03a771288995138f5bf0b",
## "file_id": "66b96600-96e5-4b49-83c7-feb1412830fb",
## "file_name": "NPS-Vegetation-MOD11A2-061-results.csv",
## "file_size": 42948,
## "file_type": "csv"
## },
## {
## "sha256": "8ba6ca914fb19e42cfb2a2dc4a3ac415cd9d6346fc4e49c1fe9dcc59ce0d819e",
## "file_id": "da09564d-19ee-478c-a273-e9dee144995a",
## "file_name": "NPS-Vegetation-granule-list.txt",
## "file_size": 32008,
## "file_type": "txt"
## },
## {
## "sha256": "09931fe16f995c743dc46bd0b40860cd237c13302fb90682e49a1f9d98ac2d72",
## "file_id": "836e0149-aed9-4642-af10-bf3c1c2adf5b",
## "file_name": "NPS-Vegetation-request.json",
## "file_size": 935,
## "file_type": "json"
## },
## {
## "sha256": "77d5f403d970ea5848bef71dadf7bdf5a8807f692ebb7743347d81f4af04e168",
## "file_id": "d96fe995-2741-47f9-b575-7538d6cc96c0",
## "file_name": "NPS-Vegetation-MCD15A3H-006-metadata.xml",
## "file_size": 17237,
## "file_type": "xml"
## },
## {
## "sha256": "1eb0396add64eda3b5b421428420cd723d02467b40db28873a27f5c0e535a443",
## "file_id": "039dba16-73ea-42ae-bc45-23f3c8bf825e",
## "file_name": "NPS-Vegetation-MOD11A2-061-metadata.xml",
## "file_size": 17191,
## "file_type": "xml"
## },
## {
## "sha256": "19c972972442be97608914a6f862fce499ba6915bda479901b2e8b7ad8f3d32c",
## "file_id": "f0ef69fb-0175-43cb-8aa0-7bff8ea4bc39",
## "file_name": "README.md",
## "file_size": 18785,
## "file_type": "txt"
## }
## ],
## "created": "2022-04-06T03:51:08.622651",
## "task_id": "0dd57143-d242-46f7-b608-e31e783b7070",
## "updated": "2022-04-06T03:51:57.033457",
## "bundle_type": "point"
## }
##
4b. Download Files in a Request (Automation)
The bundle API provides information about completed tasks. For any completed task, a bundle can be queried to return the files contained as a part of the task request. Below, call the bundle API and return all of the output files. Next, read the contents of the bundle in JSON format and loop through file_id to automate downloading all of the output files into the output directory. For more information, please see AppEEARS API Documentation.
bundle <- fromJSON(bundle_response)$files
for (id in bundle$file_id){
# retrieve the filename from the file_id
filename <- bundle[bundle$file_id == id,]$file_name
# create a destination directory to store the file in
filepath <- paste(outDir,filename, sep = "/")
suppressWarnings(dir.create(dirname(filepath)))
# write the file to disk using the destination directory and file name
response <- GET(paste0(API_URL, "bundle/", task_id, "/", id),
write_disk(filepath, overwrite = TRUE),
progress(),
add_headers(Authorization = token))
}
5. Explore AppEEARS Quality Service
The quality API provides quality details about all of the data products available in AppEEARS. Below are examples of how to query the quality API for listing quality products, layers, and values. The final example (Section 5c.) demonstrates how AppEEARS quality services can be leveraged to decode pertinent quality values for your data. For more information visit AppEEARS API documentation.
First, reset pagination to include offset which allows you to set the number of results to skip before starting to return entries. Next, make a call to list all of the data product layers and the associated quality product and layer information.
params <- list(limit = 6, offset = 20, pretty = TRUE) # Set up the query parameters
q_req <- GET(paste0(API_URL, "quality"), query = params) # Request the quality info from quality API_URL
q_content <- content(q_req) # Retrieve the content of request
q_response <- toJSON(q_content, auto_unbox = TRUE) # Convert the info to JSON object
remove(params, q_req, q_content) # Remove the variables that are not needed
prettify(q_response) # Print the prettified quality information
## [
## {
## "ProductAndVersion": "CU_LT05.001",
## "Layer": "SRB1",
## "QualityProductAndVersion": "CU_LT05.001",
## "QualityLayers": [
## "PIXELQA"
## ],
## "Continuous": false,
## "VisibleToWorker": true
## },
## {
## "ProductAndVersion": "CU_LT05.001",
## "Layer": "SRB2",
## "QualityProductAndVersion": "CU_LT05.001",
## "QualityLayers": [
## "PIXELQA"
## ],
## "Continuous": false,
## "VisibleToWorker": true
## },
## {
## "ProductAndVersion": "CU_LT05.001",
## "Layer": "SRB3",
## "QualityProductAndVersion": "CU_LT05.001",
## "QualityLayers": [
## "PIXELQA"
## ],
## "Continuous": false,
## "VisibleToWorker": true
## },
## {
## "ProductAndVersion": "CU_LT05.001",
## "Layer": "SRB4",
## "QualityProductAndVersion": "CU_LT05.001",
## "QualityLayers": [
## "PIXELQA"
## ],
## "Continuous": false,
## "VisibleToWorker": true
## },
## {
## "ProductAndVersion": "CU_LT05.001",
## "Layer": "SRB5",
## "QualityProductAndVersion": "CU_LT05.001",
## "QualityLayers": [
## "PIXELQA"
## ],
## "Continuous": false,
## "VisibleToWorker": true
## },
## {
## "ProductAndVersion": "CU_LT05.001",
## "Layer": "SRB7",
## "QualityProductAndVersion": "CU_LT05.001",
## "QualityLayers": [
## "PIXELQA"
## ],
## "Continuous": false,
## "VisibleToWorker": true
## }
## ]
##
5a. List Quality Layers
This API call will list all of the quality layer information for a product. For more information visit AppEEARS API documentation
productAndVersion <- 'MCD15A3H.006' # Assign ProductAndVersion to a variable
# Request the quality info from quality API for a specific product
MCD15A3H_req <- GET(paste0(API_URL, "quality/", productAndVersion))
MCD15A3H_content <- content(MCD15A3H_req) # Retrieve the content of request
MCD15A3H_quality <- toJSON(MCD15A3H_content, auto_unbox = TRUE)# Convert the info to JSON object
remove(MCD15A3H_req, MCD15A3H_content) # Remove the variables that are not needed anymore
prettify(MCD15A3H_quality) # Print the prettified quality information
## [
## {
## "ProductAndVersion": "MCD15A3H.006",
## "Layer": "Fpar_500m",
## "QualityProductAndVersion": "MCD15A3H.006",
## "QualityLayers": [
## "FparLai_QC",
## "FparExtra_QC"
## ],
## "VisibleToWorker": true
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "Layer": "FparStdDev_500m",
## "QualityProductAndVersion": "MCD15A3H.006",
## "QualityLayers": [
## "FparLai_QC",
## "FparExtra_QC"
## ],
## "VisibleToWorker": true
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "Layer": "Lai_500m",
## "QualityProductAndVersion": "MCD15A3H.006",
## "QualityLayers": [
## "FparLai_QC",
## "FparExtra_QC"
## ],
## "VisibleToWorker": true
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "Layer": "LaiStdDev_500m",
## "QualityProductAndVersion": "MCD15A3H.006",
## "QualityLayers": [
## "FparLai_QC",
## "FparExtra_QC"
## ],
## "VisibleToWorker": true
## }
## ]
##
5b. Show Quality Values
This API call will list all of the values for a given quality layer.
quality_layer <- 'FparLai_QC' # assign a quality layer to a variable
# Request the specified quality layer info from quality API
quality_req <- GET(paste0(API_URL, "quality/", productAndVersion, "/", quality_layer, sep = ""))
quality_content <- content(quality_req) # Retrieve the content of request
quality_response <- toJSON(quality_content, auto_unbox = TRUE) # Convert the info to JSON object
remove(quality_req, quality_content) # Remove the variables that are not needed
prettify(quality_response) # Print the quality response as a data frame
## [
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "MODLAND",
## "Value": 0,
## "Description": "Good quality (main algorithm with or without saturation)",
## "Acceptable": true
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "MODLAND",
## "Value": 1,
## "Description": "Other Quality (back-up algorithm or fill values)",
## "Acceptable": false
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "Sensor",
## "Value": 0,
## "Description": "Terra",
## "Acceptable": {
##
## }
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "Sensor",
## "Value": 1,
## "Description": "Aqua",
## "Acceptable": {
##
## }
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "DeadDetector",
## "Value": 0,
## "Description": "Detectors apparently fine for up to 50% of channels 1, 2",
## "Acceptable": {
##
## }
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "DeadDetector",
## "Value": 1,
## "Description": "Dead detectors caused >50% adjacent detector retrieval",
## "Acceptable": {
##
## }
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "CloudState",
## "Value": 0,
## "Description": "Significant clouds NOT present (clear)",
## "Acceptable": {
##
## }
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "CloudState",
## "Value": 1,
## "Description": "Significant clouds WERE present",
## "Acceptable": {
##
## }
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "CloudState",
## "Value": 2,
## "Description": "Mixed cloud present in pixel",
## "Acceptable": {
##
## }
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "CloudState",
## "Value": 3,
## "Description": "Cloud state not defined, assumed clear",
## "Acceptable": {
##
## }
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "SCF_QC",
## "Value": 0,
## "Description": "Main (RT) method used, best result possible (no saturation)",
## "Acceptable": {
##
## }
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "SCF_QC",
## "Value": 1,
## "Description": "Main (RT) method used with saturation. Good, very usable",
## "Acceptable": {
##
## }
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "SCF_QC",
## "Value": 2,
## "Description": "Main (RT) method failed due to bad geometry, empirical algorithm used",
## "Acceptable": {
##
## }
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "SCF_QC",
## "Value": 3,
## "Description": "Main (RT) method failed due to problems other than geometry, empirical algorithm used",
## "Acceptable": {
##
## }
## },
## {
## "ProductAndVersion": "MCD15A3H.006",
## "QualityLayer": "FparLai_QC",
## "Name": "SCF_QC",
## "Value": 4,
## "Description": "Pixel not produced at all, value couldn't be retrieved (possible reasons: bad L1B data, unusable MOD09GA data)",
## "Acceptable": {
##
## }
## }
## ]
##
5c. Decode Quality Values
This API call will decode the bits for a given quality value.
quality_value <- 1 # Assign a quality value to a variable
# Request and retrieve information for provided quality value from quality API URL
response <- content(GET(paste0(API_URL, "quality/", productAndVersion, "/", quality_layer, "/", quality_value)))
q_response <- toJSON(response, auto_unbox = TRUE) # Convert the info to JSON object
remove(response) # Remove the variables that are not needed anymore
prettify(q_response) # Print the prettified response
## {
## "Binary Representation": "0b00000001",
## "MODLAND": {
## "bits": "0b1",
## "description": "Other Quality (back-up algorithm or fill values)"
## },
## "Sensor": {
## "bits": "0b0",
## "description": "Terra"
## },
## "DeadDetector": {
## "bits": "0b0",
## "description": "Detectors apparently fine for up to 50% of channels 1, 2"
## },
## "CloudState": {
## "bits": "0b00",
## "description": "Significant clouds NOT present (clear)"
## },
## "SCF_QC": {
## "bits": "0b000",
## "description": "Main (RT) method used, best result possible (no saturation)"
## }
## }
##
6. BONUS: Load Request Output and Visualize
Here, load the CSV file containing the results from your request using readr package, and create some basic visualizations using the ggplot2 package.
6a. Load a CSV
Use the readr package to load the CSV file containing the results from the AppEEARS request.
# Make a list of csv files in the output directory
files <- list.files(outDir, pattern = "\\MOD11A2-061-results.csv$")
# Read the MOD11A2 results
df <- read_csv(paste0(outDir,"/", files), show_col_types = FALSE)
Select the MOD11A2.061 LST Day column for the data from Grand Canyon National Park using dplyr package.
lst_GC <- df %>%
# Filter df for the point from GC
filter(Category == "Grand Canyon") %>%
# Select desired columns
select(Latitude, Longitude, Date ,MOD11A2_061_LST_Day_1km, MOD11A2_061_LST_Night_1km)
Extract information for LST_DAY_1KM from MOD11_response of product service call from earlier in the tutorial.
#fromJSON(MOD11_response)$LST_Day_1km # Extract all the info for LST_Day_1km layer
fillValue <- fromJSON(MOD11_response)$LST_Day_1km$FillValue # Assign fill value to a variable
unit <- fromJSON(MOD11_response)$LST_Day_1km$Units # Assign unit to a variable
sprintf("Fill value for LST_DAY_1KM is: %i" ,fillValue) # Print LST_DAY_1KM fill value
sprintf("Unit for LST_DAY_1KM is: %s" ,unit) # Print LST_DAY_1KM unit
## [1] "Fill value for LST_DAY_1KM is: 0"
## [1] "Unit for LST_DAY_1KM is: Kelvin"
6b. Plot Results (Line/Scatter Plots)
Next, plot a time series of daytime LST for the selected point in Grand Canyon National Park for 2018. Below, filter the LST data to exclude fill values.
lst_GC <- lst_GC %>%
# exclude NoData
filter(MOD11A2_061_LST_Day_1km != fillValue)%>%
filter(MOD11A2_061_LST_Night_1km != fillValue)
Next, plot LST Day as a time series with some additional formatting using ggplot2.
ggplot(lst_GC)+
geom_line(aes(x= Date, y = MOD11A2_061_LST_Day_1km), size=1, color="blue")+
geom_point(aes(x= Date, y = MOD11A2_061_LST_Day_1km), shape=18 , size = 3, color="blue")+
labs(title = "Time Series",
x = "Date",
y = sprintf( "LST_Day_1km (%s)", unit))+
scale_x_date(date_breaks = "16 day", limits = as.Date(c('2018-01-01','2019-01-01')))+
scale_y_continuous(limits = c(250, 325), breaks = seq(250, 325, 10))+
theme(plot.title = element_text(face = "bold",size = rel(2.5),hjust = 0.5),
axis.title = element_text(face = "bold",size = rel(1)),
panel.background = element_rect(fill = "lightgray", colour = "black"),
axis.text.x = element_text(face ="bold",color="black", angle= 315 , size = 10),
axis.text.y = element_text(face ="bold",color="black", angle= 0, size = 10)
)
Using the tidyr package, the LST Day and Night values for Grand Canyon NP are being gathered in a single column to be used to make a plot including both LST_Day_1km and LST_Night_1km.
lst_GC_DN <- tidyr::gather(lst_GC, key = Tstat , value = LST, MOD11A2_061_LST_Day_1km, MOD11A2_061_LST_Night_1km)
lst_GC_DN[1:5,] # print the five first observations
Next, plot LST Day and Night as a time series with some additional formatting.
ggplot(lst_GC_DN)+
geom_line(aes(x= Date, y = LST, color = Tstat), size=1)+
geom_point(aes(x= Date, y = LST, color = Tstat), shape=18 , size = 3)+
scale_fill_manual(values=c("red", "blue"))+
scale_color_manual(values=c('red','blue'))+
labs(title = "Time Series",
x = "Date",
y = sprintf( "LST_Day_1km (%s)",unit))+
scale_x_date(date_breaks = "16 day", limits = as.Date(c('2018-01-01','2019-01-01')))+
scale_y_continuous(limits = c(250, 325), breaks = seq(250, 325, 10))+
theme(plot.title = element_text(face = "bold",size = rel(2.5), hjust = 0.5),
axis.title = element_text(face = "bold",size = rel(1)),
panel.background = element_rect(fill = "lightgray", colour = "black"),
axis.text.x = element_text(face ="bold",color="black", angle= 315 , size = 10),
axis.text.y = element_text(face ="bold",color="black", angle= 0, size = 10),
legend.position = "bottom",
legend.title = element_blank()
)
Finally, bring in the daytime LST data from Zion National Park, and compare with daytime LST at Grand Canyon National Park, shown below in a scatter plot using ggplot2 package.
Here, the dplyr is used to extract the LST_DAY_1km for Zion National Park.
lst_Z <- df %>%
filter(MOD11A2_061_LST_Day_1km != fillValue) %>% # Filter fill value
filter(Category == "Zion")%>% # Filter Zion national park
select(Date, MOD11A2_061_LST_Day_1km) # Select desired columns
Make a scatter plot.
ggplot()+
geom_point(aes(x=lst_Z$MOD11A2_061_LST_Day_1km, y=lst_GC$MOD11A2_061_LST_Day_1km), shape=18 , size = 3, color="blue")+
labs(title = "MODIS LST: Grand Canyon vs. Zion National Park, 2018",
x = sprintf("Zion: LST_Day_1km (%s)",unit),
y = sprintf( "Grand Canyon: LST_Day_1km (%s)",unit))+
theme(plot.title = element_text(face = "bold",size = rel(1.5), hjust = 0.5),
axis.title = element_text(face = "bold",size = rel(1)),
panel.background = element_rect(fill = "lightgray", colour = "black"),
axis.text.x = element_text(face ="bold",color="black", size = 10),
axis.text.y = element_text(face ="bold",color="black", size = 10)
)
This example can provide a template to use for your own research workflows. Leveraging the AppEEARS API for searching, extracting, and formatting analysis ready data, and loading it directly into R means that you can keep your entire research workflow in a single software program, from start to finish.
## Contact Information
Material written by Mahsa Jami1 and Cole Krehbiel1
Contact: LPDAAC@usgs.gov
Voice: +1-866-573-3222
Organization: Land Processes Distributed Active Archive Center (LP DAAC)
Website: https://lpdaac.usgs.gov/
Date last modified: 06-12-2020
1 KBR, Inc., contractor to the U.S. Geological Survey, Earth Resources Observation and Science (EROS) Center,
Sioux Falls, South Dakota, USA. Work performed under USGS contract G15PD00467 for LP DAAC2.
2 LP DAAC Work performed under NASA contract NNG14HH33I.