How to Access LP DAAC Data from the Command Line

Details:

Published: Feb. 7, 2019

All publicly available Earth observation data from NASA’s Land Processes Distributed Active Archive Center (LP DAAC) are available online via Hypertext Transfer Protocol Secure (HTTPS). However, NASA Earthdata login credentials are required to download the data. Users looking to download LP DAAC data directly via a command line utility will need to pass their Earthdata login username and password to do so. This document demonstrates how to set up a .netrc file or a .wgetrc file to store your NASA Earthdata login username and password, which can be used to authenticate your credentials when attempting to download LP DAAC data from the command line. Additionally, if you are looking to download data directly in R or Python, be sure to check out the scripts provided in How to Access the LP DAAC Data Pool with R and How to Access the LP DAAC Data Pool with Python.

Two common command line utilities for downloading data over the internet are curl and wget. These tools each have different command line arguments and parameters that will need to be passed to them to authenticate your NASA Earthdata login credentials when downloading data from the LP DAAC.

The instructions below demonstrate how to set up a .netrc file, for use with curl, or a .wgetrc file, for use with wget, to successfully download a file from the LP DAAC via HTTPS.

  1. Set up a .netrc file in your home directory

    1. .netrc file format:
      1. machine urs.earthdata.nasa.gov
      2. login
      3. password
      4. Template .netrc file
    2. Windows:
      1. In the Command Prompt, enter the following:
        1. echo %userprofile%
      2. This should return the home directory on your OS:
        1. Ex: C:\Users\jdoe
        2. If it does not return a directory, try replacing %userprofile% with %home% (above and below)
      3. to create a .netrc file, enter the following in the Command Prompt:
        1. type NUL >> %userprofile%\.netrc | echo machine urs.earthdata.nasa.gov >> %userprofile%\.netrc
      4. To insert your NASA Earthdata login username and password into the file, enter the following in the Command Prompt:
        1. echo login REPLACEWITHUSERNAME >> %userprofile%\.netrc | echo password REPLACEWITHPASSWORD >> %userprofile%\.netrc
    3. MacOS:
      1. To create a .netrc file, enter the following in Terminal:
        1. touch ~/.netrc | chmod og-rw ~/.netrc | echo machine urs.earthdata.nasa.gov >> ~/.netrc
  2. Download LP DAAC data from the command line

    1. Using curl
      1. Install the latest version of curl
        1. This documentation was tested using curl version 7.61.0
      2. Make sure that you have successfully completed Step 1 (create a .netrc file) You should now be able to run curl using the command below:
        1. curl -O -b ~/.urs_cookies -c ~/.urs_cookies -L -n https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2013.01.09/MYD13Q1.A2013009.h09v07.006.2015254175244.hdf
    2. Using wget
      1. Install the latest version of wgetrc
        1. This documentation was tested using Wget version 1.19.1
      2. If you are running MacOS, Linux, or Cmder [on Windows]
        1. Navigate to your home directory
        2. continue to step 4
      3. If you are running Windows Command Prompt
        1. Navigate to the directory (in command prompt using cd) containing the wget.exe executable file
          1. Tip: try where wget in the command prompt to find the directory
          2. Continue to step 4
      4. Create a .wgetrc file in this directory by entering the following into Command Prompt:
        1. MacOS, Linux, Cmder:
          1. touch .wgetrc | chmod og-rw .wgetrc
        2. Windows:
          1. type NUL >> .wgetrc
        3. To insert your NASA Earthdata login username and password into the file, enter the following in the Command Prompt:
          1. echo http-user=REPLACEWITHUSERNAME >> .wgetrc | echo http-password=REPLACEWITHPASSWORD >> .wgetrc
      5. You should now be able to run wget command to download data directly from the LP DAAC data pool:
        1. wget https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2013.01.09/MYD13Q1.A2013009.h09v07.006.2015254175244.hdf
        2. wget -i filelistcontainingmultipleURLs.txt

Relevant Products

Product Long Name

Tools

Name Filters Description
Data Pool Direct Download

The Data Pool is the publicly available portion of the LP DAAC online h...

Data Prep Scripts Direct Download, Web Service

This collection of R and Python scripts can be used to download data an...