NASA's Earth Science Data Systems (ESDS) Program evolves continuously through a combination of competitive awards and sustained and strategic investments.
- Earth Science Open Data and Open Source Policy
- Cloud Evolution
- Interagency Implementation and Advanced Concepts Team (IMPACT)
- Competitive Programs
- Science Tools and Portals
- Data Access Systems
- Earth Science Data System Working Groups (ESDSWG)
- Program Review
For more than half a century, NASA Earth observing data have been an invaluable resource for scientists and researchers. Since 1994, all data produced by NASA – including the code and algorithms used to produce these data – have been free and open to all users for any purpose and provided as rapidly as practical after instrument checkout and calibration.
Starting in 2015, all data systems software developed through research and technology awards, including the competitive programs mentioned below or through in-house development, has been made publicly available as Open Source Software (OSS).
NASA’s Earth Observing System Data and Information System (EOSDIS) has continuously evolved to take advantage of new and innovative technologies. These evolutionary activities ensure that NASA's EOSDIS has the capability to meet not only the increasing demands of data providers and users, but also efficiently provide access to the increasing volume of EOSDIS data available to these users.
Over the 2020 Fiscal Year (1 October 2019 to 30 September 2020), for example, the EOSDIS archive had an ingest rate of approximately 33 terabytes (TB) per day, or roughly 12 petabytes (PB) per year. With the scheduled launch of high-volume data missions over the next five years, such as the Surface Water and Ocean Topography (SWOT) and the NASA-Indian Space Research Organization (ISRO) Synthetic Aperture Radar (NISAR) missions, this ingest rate is projected to grow to as much as 48 PB per year. As this ingest rate increases, the total volume of data in the EOSDIS archive is expected to grow from its current size of approximately 42 PB to as much as 250 PB by 2025.
This anticipated growth in both data ingest and archive volume poses challenges for archiving, distributing, and analyzing data. To address these challenges and meet the needs of future high-data-volume missions, the ESDS Program adopted a strategic vision to develop and operate multiple components of NASA's EOSDIS in a commercial cloud environment and provide improved data management and user access for many ongoing Earth science missions.
ESDS has completed an evaluation of the relative cost, technical performance, and security implications of utilizing the Amazon Web Services (AWS) commercial cloud environment for data ingest, archive, management, and distribution. Based on the findings of this evaluation, the program implemented "Cumulus," an open source, cloud-based framework.
A cloud-based architecture for EOSDIS data provides numerous benefits for data users:
- Improved efficiency of EOSDIS data system operations and the continued free and open access to EOSDIS data
- An architecture designed to scale easily to handle high-data-volume missions
- Increased opportunities for researchers and commercial users to access and work with large data collections in the cloud and only download analysis results
- A transparent/extendable open source processing framework
IMPACT is an interdisciplinary team that works to further the ESDS goal of overseeing the lifecycle of Earth science data to maximize the scientific return of NASA's missions and experiments for research and applied scientists, decision makers, and society. IMPACT’s three focus areas are interagency collaboration, assessment and evaluation, and advanced concepts.
The IMPACT Analysis and Review of CMR (ARC) team developed a novel tool to review EOSDIS collection and granule level metadata in the Common Metadata Repository (CMR) for correctness, completeness, and consistency. During 2020, the ARC team completed metadata quality assessments for approximately 3,500 data products, making significant progress toward the major milestone of completing metadata assessments
for NASA’s entire collection of more than 8,000 Earth science data products. This work is improving the consistency of metadata across the EOSDIS DAACs, creating more complete metadata records, and improving the data stewardship and usefulness of NASA Earth observing data.
IMPACT supports flight missions through the Airborne Data Management Group (ADMG), which ensures that airborne scientific data gathered in flight missions are discoverable and usable to the airborne and other scientific communities.
Competitive awards provide a constant source of innovation to improve data products, advance systems, and nurture interdisciplinary tools. ESDS Competitive Award Programs include:
Advancing Collaborative Connections for Earth System Science (ACCESS)
The ACCESS Program develops and implements technologies to effectively manage, discover, and utilize NASA’s archive of Earth observations for scientific research and applications. Program objectives are to enhance and improve existing components of NASA's EOSDIS that support NASA's Earth science research goals.
Citizen Science for Earth Systems Program (CSESP)
CSESP is focused on developing and implementing projects that harness contributions from members of the general public to advance the understanding of Earth as a system. Recent selected projects are focusing on air quality, snow observations, animal biodiversity, aerosol optical depth measurements, water storage tracking, and global kelp cover assessment.
Making Earth System Data Records for Use in Research Environments (MEaSUREs) Program
Through the MEaSUREs Program, NASA is continuing its commitment to develop consistent records to support Earth science explorations. Program objectives are to develop long-term, consistent, and calibrated data and products that are valid across multiple missions and satellite sensors. Awarded projects provide Earth science data products and services driven by NASA’s Earth science goals and that contribute to advancing the NASA Earth system “missions to measurements” concept.
Science portals comprehensively address and communicate information about a specific scientific topic in collaboration with research and analysis programs. The first of these portals, the Sea Level Change Portal (SLCP) provides a gateway to information about regional and global sea level change. The site includes access to data, analysis tools, and simulations of glaciers, ice sheets, sea level, and the solid Earth.
Read the Data Chat about NASA's Sea Level Change Portal.
ESDS continues to develop and enhance tools and systems to improve data access. These systems allow users to find, download, and visualize data appropriate for their needs.
Data Access Systems include:
- Earthdata Search
- NASA Worldview
- Global Imagery Browse Services (GIBS)
- Land, Atmosphere Near real-time Capability for EOS (LANCE)
- Common Metadata Repository (CMR)
NASA's Earth Science Data System Working Groups (ESDSWG) provide a forum for ESDS Competitive Program Principal Investigators and EOSDIS system managers and engineers to collaborate and infuse new ideas and technology cooperatively. Membership is comprised of working groups organized around key technology and information system issues. Membership is limited to NASA-funded researchers and EOSDIS personnel.
Significant evolutionary activities are initiated as a result of program reviews. In 2015, NASA proactively tasked the Earth Science Division (ESD)/Earth Science Data and Information System (ESDIS) Project to sponsor an independent, holistic review to study potential efficiencies and enhanced capabilities from a variety of perspectives based on:
- Science discipline
- Optimizing common data operation tasks across the DAACs