NASA’s Earth Science Data Systems (ESDS) Program depends on strategic partnerships with public and private companies to help further its data management and data development efforts. The most common means for creating these partnerships is through Space Act Agreements (SAAs), which NASA has relied on to further agency research since the agency’s establishment. NASA uses the term partnership to describe a wide variety of relationships with various external entities (e.g., contractors, academia, the public, other stakeholders). These partnerships are a distinct type of non-procurement business relationship and do not involve the acquisition of goods and services for the direct benefit of the agency; they are non-reimbursable, and no funds are exchanged.
New and existing partnerships are helping ESDS address several current needs:
- Support fast and parallel data access and utilize cost-effective storage structures
- Minimize data wrangling and preprocessing
- Help NASA assess gaps or capabilities for maximizing the use of the Earthdata Cloud platform
- Demonstrate the potential of cloud computing by bringing algorithms to the data to enable efficient processing and large-scale science analysis
- Infuse commercial Artificial Intelligence (AI) and Machine Learning (ML) techniques and expertise to address existing data discovery, access, and use challenges
Ongoing SAAs are enabling ESDS to develop a robust cloud infrastructure for NASA Earth science data and improve the discovery of NASA data and resources for using these data through commercial search engines.
Four ESDS SAAs are either in effect or being finalized:
Amazon Web Services (AWS)
Through an ongoing SAA, NASA and AWS are collaborating on cloud-based discovery, access, and use of high-value science datasets in NASA’s Earth Observing System Data and Information System (EOSDIS) collection. The AWS partnership features two primary activities: a design and development sandbox through the AWS Open Data Program to serve as a technology showcase and demonstrate ESDS data storage, archive, and use efforts using the AWS Commercial Cloud; and support for How to Cloud workshops, Advancing Collaborative Connections for Earth System Science (ACCESS) Program principal investigator (PI) training, and similar efforts.
The AWS Open Data Sponsorship Program supports the storage of publicly available high-value cloud-optimized datasets. Several EOSDIS datasets are available through the Registry of Open Data on AWS.
In addition, AWS provided hosting for several datasets to support the Agency-sponsored COVID-19 Space Apps Challenge that took place in May 2020. These datasets were made available in Cloud Optimized GeoTIFF (COG) format to participants.
As NASA looks more towards cloud-based archives, there’s an increased need to ensure that end-users can fully leverage these new capabilities. Along with the design and development sandbox, the second SAA AWS partnership provides AWS credits for workshops that will advance access and use of NASA data on AWS. Workshops considered for support will focus on enabling NASA science using NASA data hosted on AWS. Interested NASA-funded PIs and research teams are required to submit a NASA AWS Credit Application. For more information, please contact Dr. Justin Rice at NASA’s Earth Science Data and Information System (ESDIS) Project.
Google Cloud Platform
Through an ongoing SAA, NASA and Google are collaborating on the discovery, access, and use of high-value NASA science datasets. The overall goals of this collaboration include:
- Investigate the transfer, storage, and value of making large volumes of NASA EOSDIS datasets available on the Google cloud and Google Earth Engine;
- Make NASA EOSDIS data discoverable to users via the Google Cloud Public Datasets search engine and Earth Engine Catalog; and
- Demonstrate and evolve ESDS AI capacity by joining forces with NASA’s Frontier Development Lab (FDL) Challenges.
More than 95 NASA EOSDIS datasets from 10 EOSDIS DAACs are available through Google Earth Engine.
NASA’s Frontier Development Lab (FDL) is an applied AI research partnership with Google focusing on developing future AI applications in data-intensive challenges related to space science, planetary science, and Earth science. Members of NASA’s Interagency Implementation and Advanced Concepts Team (IMPACT) hosted and participated in the FDL 2020 Challenges, which is part of the executed Google SAA. IMPACT challenged Earth science FDL researchers to design a Knowledge Discovery Framework (KDF) using AI and machine learning (ML) techniques. The objective of the KDF is to discover unusual patterns and events in NASA Earth observation data by enabling users to provide an example image for which the KDF would search for similar images in the NASA data repository.
IMPACT also is working with FDL on the SpaceML initiative, which is finding ways to apply AI and ML to scientific data. A recent success story in this effort is development of the Worldview Imagery Search Pipeline, which applies ML to imagery in NASA’s Global Imagery Browse Services (GIBS) to enable rapid searches for specific imagery in NASA Worldview.
IBM and Microsoft
ESDS is completing SAAs with IBM Research and Microsoft to jointly work on common research problems to accelerate the development, delivery, and adoption of AI to further NASA’s science research and applications and explore new opportunities in cloud technologies that enable and accelerate open science.
The Planetary Computer was recently launched by Microsoft as their newest platform. Using open-source tools and supporting open standards, the platform allows users to leverage the cloud for accelerating environmental sustainability and Earth science. NASA’s Moderate Resolution Imaging Spectroradiometer (MODIS) and Harmonized Landsat Sentinel (HLS) collections are already part of the Planetary Computer Data Catalog.
The Planetary Computer computing environment—the Hub—is a development environment that makes data and APIs accessible through familiar, open-source tools. The Hub builds on the work done by the Pangeo project to put open-source tools of data science to work for the Earth science community. Pangeo is supported through ACCESS.
The Microsoft SAA partnership with ESDS also highlights Microsoft’s commitment to make it easier for NASA to adopt tools on Microsoft’s Azure cloud computing service, as well as guide and communicate best practices for Planetary Computer users. Other collaborations include work with the Earth Science Information Partners Lab to advise new potential users on how to work with Earth science data on Azure.