Letter from the NASA ESDS Program Executive
NASA is joining the White House and other federal agencies in declaring 2023 A Year of Open Science, and the Earth Science Data Systems (ESDS) Program is well-positioned to continue its leadership developing innovative tools, techniques, and policies to enable the open use of NASA Earth science data. The following highlights are a small sampling of the many ESDS accomplishments during Fiscal Year 2022 (FY22; October 1, 2021, to September 30, 2022).
ESDS efforts facilitating unrestricted access to NASA Earth science data—along with the tools, applications, and systems for working with these data—encourage collaborative work in the scientific process among people of diverse backgrounds and further scientific discovery for societal benefit. Another program focus is using these data to strengthen environmental justice. Combining Earth science data with socioeconomic data creates a powerful tool for identifying vulnerable populations, and the program is making great strides in enhancing these connections.
Throughout FY23, ESDS will continue its work broadening the use of NASA Earth science data and creating the tools and policies to ensure the efficient, equitable use of these data. These highlights only scratch the surface of the groundbreaking work being done by the ESDS team. NASA Earth science data are yours to use, and we look forward to reading about your discoveries.
Dr. Cerese Albers, ESDS Program Executive
- Furthering Open Science, Equity, and Environmental Justice
Throughout FY22, ESDS continued its support for open science and contributed significantly to agency-wide efforts to broaden the use of NASA data to support environmental justice initiatives. ESDS will remain at the forefront of these efforts throughout 2023—A Year of Open Science.
ESDS defines open science as a collaborative culture enabled by technology that empowers the open sharing of data, information, and knowledge within the scientific community and the wider public to accelerate scientific research and understanding. The open, unrestricted provision of NASA Earth science data, tools, and services is also a key element for furthering environmental justice. Environmental justice refers to the fair treatment and meaningful involvement of all people, regardless of race, color, national origin, or income, in the development, implementation, and enforcement of environmental laws, regulations, and policies.
ESDS continued its support of the agency-wide Open-Source Science Initiative (OSSI). Led by NASA’s Science Mission Directorate (SMD), OSSI is a coordinated program of activities to move science and the scientific process towards greater openness. A major OSSI component is Transform to Open Science (TOPS), which is helping agencies, organizations, and communities evolve to an inclusive culture of open science. ESDS supports the three primary TOPS goals:
- Increase understanding and adoption of open science principles and techniques in mission and research communities
- Accelerate major scientific discoveries through supporting the adoption of open science
- Broaden participation by historically excluded communities
In FY22, ESDS assisted in the development of two TOPS products: Stories of Open Science and the TOPS Community Panel Summary Report. Stories of Open Science features open science practitioners discussing what open science means to them, their motivations for doing open science, the challenges they face in practicing open science, and the benefits to their research from their use of open science.
The TOPS Community Panel was created to provide feedback on the TOPS mission, plans, and activities. The report summarizes findings from a virtual event held in May 2022; these findings were used to improve plans for A Year of Open Science, a new open science curriculum (OpenCore), community engagement, and overall missions and goals.
Another ESDS open science initiative in FY22 was co-leadership of the Open Source Science for the Earth System Observatory Mission Data Processing Study Workshops. In line with the open science, integrated mission approach of NASA's Earth System Observatory (ESO), Kevin Murphy, SMD Chief Science Data Officer, issued a challenge to the ESO mission science data processing community: identify and assess potential architectures that can meet ESO mission science processing objectives, enable data system efficiencies, promote open science principles, and seek opportunities that support Earth system science. Three open workshops were held in FY22 to explore potential data processing architecture and data system efficiencies.
Another ESDS-supported open science effort is Openscapes. This NASA-funded three-year initiative began in FY21 and provides training and resources to help researchers migrate workflows to the Earthdata Cloud. At the end of FY22, Openscapes was working with mentors from seven NASA Distributed Active Archive Centers (DAACs). ESDS-supported Openscapes FY22 activities included events such as the 2021 Cloud Hackathon and DAAC internal trainings along with the development of open-source resources such as the Earthdata Cloud Cookbook, the Python Library for NASA Earthdata APIs, and various guides to aid researchers transitioning to cloud-based workflows.
Openscapes Champions is a key element of Openscapes activities and brings together DAAC mentors with environmental and Earth science research teams to explore open data science practices. Participants attend as a team with their research group in a cohort with other teams, together learning how to reframe data-intensive science as a collaborative effort. The first NASA Openscapes Champions Cohort took place in March and April 2022 and featured 10 research teams. Additional Openscapes Champions Cohorts are scheduled for FY23.
Along with open science, ESDS supported initiatives to further equity and environmental justice (EEJ). The agency has established four foundational EEJ focus areas as part of the NASA Equity Action Plan, and NASA’s Earth Science Division (ESD) is directly acting on Focus Area 3: Leverage Earth Science and Socioeconomic Data to Help Mitigate Environmental Challenges in Underserved Communities.
The ESD EEJ team includes ESDS, NASA’s Research and Analysis Program (R&A), and NASA’s Applied Sciences Program. The EEJ team along with NASA’s Socioeconomic Data and Applications Center (SEDAC) coordinated an Equity and Environmental Justice Virtual Workshop on October 20, 2021. The workshop helped identify priorities, gaps, barriers, and opportunities in using NASA data, research, and applications to address environmental justice and in support of environmental justice research and stakeholder communities.
In response to a workshop recommendation to increase awareness of NASA Earth observation data, ESDS created the Environmental Justice at NASA Backgrounder. The Backgrounder provides information about how NASA data are supporting environmental and climate justice efforts along with use cases showing how scientists and decision-makers are applying NASA data for assessing vulnerability and exposure to environmental challenges.
One of the actions from the NASA Equity Action Plan is to make data available on the cloud and convert existing datasets to widely accepted formats for broader use by more communities. The goal was to make the 50 most-requested environmental datasets of 2021 cloud-native by 2022, with full migration of NASA Earth science data to the cloud by 2025. One result of this data migration was the development of the first interactive environmental justice story as part of an Environmental Justice theme area in the new Visualization, Exploration, Data, and Analysis (VEDA) dashboard.
The ESDS Understanding Needs to Broaden Outside Use of NASA Data project Environmental Justice workshop (UNBOUND-EJ) took place in April, May, and June 2022. UNBOUND seeks to make NASA data, tools, and resources more usable and accessible to a broader community. The UNBOUND-EJ workshop consisted of a series of sessions with data practitioners who were not currently using NASA data to help ESDS identify data user needs.
Finally, the EEJ team created the Environmental Justice Data Catalog to provide a central repository of NASA Earth science datasets useful for environmental justice research and applications. The catalog contains approximately 140 datasets organized by eight environmental justice indicators, such as disasters and air quality. The Data Catalog supports NASA ESD goals to constantly expand awareness, accessibility, and use of Earth science data and enable contributions to Earth science research and applications.
- Data Innovation and Support for User Needs
ESDS components had significant achievements throughout FY22 ensuring that NASA Earth science data are available openly, efficiently, and in the many formats required by global users.
The Earth Science Data and Information System (ESDIS) Project continued to provide outstanding management of a data collection that grew in FY22 from a total volume (including in-cloud data) of 59.2 petabytes (PB) to 76.4 PB. Their success managing, archiving, and distributing these data was reflected in a strong score of 79/100 on the 2022 American Customer Satisfaction Index (ACSI) survey. While this score is two points lower than the 2021 score, it maintains the strong four-year rolling average of 79 and indicates high customer satisfaction with products and services. The 2022 survey marks the 19th year NASA has performed an independent review of user satisfaction with NASA DAACs.
Migration of data to the Earthdata Cloud remains an ESDS priority, especially with the launch of the first high-data-volume ESO missions in FY23. At the end of FY22, 20.2 PB of data in NASA's Earth Observing System Data and Information System (EOSDIS) collection have been migrated, and the entire collection remains on track to be migrated over the next two years.
As noted in the previous section, the top 50 most popular Earth science datasets were migrated to the Earthdata Cloud. This migration culminated with a transfer of over 55 million data granules comprising 15 PB of data to the commercial cloud. DAACs were challenged a stretch goal of migrating 25 additional datasets, which contributed an additional 2.7 PB of data. These datasets are now available for opportunities to enable new science.
The Global Imagery Browse Services (GIBS) team also continued its data migration, an effort called GIBS-in-the-Cloud (GITC). At the end of FY22, GIBS had reduced its on-premises footprint from 12 hosts to seven and expects its on-premises footprint to be fully cloud-native by the end of 2023.
Complementing GITC efforts, NASA Worldview implemented several improvements and updates in FY22 to enhance the functionality and utility of the application. The Worldview team created new tour stories for geostationary imagery and nighttime lights, and added new features including orbit tracks as vectors; the ability to convert temperatures between Celsius, Fahrenheit, and Kelvin; and enhancements to the Worldview Snapshots tool. Users can now save Worldview as a Progressive Web Application on mobile and desktop devices, which allows users to take advantage of full screen mode, launch Worldview from their home screen or desktop more easily, and use local caching for faster load times. Finally, Worldview completed its migration to the cloud. While it took a good bit of effort behind the scenes to move the application from on-premises to the cloud, users won't notice any changes.
The Land, Atmosphere Near real-time Capability for EOS (LANCE) team added several products to the Fire Information for Resource Management System (FIRMS) and FIRMS US/Canada. These include an ultra real-time (URT) active fire data product available within a minute of satellite observation for much of the U.S. and Canada; a Landsat Fire and Thermal Anomalies (LFTA) product offering 30-meter active fire data available within 30-60 minutes of observation for the continental U.S., southern Canada, and northern Mexico; and a prototype Harmonized Landsat Sentinel-2 (HLS) short-wave infrared false color composite imagery layer that provides greater resolution and contrast for identifying and monitoring burned areas.
LANCE also introduced near real-time (NRT) soil moisture data from the Soil Moisture Active Passive (SMAP) mission and expedited Ice, Cloud and land Elevation Satellite-2 (ICESat-2) data. Although not available in NRT, ICESat-2 expedited data are provided within three days of satellite observation, which is significantly faster than the approximate 45-day latency for the standard science product.
Furthering open-source collaborative research efforts, ESDS completed version 2.0 of the Multi-Mission Algorithm and Analysis Platform (MAAP). MAAP is a joint NASA/ESA (European Space Agency) open-source project focused on global biomass and forest structure. A key design element of MAAP is that the platform and its components can adapt for collaborative research into a wide range of fields. In FY22 MAAP scientists used ICESat-2 data to compute the weight of all trees in boreal forests. These data were added to estimates derived from Global Ecosystem Dynamics Investigation (GEDI) data to provide a more accurate global forest biomass estimate. Another significant MAAP accomplishment was its selection as a collaborative data platform to support the first Global Harmonization Activity for Biomass in the United Nations Framework Convention on Climate Change Global Stocktake of Carbon. This work puts all major estimates of global forest carbon together in one place and evaluates them against one-another for agreement, variability, and accuracy.
The ESDS Interagency Implementation and Advanced Concepts Team (IMPACT) continued its groundbreaking work finding new ways to use and analyze NASA Earth science data. One significant IMPACT accomplishment in FY22 is the VEDA project. VEDA is NASA’s unified open-source science cyberinfrastructure for data processing, visualization, exploration, and geographic information systems (GIS) capabilities. The project originated from the communal need for a standardized set of tools to analyze, explore, interact, and process NASA Earth science data in the cloud. VEDA’s public-facing component, VisEx, is an interactive dashboard supporting the exploration, visualization, and analysis of NASA datasets and features.
IMPACT’s Science Discovery Engine (SDE) project is a key component in NASA’s effort to construct an open-source science infrastructure. The SDE is building an integrated NASA SMD search capability to enable the discovery of open data and information across SMD’s five divisions. The Beta release of the SDE was made available to NASA users in May 2022 and was released publicly on the SMD website in December 2022. At the end of FY22, the SDE team had indexed data and information from all SMD divisions.
Additional IMPACT FY22 highlights include:
- Reverse Image Search: IMPACT partnered with the SpaceML initiative, an international artificial intelligence (AI) accelerator for citizen scientists and a branch of Frontier Development Lab, to develop the prototype model for a reverse image search tool using decades of imagery served by GIBS. The results of this effort are an easy-to-use interface for data search at scale, a set of open-source tools that simplify using NASA’s Earth science archive for machine learning, and cross-domain utilization of a similar approach for space observation data.
- Algorithm Publication Tool (APT): Designed to enhance the production of Algorithm Theoretical Basis Documents (ATBDs), the APT is now available to science teams. IMPACT, in partnership with developers at Development Seed, designed and created the APT to ensure ATBD contents are consistent, searchable, citable, and obtainable. Several authorized users are currently writing their ATBDs in the APT, and a demo ATBD is available for public viewing.
- SNWG 2022 Survey: IMPACT’s Satellite Needs Working Group (SNWG) Management Office launched the biennial Satellite Needs Survey in June. Participation in the survey increased by 50% in 2020, and the 2022 survey received a similar number of submissions. During FY22, the SNWG Management Office initiated formulation for several activities identified in the 2020 survey. An important addition to the 2022 survey cycle is the Stakeholder Engagement Program (SEP). Led by the SNWG Management Office, the SEP ensures that training and data products necessary to support implemented services are established and enhanced.
During FY22, the ESDS Commercial Smallsat Data Acquisition (CSDA) program on-boarded two qualified vendors from the second Request for Information (RFI) for data evaluation: Airbus DS GEO (US), Inc., and BlackSky Geospatial Solutions, Inc. At the end of FY22, GHGSat, Inc., was awarded a Blanket Purchase Agreement (BPA) to provide a comprehensive catalogue of data products for evaluation. GHGSat specializes in high-resolution remote sensing of greenhouse gases from space, and its data are being acquired as part of CSDA’s On-ramp #3.
The ESDS GIS Team (EGIST) continued its work enabling the use and adoption of GIS technology with NASA Earth science data. EGIST contributed to a Space Act Agreement with Esri, the world’s largest commercial GIS company, to enhance NASA/Esri collaboration on optimizing the use of geospatial technology and broaden the discovery, access, and use of NASA data by the GIS community (this agreement was finalized in November 2022). EGIST continued its engagement with GIS teams across the ESD through the ArcGIS DAAC Collaboration.
EGIST also coordinated the nomination of NASA GIS-ready content to publicly accessible online platforms used by the GIS community through an Esri NASA_Earthdata account. Nearly 30 layers were shared via the account and 18 layers were nominated to be included in Esri’s Living Atlas of the World. Additionally, EGIST helped curate and develop a cross-DAAC GIS content collection, which includes thematic-based web maps, apps, and StoryMaps. These content items provide endpoints that can be accessed and used in a variety of geospatial clients and tools, including open source software such as QGIS.
Finally, the ESDS Citizen Science for Earth Systems Program (CSESP) conducted a planned down-select from the pilot projects previously funded in FY21. The five projects selected in FY22 to receive three years of continuous funding have already moved into their implementation stages and successfully started to make a difference from pole to pole. The Fresh Eyes on Ice project kicked off its 2022-2023 season working with Alaskan Indigenous people to track annual Arctic ice movements and breakup. At the other end of Earth, FjordPhyto collected data about phytoplankton and glacial meltwater in Antarctica. High in the mountains, the Mountain Rain or Snow project collected its 10,000th observation, sorting out the interface between rain and snow (a measurement difficult to obtain from satellite-based sensors). And on the U.S. East Coast, Chesapeake Water Watch combined volunteer-collected water quality data with remote sensing to better monitor the health of the Bay. The global Lake Observations by Citizen Scientists and Satellites (LOCSS) project greatly expanded their observations, including a significant success story in Bangladesh.
- Outreach and Engagement
Communicating the benefits of open science, engaging new communities, and creating new avenues through which ESDS data, products, and services can be shared are a foundation of program activities.
A significant FY22 accomplishment was the release of an updated version of the Earthdata website. The enhanced website provides easier access to ESDS resources, a better organization to support ESDS components, and greater opportunities to find, understand, retrieve, and employ Earth science data. Along with the enhanced website, a new ESDS monthly newsletter—Earthdata Discovery—was inaugurated in July. The goal of this publication is to highlight Earthdata content and drive readers to the website. The newsletter currently has more than 1,500 subscribers.
The ESDS and EOSDIS communications teams published 105 articles on the Earthdata website, including 12 Data User Profiles, 4 Data Chats, 10 articles highlighting the release of new DAAC data products, 3 quarterly EOSDIS newsletters (which were replaced in July by the monthly program-wide newsletter), and numerous announcements and news items about webinars and open science meetings. In addition, writers from IMPACT, GIBS, and FIRMS contributed 45 publications through the IMPACT blog, GIBS blog, and FIRMS blog.
The program also continued enhancing the popular Data Pathfinders. Existing Data Pathfinders were reviewed and updated for greater usability and a new GIS Data Pathfinder was created. Complementing the Pathfinders, new Resource Spotlights for Health and Air Quality, Biological Diversity and Ecological Conservation, Agriculture, and Open Science were published.
ESDS also made great strides bringing NASA Earth science data to new communities and creating new synergies with existing data users. Over FY22, the ESDS Outreach Team directly engaged more than 1,100 data users of all abilities at conferences, workshops, and seminars. ESDS and ESDIS outreach staff partnered with Rensselaer Polytechnic Institute to create and test data tools for a new Energy Resources Data Pathfinder scheduled for publication in FY23. In addition, ESDS outreach staff collaborated with the American Society of Agronomy (ASA) airborne and spaceborne remote sensing and precision agriculture system communities to produce the “Leveraging NASA Geospatial Data Products for Agriculture Applications” preconference workshop.
GIS also remains a program focus, and EGIST reached out to relevant user groups and existing partnerships through several events. The team conducted presentations, staffed booths, and participated in user groups at the Esri International User and American Geographical Society conferences, often in collaboration with NASA’s Applied Sciences Program.
ESDS and EGIST outreach teams joined the TOPS and Applied Sciences Indigenous Peoples Capacity Building Initiative (IPCBI) teams to provide a hands-on workshop for the 12th annual Tribal GIS Conference.
The EOSDIS communications team continued its popular Earthdata webinar series. The 13 data discovery and data access webinars in FY22 had more than 2,500 participants from more than 120 countries. Webinar recordings gained 134,355 additional online views (up 32.1% over FY21), 8,300 hours of watch time, and 724,000 impressions on the NASA Earthdata YouTube channel.
Along with webinars and tutorials, numerous new playlists were added to the Earthdata YouTube channel. These included playlists for CSDA, the Delta-X Applications Workshop, the 2022 Prediction of Worldwide Energy Resources (POWER) Global Community Workshop, Earthdata GIS resources, and 13 special feature videos or short data tutorials. The EOSDIS team worked with NASA Worldview staff to create a tutorial on Getting Started with NASA Worldview and contributed to numerous agency-wide and high visibility campaigns, including the 20th anniversary of NASA’s Aqua satellite and Earth Day 2022.
Program components also provided support to the 2022 NASA International Space Apps Challenge, which featured numerous challenges that included ESDS products, services, and tools. Outreach staff joined with IMPACT’s Airborne Data Management Group (ADMG) to create the Take Flight: Making the Most of NASA’s Airborne Data challenge. The challenge was a huge success, with 140 Teams participating and 74 project submissions.
Finally, the NASA Earthdata Forum continued to grow with the addition of three new DAACs: NASA’s Alaska Satellite Facility DAAC (ASF DAAC), the National Snow and Ice Data Center DAAC (NSIDC DAAC), and the Oak Ridge National Laboratory DAAC (ORNL DAAC). Along with the Global Change Master Directory (GCMD), 10 DAACs are Forum participants.
- Building on ESDS Accomplishments: The Year Ahead
From the incorporation of data from the first ESO missions into the Earthdata Cloud to A Year of Open Science, FY23 promises to bring more NASA Earth science data to a broader global audience than ever before.
Building upon the success of FY22 open science efforts, ESDS will support activities in the Transform to Open Science Training (TOPST) solicitation through the provision of data and training opportunities and support for the OpenCore curriculum. In addition, an air quality-focused UNBOUND workshop takes place in early-2023 with additional workshops expected during FY23 as community proposals in development are evaluated.
NASA DAACs will continue to serve as mentors in the NASA-funded Openscapes initiative. A second NASA Openscapes Champions Cohort is scheduled for spring 2023, and work continues developing Openscapes resources to help new users get started working in the Earthdata Cloud.
EGIST outreach and engagement will continue with both internal and external GIS user groups, and the team will engage with UNBOUND workshops to provide demonstrations and overviews of GIS resources while also working with the ESDS communications team to add geospatial data, services, and resources to new and existing Data Pathfinders.
A major FY23 ESDS initiative is the Web Unification project, the ESDS Program’s response to the agency’s Web Modernization project. This effort will consolidate all ESDS-funded websites into the Earthdata website. Over FY23, the Web Unification team will execute a full content and applications assessment and establish regularly occurring and open working groups.
While NASA Earth science data will continue to be migrated to the Earthdata Cloud, new tools and applications for using these data also will be enhanced. At IMPACT, VEDA will highlight NASA Earth science data discoveries and environmental justice stories through the addition of new theme areas and datasets and the SDE project will continue to refine NASA SMD term lists of platforms, instruments, and missions. Development of the APT continues, with public access to completed ATBDs anticipated by the end of FY23.
Finally, CSDA continues to enhance its commercial data archiving and distributing capabilities. In FY23, CSDA will shift to using EOSDIS core services for ingesting, archiving, cataloging, and distributing purchased commercial data distributed by CSDA. Additionally, the 1-year scientific evaluation of commercial data from On-ramp #2 will culminate with the NASA ESD summary report based on the findings and recommendations from the evaluation teams.
In FY23, CSDA anticipates the award of the additional on-ramp vendors: Capella Space, ICEYE US, Inc., and GeoOptics. A sole source synopsis for each vendor has been released and the scientific evaluation of commercial data will start after contract award. The RFP for the Multiple-Award Indefinite Delivery/Indefinite Quantity (IDIQ) contract is the next on-ramp opportunity for new companies as well as for sustained CSDA vendors. For these acquisitions, NASA will continue to seek end user license agreements (EULAs) that will enable broad levels of dissemination and shareability with U.S. government agencies and partners.
These FY22 highlights only scratch the surface of the tremendous work being done by NASA’s ESDS Program. By ensuring that NASA Earth observation data are findable, available, interoperable, and reusable (FAIR)—and openly available as early in the scientific process as feasible—ESDS will continue to broaden the collaborative use of these data by a larger, more diverse community of data users.