Open Source Science for the Earth System Observatory Mission Data Processing Study Workshops
The workshops will be open to the public and a recording of the event will be made public after each workshop.
Updated circular image showing open science in the center with the benefits of open science around the core
Image based on a similar image created by Ramachandran, et al., and published in "From Open Data to Open Science." Earth and Space Science. doi:10.1029/2020EA001562

NASA is formulating the Earth System Observatory (ESO), a set of Earth-focused missions to provide key information to guide efforts related to monitoring climate change, mitigating disasters, fighting forest fires, and improving real-time agricultural processes. With this integrated approach, Kevin Murphy (Chief Science Data Officer, Science Mission Directorate, and Program Manager, Earth Science Data Systems (ESDS) Program) has set forth a challenge to the mission processing community to:

Identify and assess potential architectures that can meet the ESO mission science processing objectives, enable data system efficiencies, promote open science principles, and seek opportunities that support Earth system science.

A mission data processing system is the set of algorithms, software, compute infrastructure, operational procedures, documentation, and teams that automatically process raw instrument data through to science-quality data products. This includes the software tools that support the development of the processing algorithms and the validation and analysis of the processed data.

Open science is a foundational objective of NASA's Science Mission Directorate (SMD) and is defined as “a collaborative culture enabled by technology that empowers the open sharing of data, information, and knowledge within the scientific community and the wider public to accelerate scientific research and understanding” (Ramachandran, R., Bugbee, K. & Murphy, K.J. From Open Data to Open Science. Earth and Space Science, 8(5),doi:10.1029/2020EA001562). On October 14, SMD will report out on open science and activities at NASA's Open Source Science for Data Processing and Archives Workshop.

NASA's Earth System Observatory a new set of Earth-focused missions to provide key information to guide efforts related to climate change, natural hazard mitigation, fighting forest fires, and improving real-time agricultural processes.

The Open Source Science for ESO Mission Science Data Processing Study has a Steering Committee and System Architecture Working Group (SAWG) to address the challenge laid out above. The Study will consist of three workshops and a trade study designed to solicit input, perform systematic trade, and make a recommendation.

NASA Open-Source Science Initiative will be holding open science listening sessions in 2022. For more information on these sessions, please subscribe to the open-source mailing list.

Workshop #1, October 19-20, 2021

We began collecting information with our first virtual two-day workshop on Tuesday, October 19, 2021 (programmatic presentations) and Wednesday, October 20, 2021 (flight project presentations).

The input provided, along with information collected in the second workshop (scheduled for February 2022) on the state-of-the-art in science processing and open science, will be used by the ESO Open Source Processing Study Architecture Working Group to develop and assess potential architectures.

Agenda

Open science is defined as a collaborative culture enabled by technology that empowers the open sharing of data, information and knowledge within the scientific community and the wider public to accelerate scientific research and understanding. Under the guidelines set forth in the 2017 Decadal survey, NASA's new set of Earth-focused missions that form the ESO will adhere to open data, software, algorithm, and publication policies where missions will develop algorithms (in Levels 1–4), software, and documentation in open systems from inception. In order to generate community input and support, this first workshop solicited input from NASA programmatic officials and flight project teams.

Recordings of the workshop are available: ESO Mission: Data Processing Study Workshop

The workshop report is also available: ESO Mission Data Processing Study - Summary of NASA Program Offices and ESO Missions Requirements, Constraints, Recommendations, and Opportunities.

 

Day 1: Tuesday October 19th, 2021 (1 - 5 PM EST)

Duration EST PST Topic Speaker
0:10 1:00 PM 10:00 AM Welcome and Purpose Andrew Mitchell
0:15 1:10 PM 10:10 AM Earth Science Division (ESD) Opening Remarks Kevin Murphy
0:15 1:25 PM 10:25 AM SMD Scientific Information Policy Steven Crawford
0:15 1:40 PM 10:40 AM ESDS Perspective of Open Science Katie Baynes
0:15 1:55 PM 10:55 AM Overview of the Mission Processing Study

Andrew Bingham
0:15 2:10 PM 11:10 AM Q&A with the SAWG  
0:10 2:25 PM 11:25 AM Break  
0:15 2:35 PM 11:35 AM Flight Projects Program Perspective Charles Webb, Kathleen Boggs
0:15 2:50 PM 11:50 AM Earth Science Technology Office (ESTO) Perspective

Pam Millar, Jacqueline Lemoigne-Stewart, Ben Smith
0:15 3:05 PM 12:05 PM R&A Program Perspective Jack Kaye
0:15 3:20 PM 12:20 PM Q&A with the SAWG  
0:10 3:35 PM 12:35 PM Break

 
0:15 3:45 PM 12:45 PM Applied Science Program Perspective Nancy Searby
0:15 4:00 PM 1:00 PM

High Performance Computing (HPC) Perspective

 Tsengdar Lee
0:10 4:15 PM 1:15 PM Q&A with the SAWG  
0:30 4:25 PM 1:25 PM Open Discussion  
  4:55 PM 1:55 PM END  

 

Day 2: Wednesday October 20th, 2021 (12:20 - 5 PM EST)

Duration EST PST Topic Speaker
0:10 12:20 PM 9:20 AM Recap and Agenda  
0:15 12:30 PM 9:30 AM NISAR Programmatic Perspective Gerald Bawden
0:15 12:45 PM 9:45 AM NISAR Project Science Paul Rosen
0:15 1:00 PM 10:00 AM NISAR Project Data Processing Systems Hook Hua
0:10 1:15 PM 10:15 AM Q&A with the SAWG  
0:05 1:25 PM 10:25 AM Break  
0:15 1:30 PM 10:30 AM Atmosphere Observing System (AOS) Programmatic Perspective Hal Maring
0:15 1:45 PM 10:45 AM AOS Project Science Scott Braun
0:15 2:00 PM 11:00 AM AOS Project Data Processing Systems Robert Wolfe
0:10 2:15 PM 11:15 AM Q&A with the SAWG  
0:05 2:25 PM 11:25 AM Break  
0:15 2:30 PM 11:30 AM Mass Change (MC) Programmatic Perspective Lucia Tsaoussi
0:15 2:45 PM 11:45 AM MC Project Science David Wiese 
0:15 3:00 PM 12:00 PM MC Project Data Processing Systems David Wiese
0:10 3:15 PM 12:15 PM Q&A with the SAWG  
0:05 3:25 PM 12:25 PM Break  
0:20 3:30 PM 12:30 PM Surface Biology and Geology (SBG) Overview and Project Science Dave Schimel
0:20 3:50 PM 12:50 PM SBG Project Data Processing Systems Jeff Pon
0:05 4:10 PM 1:10 PM SBG Programmatic Perspective Ben Phillips
0:10 4:15 PM 1:15 PM Q&A with the SAWG  
0:30 4:25 PM 1:25 PM Open Discussion  
  4:55 PM 1:55 PM END  
Workshop #2, March 1-4, 2022

This workshop focused on understanding the state-of-the-art in Big Data processing systems. The workshop was open to the public. A Request For Information (RFI; submission period now closed) sought input from organizations with relevant expertise in Big Data processing and open science to help guide the study and workshop.

Recordings of the workshop are available: ESO Mission: Data Processing Study Workshop.

The workshop report is also available: ESO Mission Data Processing Study: Summary of State-of-the-Practice and State-of-the-Art Mission Data Processing System Architectures.

Please direct questions about the Open Source Science for the Earth System Observatory Mission Science Data Processing Study Workshops to: karen.yuen@jpl.nasa.gov or sara.h.lubkin@nasa.gov.

Agenda

Day 1: Tuesday, March 1, 2022 (12 - 5 pm EST)

Time
ET		 Time
PST		 Title Speaker Organization
12:00 PM 9:00 AM Welcome and Study Overview Andy Mitchell and Andy Bingham GSFC/JPL
12:15 PM 9:15 AM Logistics Sara Lubkin and Karen Yuen GSFC/JPL
12:20 PM 9:20 AM Opening Remarks Kevin Murphy NASA
12:25 PM 9:25 AM Open Source Science / SPD-41
doi:10.5281/zenodo.6320745		 Katie Baynes NASA
12:35 PM 9:35 AM Session 1 introduction: Science collaboration approaches Chelle Gentemann Farallon Institute
12:40 PM 9:40 AM SAR data exploration in environmental sciences: challenges and opportunities Naiara Pinto JPL
12:52 PM 9:52 AM Imagining a new NASA computing platform with AI + big data-supported analysis
doi:10.5281/zenodo.6320918

 

 Willow Coleman Harvey Mudd College
1:04 PM 10:04 AM Big Data Community Algorithms: Deep Learning for Mapping
doi:10.5281/zenodo.6320982		 Morteza Karimzadeh University of Colorado
1:16 PM 10:16 AM Cloud Computing Platforms for Processing Geospatial Big Data: Current Status and Challenges. doi:10.5281/zenodo.6307732 Qiusheng Wu University of Tennessee, Knoxville
1:28 PM 10:28 AM Project Jupyter - Lessons and Principles from a Community-Driven Open Source Project
doi:10.5281/zenodo.6320538		 Fernando Perez University of California, Berkeley
1:40 PM 10:40 AM Fishbowl Discussion Q&A led by the SAWG  
2:00 PM 11:00 AM Break    
2:10 PM 11:10 AM Session 2 introduction: NASA Earth Systematic Mission Processing Architectures Andy Mitchell GSFC
2:15 PM 11:15 AM Terra MODIS
doi:10.5281/zenodo.6320958		 Robert Wolfe GSFC
2:30 PM 11:30 AM CLAREO
doi:10.5281/zenodo.6345252		 Chris Currey LaRC
2:45 PM 11:45 AM ICESat-2
doi:10.5281/zenodo.6320538		 Tom Neumann GSFC
3:00 PM 12:00 PM PACE
doi:10.5281/zenodo.6313234		 Sean Bailey and Bryan Franz GSFC
3:15 PM 12:15 PM SWOT
doi:10.5281/zenodo.6315458		 Oh-Ig Kwoun JPL
3:30 PM 12:30 PM Fishbowl Discussion Q&A led by the SAWG  
3:45 PM 12:45 PM Break    
3:55 PM 12:55 PM Break Room Topics & Logistics    
4:10 PM 1:10 PM Breakout Rooms
1. System development approaches & challenges
2. System operations approaches & challenges
3. Open-sourced science approaches & challenges
4. Data analysis needs for Mission Data Processing System (MDPS)
5. Open source software approaches & challenges
6. MDPS Architectures now and the future		    
4:40 PM 1:40 PM Report out from Breakout rooms    
5:10 PM 2:10 PM END    

 

Day 2: Wednesday, March 2, 2022 (12 - 4:30 pm EST)

Time
ET		 Time
PST		 Title Speaker Organization
12:00 PM 9:00 AM Logistics Sara Lubkin and Karen Yuen GSFC/JPL
12:05 PM 9:05 AM Session 3 introduction: NASA Earth System Science Pathfinder mission processing architectures Andy Mitchell GSFC
12:10 PM 9:10 AM OCO-2/OCO-3
doi:10.5281/zenodo.6321343		 Cecilia Cheng JPL
12:25 PM 9:25 AM EMIT Phil Brodrick JPL
12:40 PM 9:40 AM MAIA
doi:10.5281/zenodo.6323718		 Jeff Walter LaRC
12:55 PM 9:55 AM Break    
1:05 PM 10:05 AM TROPICS
doi:10.5281/zenodo.6323291		 Jess Braun University of Wisconsin, Madison
1:20 PM 10:20 AM GEDI Scott Luthke GSFC
1:35 PM 10:35 AM Fishbowl Discussion Q&A led by the SAWG  
1:50 PM 10:50 AM Session 4 introduction: Non-NASA Earth science mission processing architectures Andy Bingham JPL
1:55 PM 10:55 AM National Oceanic and Atmospheric Administration
doi:10.5281/zenodo.6307369		 Lihang Zhou and Joseph Mani NOAA
2:10 PM 11:10 AM U.S. Geological Survey Chris Engebretson USGS
2:25 PM 11:25 AM Fishbowl Discussion Q&A led by the SAWG  
2:40 PM 11:40 AM Break    
2:50 PM 11:50 AM Breakout Rooms
1. System development approaches & challenges
2. System operations approaches & challenges
3. Open-sourced science approaches & challenges
4. Data analysis needs for Mission Data Processing System (MDPS)
5. Open source software approaches & challenges
6. MDPS Architectures now and the future		    
3:20 PM 12:20 PM Report out from Breakout rooms    
3:50 PM 12:50 PM Japan Aerospace Exploration Agency
doi:10.5281/zenodo.6319381		 Osamu Ochia JAXA
4:05 PM 1:05 PM END    

 

Day 3: Thursday, March 3, 2022 (12 - 5:20 pm EST)

Time
ET		 Time
PST		 Title Speaker Organization
12:00 PM 9:00 AM Logistics    
12:05 PM 9:05 AM National Remote Sensing Center, India Space Research Organization (ISRO)
doi:10.5281/zenodo.6302928		 Uzair Mujeeb ISRO
12:20 PM 9:20 AM Italian Space Agency (ASI) Francesco Tataranni ASI
12:35 PM 9:35 AM German Aerospace Center (DLR)
doi:10.5281/zenodo.6315122		 Conrad Albrecht DLR
12:50 PM 9:50 AM European Space Agency (ESA) Anca Anghelea, Klaus Scipal, and Clement Albinet ESA
1:05 PM 10:05 AM Fishbowl Discussion Q&A led by the SAWG  
1:20 PM 10:20 AM Break    
1:30 PM 10:30 AM Session 5 introduction: Non-Earth science mission processing architectures Luke Dahl  
1:35 PM 10:35 AM Vera C. Rubin Observatory's Legacy Survey of Space and Time (LSST)
doi:10.5281/zenodo.6311597		 KT Lim Vera C. Rubin Observatory
1:50 PM 10:50 AM Nancy Grace Roman Space Telescope
doi:10.5281/zenodo.6326463		 Megan Soey Space Telescope Science Institute (STScI)
2:05 PM 11:05 AM IPAC Missions Gregory Dubois-Felsmann Caltech
2:20 PM 11:20 AM Fishbowl Discussion Q&A led by the SAWG  
2:35 PM 11:35 AM Break    
2:45 PM 11:45 AM Session 6 introduction: System interfaces and standards Andy Bingham  
2:50 PM 11:50 AM Earth Science Data and Information System (ESDIS)

doi:10.5281/zenodo.6402353

 Sara Lubkin GSFC
3:05 PM 12:05 PM NASA Transform to Open Science (TOPS)
doi:10.5281/zenodo.6324320		 Chelle Gentemann Farallon Institute
3:20 PM 12:20 PM NASA's High-End Computing Capability: Growing to Support Science Data Processing for the Earth System Observatory Missions
doi:10.5281/zenodo.6330014		 Jon Jenkins Ames Research Center
3:35 PM 12:35 PM Data System Standards Ingo Simonis OGC
3:50 PM 12:50 PM Fishbowl Discussion Q&A led by the SAWG  
4:05 PM 1:05 PM Break    
4:05 PM 1:05 PM Breakout Rooms
1. System development approaches & challenges
2. System operations approaches & challenges
3. Open-sourced science approaches & challenges
4. Data analysis needs for Mission Data Processing System (MDPS)
5. Open source software approaches & challenges
6. MDPS Architectures now and the future		    
4:35 PM 1:35 PM Report out from Breakout rooms    
5:05 PM 2:05 PM END    

 

Day 4: Friday, March 4, 2022 (12 - 4 pm EST)

Time
ET		 Time
PST		 Title Speaker Organization
12:00 PM 9:00 AM Logistics    
12:05 PM 9:05 AM Session 7 introduction: Other Big Data processing system architectures Luke Dahl  
12:10 PM 9:10 AM CyVerse Tyson Swetnam, Edwin Skidmore University of Arizona
12:25 PM 9:25 AM Pangeo
doi:10.5281/zenodo.6330215		 Joe Hamman Carbon Plan
12:40 PM 9:40 AM Alsaka SAR Facility

doi:10.5281/zenodo.6354724

 Kirk Hogenson Alaska Satellite Facility
12:55 PM 9:55 AM Raytheon
doi:10.5281/zenodo.6325581		 Shawn Miller Raytheon
1:10 PM 10:10 AM Element 84 STAC
doi:10.5281/zenodo.6326572		 Dan Pilone Element 84
1:25 PM 10:25 AM Break    
1:35 PM 10:35 AM Fishbowl Discussion Q&A led by the SAWG  
1:50 PM 10:50 AM Red Hat Dan McGuan Red Hat
2:05 PM 11:05 AM Amazon Web Services (AWS)   AWS
2:20 PM 11:20 AM Multi-Mission Algorithm and Analysis Platform (MAAP)
doi:10.5281/zenodo.6345081		 George Chang JPL
2:35 PM 11:35 AM OpenNEX
doi:10.5281/zenodo.6345019		 Jennifer Dungan and Ian Brosnan ARC
2:50 PM 11:50 AM Break    
3:00 PM 12:00 PM Unity (Science Data System as a Service)
doi:2014/54386		 Hook Hua and Laura Jewell JPL
3:15 PM 12:15 PM Science Data Analytics Platform (SDAP)
doi:10.5281/zenodo.6310363		 Nga Chung and Thomas Huang JPL
3:30 PM 12:30 PM NASA Earth Information System (EIS)

doi.10.5281/zenodo.6353694

 Alexey Shiklomanov GSFC
3:45 PM 12:45 PM Fishbowl Discussion Q&A led by the SAWG  
4:00 PM 1:00 PM END of Workshop    
Study Team

Dr. Andrew Bingham has more than 20 years of experience managing and leading teams responsible for delivering science processing, archive, distribution, and analysis systems for Earth and planetary missions. He is the manager for the Science Data and Instrument Operations System Section at NASA's Jet Propulsion Laboratory (JPL).

Luke Dahl has more than 20 years of experience as an architect and systems engineer supporting diverse activities ranging from enterprise infrastructure and applications, multi-mission instrument operations, and Earth and planetary science data systems.

Dr. Chelle Gentemann is a senior scientist at the Farallon Institute leading research on open science, cloud computing, remote sensing, and physical oceanography. For more than 20 years, she has worked on every aspect of passive microwave satellite missions, both domestically and internationally, from launch through decommission, including calibration, algorithm development, validation, data distribution, and science applications.

Dr. Sara Lubkin has a broad background in Earth science. She has worked with NASA since 2015 and is a Distributed Active Archive Center (DAAC) Operations Engineer with NASA’s Earth Science Data and Information System (ESDIS) Project at NASA's Goddard Space Flight Center.

Andrew Mitchell has multiple years of experience executing the technical and financial management of the science systems of NASA's Earth Observing System Data and Information System (EOSDIS). He is responsible for managing the processing, archiving, and distribution of Earth science data while ensuring scientists and the public have access to these data to enable the study of Earth. He is the project manager for ESDIS at Goddard.

Karen Yuen has 20 years of technical and management experience in science applications, project formulation, system engineering, and communications for multiple NASA missions and programs. She is the Science Data Applications Lead for the Orbiting Carbon Observatory-2 (OCO-2) and OCO-3 missions.

Study Architecture Working Group Members
Name Focus Area Organization Required Expertise
Elias Sayfi Co-Chair JPL
  • Flight mission data processing system architect.
  • Community builder
Natasha Stavros Co-Chair University of Colorado, Boulder
  • Science data processing system architect
  • Community builder
Hook Hua SBG and NISAR Science Processing Representative JPL
  • Science data processing systems
  • Level 3 (L3) requirements and mission objectives
Curt Tilmes AOS Science Processing Representative Goddard/Ames Research Center
  • Science data processing systems
  • L3 requirements and mission objectives
Bernie Bienstock MC Science Processing Representative JPL
  • Science data processing systems
  • L3 requirements and mission objectives
Wenying Su Applied Science Algorithm Developer Langley Research Center
  • Expertise developing L3+ algorithms for science applications
  • Experience integrating algorithms into a pipeline.
  • Experience working in a collaborative development environment.
Andy Michaelis Data Processing Workflow Community-focused Developer Ames
  • Expertise in modern open-source development practices, cloud-based orchestration, GitHub, Continuous Integration.
  • Experience with big data applications (science preferred, but not necessary)
Lesley Ott Geophysical Numerical Modeler Goddard
  • Expertise with HPC
  • Expertise with big data assimilation and operations
Qing Yue Instrument Algorithm Developer JPL
  • L2 algorithm development
  • Science data processing systems
Evelyn Ho Science Processing Architect from a comparable big-data science-based organization or project Earth Science Data Information System (ESDIS) Science Investigator-led Processing Systems (SIPS)  
Chris Engebretson Science Processing Architect from a comparable big-data science-based organization or project USGS  
Adrian Parker Science Processing Architect from a comparable big-data science-based organization or project NOAA  
Sean Harkin Science Processing Architect from a comparable big-data science-based organization or project Marshall Space Flight Center  

View Workshop Recordings

Read the Workshop Final Report

Related Resources

Last Updated

Technologies

Featured

April is Earth Month and a great time to explore NASA's more than 9,300 Earth science data collections