Cloud Notification Mechanism (CNM)

Summary

The Cloud Notification Mechanism (CNM) is standardized methodology for the notification of transfer of data between a cloud-based or an on-premises Earth Observing System Data and Information System (EOSDIS) platform and the Earthdata Cloud. The Request for Comments (RFC) document provides a brief introduction, with the full CNM specification detailed in Section 2 of the Cloud Notification Mechanism Interface Control Document.

Status

CNM is an approved NASA Earth Science Data Systems convention.

NASA Earth Science Community Recommendations for Use

Strengths

As ingest and archive systems migrate to commercial cloud platforms, CNM offers an updated transfer scheme to replace older or unsupported transfer protocols, such as Secure File Transfer Protocol (SFTP) and Secure Copy Protocol (SCP). CNM supersedes the notification portion of the Polling with Delivery Record (PDR) protocol for communicating the status of data transfer to a cloud platform in NASA Earth Science Data Systems. CNM is a pure push mechanism with minimal required fields for each granule and associated files to reduce latency in submission and response messages. The push mechanism is preferred over polling since listing the contents of an S3 bucket can be expensive and slow depending on bucket size.

As PDR is phased out, CNM will provide notification of data transfers between on-premises platforms to the Earthdata Cloud while utilizing the Science Data Transfer Protocol (SDTP) to comply with NASA security requirements.

Weaknesses

CNM provides a standard communication protocol for transfer of data to the cloud, which is essential for providers and the receiving entities to ensure successful ingest and archive of data. While this notification mechanism works well for collections with routine transfers of data and associated files, it may not be nominal for static datasets with a single delivery or high latency datasets due to the amount of work involved in setting up access to individual topics or subscriptions.

Applicability

The CNM has been used successfully at NASA data systems as a notification mechanism for the transfer of data between cloud-based and on-premises EOSDIS platforms and the Earthdata Cloud. CNM will be useful as a standardized methodology for submission and response messages for data delivery as the NASA Earthdata Science community migrates to the cloud.

Limitations

Both the provider and consumer of the CNM messages must agree on the push notification service and have the authorization to retrieve from or publish messages to the service for both the submission and response subscriptions.

CNM is focused on file transfers with a cloud platform, but it does not meet the immediate need to replace the PDR protocol for the large number of systems not in Earthdata Cloud.