Skip to content

Conversation

@MrFok
Copy link

@MrFok MrFok commented Aug 19, 2022

  • Generated separate files for Zarr implementation (without disturbing existing pipeline)
  • Local test generated for Lazy Load Zarr Aggregation (LLZA)
  • Minor dependency changes, bug fixes (needs checking), commenting
  • Preliminary methods for multiple parts in collection-manager and granule-ingester

Changes in Proposed Pipeline

  • Dataset metadata push for new, YAML format (needs dynamic public bucket boolean implementation)
  • Streamlined processors for xarray Dataset instead of nexus processing
  • Direct Lazy Load Zarr Aggregation write into S3 bucket instead of tile storage in Cassandra

NOTE: All changes have NOT been completely tested in a full run of the entire pipeline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant