This repository was archived by the owner on Feb 1, 2024. It is now read-only.
This repository was archived by the owner on Feb 1, 2024. It is now read-only.
archive to cold storage #79
Description
Is your feature request related to a problem? Please describe.
Data always goes into galvanalyser but never comes out..... Eventually you'll either run out of disk space or end up paying a lot for disks
Describe the solution you'd like
Functionality to archive a dataset to cold storage.
- This storage would just be an S3 bucket, defined in .env (url, username, password)
- Button in UI to initiate archive for a given dataset
- data for that dataset is written to a hdf5 file (or other suitable, compressible binary format suitable for long term storage)
- file is written to S3 bucket to a filename based on the dataset id
- dataset is deleted from database, but the metadata is retained. Add flag to metadata indicating that this dataset has been archived
- Button in UI to initiate loading from archive
- data is read from file in S3 bucket back into the database
- archive flag cleared
- General: there needs to be checks to handle archived data (e.g. don't plot datasets that have been archived). API endpoints to provide data error if the dataset has been archived.
Additional context
None.