Sync metadata #14
Sync metadata #14
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #14 +/- ##
==========================================
+ Coverage 87.86% 87.87% +0.01%
==========================================
Files 10 10
Lines 519 569 +50
Branches 30 33 +3
==========================================
+ Hits 456 500 +44
- Misses 60 65 +5
- Partials 3 4 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| """Check if the metadata rsync was successful by reading the exit code file.""" | ||
| return fs.check_exit_status(self.metadata_rsync_exitcode_file) | ||
|
|
||
| def sync_metadata(self): |
There was a problem hiding this comment.
Am I missing something or Is this not called anywhere yet?
There was a problem hiding this comment.
it's called in dataflow_transfer.py, by process_run
There was a problem hiding this comment.
I wonder how I missed that. But anyhow, so the sync is triggered once the sequencing is finished but after that we don't really ensure that it finished properly right? Not sure how to change it, but we have had occassional errors on the rsync to the nas-ns. But maybe I misinterpret and its actually attempted again if failed?
There was a problem hiding this comment.
Yes, if the exit code of the rsync is not 0 it will retry. The exception is if the first case, if run.final_sync_successful and run.has_status("transferred_to_hpc") is true, then it wouldn't get to the metadata sync step. Maybe I can add another check in that case.
There was a problem hiding this comment.
I've added some additional logic to process_run so now it should be synced even if the sync to miarka is finished
3 participants
Sync metadata to nas-ns when sequencing is done. Specify what metadata should be synced in the
metadata_rsync_optionspart of the config file.