ingest/extract_open_data: Do not compress intermediate files

[joverlee521]

Description of proposed changes

Use uncompressed outputs so that the uploaded files are no longer double compressed.¹

Once upload-to-s3 supports compressed local files, we can go back to using compressed intermediate files.

Related issue(s)

Related to #343

Checklist

• Checks pass
• Update changelog

[joverlee521]

Merging without review to fix the files on S3. Will re-run the ingest workflow after merge.

[joverlee521]

Running ingest: https://github.com/nextstrain/mpox/actions/runs/19347642766

[corneliusroemer]

Thanks and sorry for this. How did you notice the double compression? It hasn't caused any issues in any of our CI runs as far as I can tell?

[joverlee521]

Thanks and sorry for this. How did you notice the double compression? It hasn't caused any issues in any of our CI runs as far as I can tell?

Yeah, the CI uses example data and the automated phylo workflows start from the *_with_restricted files, so nothing flagged this issue. I only noticed because I was curious why the metadata diff on Slack started reporting "Binary files /tmp/s3-file-GVSf4x and results/metadata.tsv differ". This led me to realize the S3 path for the diff was incorrect and prompted me to take a look at the files on S3.

[joverlee521]

the automated phylo workflows start from the *_with_restricted files, so nothing flagged this issue.

It would be good for us to use both the OPEN and RESTRICTED files and to not host duplicate OPEN records on S3. Updating ingest to upload completely separate files for OPEN and RESTRICTED records in #347