Skip to content

tapis V2 data exploration #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
swollenburg wants to merge 6 commits into
base: master
Choose a base branch
from
Open

tapis V2 data exploration #14

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
871fd33
tapis V2 data exploration
swollenburg Feb 26, 2025
2aa3db2
added code for writing metadata into json file.
Tanzira May 16, 2025
bf9ce46
migration of public projects
schristley Jun 22, 2025
bd19e4b
public c project migration
schristley Jul 1, 2025
cf22b0a
Added user migration script.
Tanzira Aug 8, 2025
cb645d2
fix merge conflicts
schristley Oct 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 23 additions & 2 deletions conversion/tapis_v2_to_v3/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,14 @@ validated in the migration.
We assume that V2 API is not available and that the data to be migrated
resides in a JSON file.

## Docker
## Jupyter notebooks

We build a specialized docker image that brings in vdj-tapis-js and vdjserver-schema
The extract and transform fuctions are performed in the Jupyter notebooks
and generate JSONL files with meta records that can be bulk uploaded with
meta_load_records.js script in the tapis-conversion docker container. It
resdes in the adc-api-tapis-js of vdjserver-repository.

- public_projects.ipynb: Public Projects

## Users

Expand Down Expand Up @@ -82,3 +86,20 @@ so they are either old or testing.
"bioProcessing": 5,
"irplus_analysis": 3,

# Migration Tasks

## Public projects

This conversion maintains uuids for the records.

- public_projects.ipynb: Public Projects

This will generate files in the directories:

- Metadata_public_project: One file per project containing metadata records.
- Metadata_public_project_jobs: One file per project containing Tapis V2 job records.

As there aren't many files, manually load the files one at a time using
meta_load_records.js in the tapis-conversion docker.

##
Loading