Purpose of this repo is to test airflow with papermill and docker integration (and maybe kubernetes in the future)
Some files:
Dockerfiledescribes simple image with jupter server and papermilldocker-compose.ymlcontains build instructions, run$ docker-compose buildto build the imagerun_pm_from_docker.shthen runs the papermill built imagerun_pm_local.shrun papermill locally
Directories:
out/- dir for papermill to output notebooks; contains some exampleshome/- airflow home, contains dagsnotebooks/- dir with input notebooks for papermill