Workflow: define job/node options and arguments properly in glue.pipeline

[astroclark]

The glue.pipeline module has a useful method dag.write_script() which writes out a bash script with the full contents of the DAG.

In the current implementation, calling this method fails due to missing job arguments (the code searches for existing DAG node arguments and options and expands any macros present).

It's probably because the command line for each node in the dag is being constructed explicitly. See here.

Namely, you're constructing the list of options and arguments by building a string by hand.

glue.pipeline has methods to do that automatically see e.g.,:

• pipeline.add_opt
• add_arg
• add_var_opt
• add_var_arg

This is very useful if you then want to use methods like get_opts on a node as is required by the write_script method.