Fix for use of tensor flow

[jpienaar13]

Changes to OSG-sites in order for MC processing to work with TensorFlow

[pdeperio]

@briedel Does this reduce our pool significantly?

[briedel]

This does decrease the pool significantly. I don't recall: How is the tensorflow step separate from the rest? We should be able to run with a different requirements expression for that step. @sthapa ?

[sthapa]

It might be worth it to separate out the steps a bit. Run the MC stuff with one set of requirements and then to do the tensorflow with other requirements. I'll check the pegasus docs to see if this is possible.

The biggest change would be to update mc_process to specify the outputs from the steps prior to tensorflow and then the tensorflow input steps. I can modify the mc_process script to do this.

[sthapa]

Checked and it looks like we can use per job requirements and have different steps for the tensorflow jobs. Just need to determine files that need to be copied as input for that and update the mc_process.py to transfer the files and set the job requirements.

[pdeperio]

This is not so urgent now that we've merged XENON1T/hax#205 (we're not actually using TF yet in the main analysis).

Also, the stage (data reduction) that uses TF is quite fast, so still not sure it's worth copying the whole (processed/reconstructed) file per job.

So I think we can wait for sites to upgrade or some fix for TF compatibility.