python : workspace and virtual envs

[asmodehn]

When working on a python package, I usually use virtuals env, ( supported by many python tools ).
However when depending on ros sources, I usually work with a catkin devel workspace.

I was wondering if there was a way to "merge" this two concepts ?
Like, make ROS tools find and use a virtual env, or make virtualenv tools think the catkin devel workspace is a virtual env ?

[gerkey]

I'm not a virtual envs user myself, so I'm not the best person to ask. I suspect that there is some way to make virtual envs and catkin rorkspaces play together. They both seem to be directories with certain structures and setup files. It would be great if you could suggest a way of doing that as an addition to this repo.

[asmodehn]

After a quick preliminary research, there are at least two big differences that I am aware about:

• virtualenvs work with pip, which works with package versions. The whole point since we want to be sure of which version of the dependencies we are using (code changes fast).
• workspaces allow overlaying (they can be stacked), but there is no version check there(unless you do it in code)

I think this might actually make them incompatible...

• The way overlayed workspace work is that when you import mypackage it goes down the stack to find a mypackage to import, using the first one it can find. There is no version check possible here without enforcing some kind of package version structure in the code of all packages itself... and we're not here to change python, or to add restriction on which package can work with workspaces (quite the opposite actually since the point is to use any python code with their virtualenv workflow)
• Package versions in python virtualenv are insured by pip. When you specify a package with a version to install, you know that this package version will be found and used by your code. Having a stack of virtualenv would break this implicit promise (b depends on c v0.1 and a depends on b and c v0.2. Imagine stacking 2 venv and you don't know which version of c b is using anymore).

And thinking about it there is likely the issue with workspace overlay when we start caring about the versions of our packages :
workspace A overlays workspace B, A has package Pa and Pc v0.2, B has package Pb and Pc v0.1.
I run A, which calls B. Isn't B using Pc v0.2 now ?

I might need to investigate and confirm that sometime...

[asmodehn]

I m currently working on https://github.com/asmodehn/catkin_pure_python to allow catkin users to manipulate python packages directly with pip ( instead of custom catkin code ).

I tried a few different approach and the working solution so far is to treat the workspace as a "prefix" from pip options. Except a small hack, everything seems to work correctly in the workspace ( devel or install ), including overlays.

I ll PR something about it to this repo sometime soon