apt-get install python3-rospkg fails due to dependency conflict.

[linas]

I get this:

apt-get install python3-rospkg
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  python3-rospkg
0 upgraded, 1 newly installed, 0 to remove and 10 not upgraded.
Need to get 22.3 kB of archives.
After this operation, 155 kB of additional disk space will be used.
Get:1 http://packages.ros.org/ros/ubuntu/ trusty/main python3-rospkg all 1.0.31-1 [22.3 kB]
Fetched 22.3 kB in 0s (62.0 kB/s)         
Selecting previously unselected package python3-rospkg.
(Reading database ... 283255 files and directories currently installed.)
Preparing to unpack .../python3-rospkg_1.0.31-1_all.deb ...
Unpacking python3-rospkg (1.0.31-1) ...
dpkg: error processing archive /var/cache/apt/archives/python3-rospkg_1.0.31-1_all.deb (--unpack):
 trying to overwrite '/usr/bin/rosversion', which is also in package python-rospkg 1.0.30-1
Errors were encountered while processing:
 /var/cache/apt/archives/python3-rospkg_1.0.31-1_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

and removing the python2 version is not an option:

apt-get remove python-rospkg
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
...
Use 'apt-get autoremove' to remove them.
The following packages will be REMOVED:
  python-rosdep python-rosdistro python-rosinstall python-rospkg
  ros-indigo-actionlib ros-indigo-actionlib-tutorials ros-indigo-bond-core
...
0 upgraded, 0 newly installed, 147 to remove and 10 not upgraded.
After this operation, 90.1 MB disk space will be freed.
Do you want to continue? [Y/n] n

[linas]

p.s. if there is a better place to report this bug, let me know. Putting this onto the ubuntu launchpad bug system seems wrong, since this is not an ubuntu-controlled repo.

[wjwwood]

Sorry, but you cannot install the Python3 deb for rospkg at the same time as the Python2 deb. If you need to do that then I recommend that you use a virtualenv instead of the debian package for it.

I think this is a duplicate of this one: #69

[linas]

OK, makes sense, but then installing one should cause the other to be automatically uninstalled. So I just discovered that

apt-get install python3-rosinstall 

hits this all over the place, for example:

Selecting previously unselected package python3-rosinstall.
Preparing to unpack .../python3-rosinstall_0.7.3-1_all.deb ...
Unpacking python3-rosinstall (0.7.3-1) ...
dpkg: error processing archive /var/cache/apt/archives/python3-rosinstall_0.7.3-1_all.deb (--unpack):
 trying to overwrite '/usr/bin/rosws', which is also in package python-rosinstall 0.7.3-1
Errors were encountered while processing:
 /var/cache/apt/archives/python3-catkin-pkg_0.2.6-1_all.deb
 /var/cache/apt/archives/python3-rospkg_1.0.31-1_all.deb
 /var/cache/apt/archives/python3-rosdistro_0.3.7-1_all.deb
 /var/cache/apt/archives/python3-wstool_0.1.4-1_all.deb
 /var/cache/apt/archives/python3-rosinstall_0.7.3-1_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

A partial cure is to first apt-get remove python-rosinstall and then aapt-get autoremove ... I think I should be able to walk through all of these manually, but its a bit of a pain... BTW, this is driven by blender, which needs the latest python3 code.

[linas]

I spoke too soon. trying to manually resolve this gets ugly, fast: apt-get remove python-rosdistro python-rospkg python-catkin-pkg promptly results in:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
You might want to run 'apt-get -f install' to correct these:
The following packages have unmet dependencies:
 python-rosdep : Depends: python-catkin-pkg but it is not going to be installed
                 Depends: python-rospkg but it is not going to be installed
                 Depends: python-rosdistro (>= 0.3.0) but it is not going to be installed
 python3-rosinstall : Depends: python3-catkin-pkg but it is not going to be installed
                      Depends: python3-rosdistro (>= 0.3.0) but it is not going to be installed
 ros-indigo-catkin : Depends: python-catkin-pkg (>= 0.2.2) but it is not going to be installed
...
etc. and another 30 pacakges or so.

[wjwwood]

You will have to uninstall all of ROS if you want to install the Python3 version of the python packages like rospkg and friends.

[linas]

well, yes, I now understand. However, right now, I cannot install or uninstall anything, because apt errors out. Running apt-get -f install simply retries to install the uninstallable packages ... which also cannot be removed because they are not installed. In short, every possible apt command now errors out ...

Recall I got into this mess in a simple, legit two-step process:
apt-get install python3-rosinstall fails
apt-get install python3-catkin-pkg python3-rosdistro python3-rospkg fails

that's it -- apt is is now unusable, one can neither install or uninstall, and apt-get -f install does nothing at all.

[dirk-thomas]

If two packages are colliding because they try to install the same file the installation of the second package fails. But this does not leave the packages in an invalid state. You must have called something specific to trigger the problem.

I will mark this ticket as "wontfix" and close it since we don't see a way to support installing both packages side-by-side.

[linas]

I'm not asking for them to be installed side by side.

I am asking that the .debs have the correct dependencies marked in them, so that install conflicts cannot occur. Right now, the .debs fail to have the dependencies correctly marked, and so one can quickly find oneself in a huge mess. In particular, it very easily -- almost trivially, puts you into an invalid state!

[dirk-thomas]

The debs should have the correct dependencies marked in them. Please point me to a specific Debian and which dependency it is missing.

When I try to install the python3 version of the package it simply aborts the installation. It does not corrupt the package state for me. Can you please post a command sequence which allows to reproduce the problem?

The only think "missing" in the Debian package is that it could state a Conflict between the Python 2 and Python 3 package. This is partially due to the limitation of stdeb which does not provide a configuration option for conflicts in Python 3 packages (astraw/stdeb#81).

[linas]

To be clear: based on the observed behaviour, it seems that the .debs for the misc python3-ros* packages are missing a "Conflicts" field, per the usual https://www.debian.org/doc/debian-policy/ch-relationships.html section 7.4

[wjwwood]

@linas I think you are right, we should have the conflict so that it never even gets to the installation failure where the binaries collide, but as @dirk-thomas pointed out, the infrastructure we use for generating Python releases does not support it currently.

I am going to open a ticket on our python releasing infrastructure to track this problem, but for now we cannot fix it in a sustainable way.

ros-infrastructure/ros_release_python#9

[linas]

Yes, thank you. In trying to get a cleanly reproducible test, I just ended up further down the rabbit hole.

[linas]

I'm not convinced its even possible to install ros and python3-rospkg at the same time. Here's a fairly simple scenario:

Start with a clean system by purging ros-indigo-* and python-ros* (i.e. everything I could find by saying dpkg-query -l *ros*). Then

apt-get install ros-indigo-ros-base         # (installs some 154 packages)
apt-get remove python-rospkg      #  (because I know it will conflict with python3-rospkg)
apt-get autoremove        # (cleanup. Looks like it removes all 154...)
apt-get install python3-rospkg    # success!
apt-get install ros-indigo-ros-base     # fail!

I can work around this by installing the base first, and then just forcing past the error with
dpkg -i --force-overwrite python3-rospkg_1.0.31-1_all.deb

which sems mostly harmless, in the end: its just clobbering /usr/bin/rosversion with some different /usr/bin/rosversion

The dumb solution for this would be to create a package python-ros-pkg-comon which contains only /usr/bin/rosversion and then have both python-rospkg and pythong3-rospkg depend on it :-?

[dirk-thomas]

And what Python version does the shebang line in the rosversion file from your python-ros-pkg-comon package reference?

[wjwwood]

I'm not convinced its even possible to install ros and python3-rospkg at the same time. Here's a fairly simple scenario:

Even if it were possible to install them at the same time, it would not run correctly because all of the ROS debians which use the rospkg python API will not find it since they are Python2 and you'll have the Python3 one installed only.

I'll just point out my original suggestion. If you need python3-rospkg then use a virtual environment.

[linas]

Hmm. Thanks, again. I will try that. Right now, I'm in pain, as I've been given some ROS nodes that run in blender, and blender only supports python3. Its starting to feel like the good-old bad days of perl...

[roobooot]

ros/rosdistro#14054 (comment)