Improved zsh completion time from ~0.5s to near instant

[racko]

The "catkin locate -s" takes around 0.5s. It gets called whenever the cache is checked.

[wjwwood]

This is a flawed assumption, the source directory is not always <workspace dir>/src. Since it can be any folder, that's why catkin locate is required to look up the setting.

[racko]

pkg_resources.require("catkin_tools")[0].version in migrate_metadata is "the problem":

time python2 -c 'import pkg_resources; pkg_resources.require("catkin_tools")[0].version'
python2 -c   0.14s user 0.01s system 99% cpu 0.148 total

Making migrate_metadata a no-op reduces the time for get_active_metadata to less than 30ms. Of course we can't do that.

Any idea if there is a way to replace pkg_resources.require("catkin_tools")[0].version with something (a lot) faster?

TODO: get_active_metadata and update_active_metadata are broken (but unused). They miss return keywords.

---

Well ...

import catkin_tools.context
ctx = catkin_tools.context.Context.load('/home/racko/catkin_ws', None, type('Options', (object,), dict(space='src'))(), load_env=False)
print ctx.source_space_abs

takes around 250ms.

from catkin_tools.metadata import get_metadata, get_active_profile
print get_metadata('/home/racko/catkin_ws', get_active_profile('/home/racko/catkin_ws'), 'config')['source_space']

takes around 175ms. Calling get_metadata or get_active_profile alone also takes around 175ms, so among other things, we can conclude that calling get_active_profile and then reading the config.yaml with another tool will not be good enough. But still it is interesting to measure.

import yaml
print yaml.load(open('/home/racko/catkin_ws/.catkin_tools/profiles/default/config.yaml'))['source_space']

takes around 30ms.

yaml2json - < /home/racko/catkin_ws/.catkin_tools/profiles/default/config.yaml | jq '.source_space'

takes around 20ms.

Also caching the source space path is not an option: To check if the cache is invalid, we need to check the date of the config.yaml, and getting the path takes around 175ms again ...

Reimplementing the get_active_profile logic in zsh (or some compiled language) would be trivial and fast enough, but duplicating the logic is of course not an option. And I'm certain that implementing the logic in C++ and using it in python (similar to tf) was decided against long before I came along, besides it being to big of a change for this pull request :)

What about adding a special case for "$_workspace_root/src" since it's what works for 99% of all users? It is the recommended layout after all ...

Not even this special case works: Finding "$_workspace_root/src" does not mean that it is the source space :(

[wjwwood]

If the issue is only due to asking for the version number, perhaps we could delay acquiring that information until it is asked for.

However, I think in general the entry points performance is a bigger issue, see: #297

[racko]

I close this pull request because the proposed solution isn't actually working:

What about adding a special case for "$_workspace_root/src" since it's what works for 99% of all users? It is the recommended layout after all ...

Not even this special case works: Finding "$_workspace_root/src" does not mean that it is the source space :(

Also my suggestion that replacing pkg_resources.require("catkin_tools")[0].version would fix the problem seems to be false: While it certainly speeds up migrate_metadata, I couldn't observe a speed-up for catkin locate -s or the zsh auto-complete. I guess that pkg_resources.require("catkin_tools") or something similar gets called anyhow.