Messy $PATH with plugins

[stevenwalton]

Currently when plugins are added to sheldon they each get added with their own bin path. So we can get something a bit messy like

echo $PATH
/home/steven/.local/bin:/home/steven/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/steven/.local/share/sheldon/repos/github.com/StackExchange/blackbox/bin:/home/steven/.local/share/sheldon/repos/github.com/z-shell/zsh-diff-so-fancy/bin

or much much worse (this is actually truncated). A big problem is that each plugin's path is quite deep so even a few plugins can greatly expand the size of this variable and make it near unreadable. May God have mercy on those with many plugins.

So I was wondering why do this? Why do we add each of the repo's bins to PATH? Could we not have a more elegant solution like adding ${HOME}/.local/share/sheldon/bin and then when we install a plugin we perform a softlink from the location's bin into the sheldon bin? Our above would become

echo $PATH
/home/steven/.local/bin:/home/steven/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/steven/.local/share/sheldon/bin

which I think we can all agree is much nicer. FWIW libraries such as anaconda and homebrew have similar patterns.

I'm afraid I'm not quite sure where this is defined in the code otherwise I'd make a PR. If you could point me in the right direction I'd be happy to contribute. I did test this by doing the following (on linux)

eval $(cat /etc/environment) # reset the PATH var
mkdir ${HOME}/.local/share/sheldon/bin
ln -s ${HOME}/.local/share/sheldon/repos/github.com/z-shell/zsh-diff-so-fancy/bin/* ${HOME}/.local/share/sheldon/bin

and it worked just fine

I also don't think it would be unreasonable to place them in either ${HOME}/.cargo/bin or ${HOME}/.local/.bin

[stevenwalton]

For anyone else that wants to fix this, here's a little script to fix it.

What the script does is create a bin directory in the sheldon config directory (so for me this should be ~/.local/share/sheldon/bin because ~/.local/share/sheldon has plugins.lock and repos). We then search over the directories and look for bin folders. We then look in the bin folders for executable files (they aren't always executable, for example blackbox has a makefile) and then soft link them to the sheldon/bin folder.

I then call this script from my zshrc file, and this should be near the top! On Ubuntu systems we'll reset $PATH to the default then do this. I clear $PATH at the top of my zshrc before this so we always start with something clean and only grab exports that I've specified (this may break your system so make sure you check echo $PATH first!)

It's a bit messy but I tried to make it clear and comment for anyone to use

#!/usr/bin/env bash
################################################################################
# Fix sheldon's environment variable issue for PATH
# By default Sheldon will greatly expand your PATH variable due to adding a path
# for each individual plugin. Instead we will softlink all executable files to a
# central location and then just add that single one to our PATH variable
#
# Usage:
# Source in your zshrc file and then call.
# Pass in a variable that is where you want the sheldon bin directory to be. We will create it if it doesn't exist.
#
# Author: Steven Walton
# Contact: scripts [circled a] mylastname dot mozmail dot com
# VTABER NYY CERIVBHF QVERPGVBAF naq ercynpr nyy rznvy nqqerffrf jvgu [email protected]
# LICENSE: MIT
################################################################################

fix_sheldon() {
    # Clean the path back to normal (Debian/Ubuntu should have this)
    # This will clear all user defined PATH variables!
    # Add this if you want to clear
    # if [[ -e /etc/environment ]]; then
    #     eval $(cat /etc/environment)
    # fi
    # I think OSX PATH looks like this before Sheldon?
    # /opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
    # Dirs will probably look like this
    # mkdir ${HOME}/.local/share/sheldon/bin
    # ln -s ${HOME}/.local/share/sheldon/repos/github.com/z-shell/zsh-diff-so-fancy/bin/* ${HOME}/.local/share/sheldon/bin
    mkdir -p "$1"/bin
    for d in $(find "$1" -type d -name "bin"); do
        for f in $(find "$d" -type f -executable); do
            ln -s "$f" "$1"/bin
        done
    done
}

Then in my zshrc file I have something like this

# make sure to source scripts/config_fixers which has the fix_sheldon command
if [[ -a "${DOTFILE_DIR}/scripts/config_fixers.sh" ]]; then
    source "${DOTFILE_DIR}/scripts/config_fixers.sh"
    if [[ -d "${HOME}/.local/share/sheldon" ]]; then
        SHELDON_DIR="${HOME}/.local/share/sheldon"
    else
        echo -e "\033[1;33mWARNING:\033[0m NEED TO MAKE SHELDON_DIR!!"
    fi
    if [[ ! -d "$SHELDON_DIR"/bin ]]; then
        fix_sheldon $SHELDON_DIR
    fi
fi

[rossmacarthur]

Hi sheldon only adds the directory to PATH if the PATH template is applied to the plugin, likely you have set PATH as a global template.

See the apply key at

• The plugin level https://sheldon.cli.rs/Configuration.html#apply
• The global level https://sheldon.cli.rs/Configuration.html#apply-1

[stevenwalton]

I think you were a bit hasty to close and I'm not sure this resolves the issue. Can we get to a resolution or clarity before we close? If I am misunderstanding please help me understand. If I actually have contribution (I think I am), then let's talk. Neither of those have happened. I understand you're busy, I am too, but we can work together, not against each other.

Here's the issue:

1. I think this could be communicated a bit better in the documentation and getting started. This is the behavior of inaction, not action. Here is my plugins.toml. So I have not "set" anything. As you can see, I actually directly follow examples you provided. The only apply I have set is from defer. So the default action is adding to PATH. (This makes sense, since we need to execute those programs)

2. The problem isn't that all directories are being added to PATH, the problem is that there are more than 1. The commands need to go somewhere, right? And they eventually need to be added to PATH if I am ever to use them. So what I was showing is that we can have a less cluttered version where there is a singular item added to PATH regardless of the number of plugins used.

2 is the main issue here. There's a further simplification we can take too. We could add these softlinks to ~/.local/bin if we wanted to by setting a different path. The point being to get Sheldon's plugin manager to act like any other package manager, organizing and keeping environment variables clean. Many users physically look at PATH to debug, and making this over-encumbers them.

So I'd propose two solutions.

1. Binaries from plugins get placed into a directory defined by some environment variable (e.g. SHELDON_BIN) and only that path gets added to PATH (if it isn't already in PATH).
2. Do what I did, and create a unique bin directory for Sheldon and link the executables to that.

The reason I prefer option 2 is it helps organizing and determining where executables are coming from, and allows multiple versioning and options. This would be consistent with something like homebrew which installs everything into /opt/homebrew rather than into /usr/bin or /usr/local/bin. This makes the uninstalling process easier and allows users to better track down orphans when trying to clean up. brew differs a bit in that it creates a whole root like directory structure in /opt/homebrew and I'm not sure this is necessary here or even cleaner, since we probably just need the binaries.

[rossmacarthur]

Hi,

The default apply does not edit the PATH, it just applys the source template which just sources each file. It is true that the PATH template exists (and it does not check if the path is already added #177), but this template is not enabled by default. So your PATH must be being modified another way. You can view the exact source that is generated by sheldon by inspecting the output of

sheldon source

With regards to your solutions, it is possible to get this using templates. Something like the following might work, which symlinks each file to a directory of your choice.

[templates]
bin = """{% for file in files %}ln -s "{{ file }}" ~/.local/bin/$(basename "{{ file }}")
{% endfor %}
"""

[stevenwalton]

I updated the template to not error if link already exists (my original had this issue too btw)

[templates]
bin = """{% for file in files %} [ -h "{{ file }}" ] && ln -s "{{ file }}" ~/.local/bin/$(basename "{{ file }}")
{% endfor %}
"""

But I noticed fancy-diff still loaded...

I think this helped me figure out an issue, which isn't directly Sheldon but is going to end up affecting users and maybe could be accounted for? I noticed this by looking at blackbox.plugin.zsh. There are only two lines here

PLUGIN_BIN="$(dirname $0)/bin"
export PATH="${PATH}:${PLUGIN_BIN}"

Which are going to mess with our path. So yes, not Sheldon, but I think it is quite reasonable to expect many plugins to be written this way. We see a similar pattern in zsh-diff-so-fancy.plugin.zsh, though it looks like a lot are using fpath instead, which I think might be a bit more expected. (This one might be worth looking at, since they are referencing the docs)

At least this template serves as a solution for the style done by blackbox but diff-so-fancy is doing something else and even switching to fpath is not the resolution.

So, not Sheldon. I'm not sure if there's something that we can do here to unencumber users when plugins are setup this way, or if we push issues towards those maintainers. I'm now cool if you want to close or I am also willing to help find some solution.