installed terminfo does not load correctly when logging in to a zsh user over ssh #19785

Ralith · 2016-10-22T19:58:56Z

Issue description

On login, /nix/store/yw76cfca2rnhjm68zdf74sy9jwfv2rfy-set-environment:39: can't find terminal definition for xterm-termite is printed and the terminal is not fully functional. Rerunning the failed command, export TERM=$TERM, restores the terminal to full functionality despite TERMINFO_DIRS not having changed from the value it's set to earlier in that script.

Letting the default shell default to bash works around the issue.

Steps to reproduce

Configure a user to use zsh as the default shell and install termite.terminfo in environment.systemPackages.
Log into that user over ssh from inside termite.

Technical details

System: 16.09
Nix version: 1.11.4
Nixpkgs version: 16.09.824.e4fb65a

The text was updated successfully, but these errors were encountered:

pschuprikov · 2016-10-31T20:23:55Z

Probably the issue is unrelated to ssh. I use zsh as my default shell and I get the same error (although for different terminal) when I run su - $(whoami).

pschuprikov · 2016-10-31T21:11:29Z

I think, I got it. The problem is that TERMINFO_DIRS is set here and the corresponding export TERMINFO_DIRS=... is executed inside login shell, thus, login shell process itself doesn't have TERMINFO_DIRS in its own environment.

I added this to configuration.nix:

environment.sessionVariables = {
  TERMINFO_DIRS = "/run/current-system/sw/share/terminfo";
};

And it works now.

I'm not quite certain how to proceed with pull request since I don't understand the purpose of profileRelativeEnvVars and why TERMINFO_DIRS was put there and not in variables.

layus · 2016-11-08T21:42:15Z

It seems that zsh handles updates to TERMINFO correctly, but not to TERMINFO_DIRS. I have submitted a patch to zsh to handle that correctly @ http://www.zsh.org/mla/workers/2016/msg02361.html
The patch ensures that new values for TERMINFO_DIRS are exported to the environent of the running zsh process, which is needed for terminfo to catch it.

That being said, I could not find why bash handles this correctly. There is no mention of TERMINFO_DIRS in its code source... weird.

layus · 2016-11-08T21:44:21Z

Could you bisect the change that triggered this ?
It seems to me that it used to work correctly before.

layus · 2016-11-09T14:23:44Z

Bisecting gives [d2dab39] "ncurses: 5.9 -> 6.0" as the culprit. Looking further, it seems that ncurses comes with its own terminfo database that includes rxvt-256colors but not xrvt-unicode-256colors. I can only assume that it used to be smart enough to fallback to rxvt-256color. The release notes for ncurses 6.0 indicate that the terminfo interface has be reworked, maybe dropping this fallback logic.

layus · 2016-11-09T14:25:32Z

@pschuprikov What is the terminal you use and that causes trouble with su - $(whoami) ? I reproduced you issue with zsh in rxvt-unicode-256colors.

pschuprikov · 2016-11-09T16:37:11Z

Thank you for working on this problem! Yes, *rxvt-unicode-256colors *is the
terminfo that I use.

ср, 9 нояб. 2016 г. в 17:25, Guillaume Maudoux [email protected]:

@pschuprikov https://github.com/pschuprikov What is the terminal you
use and that causes trouble with su - $(whoami) ? I reproduced you issue
with zsh in rxvt-unicode-256colors.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
#19785 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABoVhSZsUdSVghAYwV5WF3X9ro0NVbUPks5q8dfegaJpZM4Kd9Ie
.

layus · 2016-11-10T09:16:07Z

@pschuprikov, @Ralith: Could you try to ln -s ~/.nix-profile/share/terminfo/ ~/.terminfo ? I think it should solve your problem. If it indeed does, then maybe there is no need to patch all the shells to support updates to TERMINFO_DIRS :-).

see NixOS/nixpkgs#19785

unode · 2017-01-24T23:17:07Z

Hi guys, just wanted to say that I also get this but that the workaround on the comment above works for me.

I'm also using zsh with rxvt_unicode. My TERM is slightly different rxvt-unicode-256color instead of colors. @corngood was also able to reproduce the issue while discussing this via IRC.

layus · 2017-01-24T23:51:37Z

Hi @unode, thanks for reporting. Could you also check that this setup does not break other terminal emulators while enabled ? Do xterm and gnome-terminal still work correctly for you ? (It works for me, but better check twice).
If so, I think the fix should be given more visibility, for example by adding it to the manual.

corngood · 2017-01-25T00:02:09Z

@layus 2cfe735 should bring in your zsh change. Is that expected to fix it?

I can test it next time I rebase on master, if nobody else has already.

I have to be honest that I don't really understand why the zsh change would fix it. Are the system environment variables (including profileRelativeEnvVars) passed to the default shell with a shell script, triggering that terminal reset in zsh?

unode · 2017-01-25T00:06:43Z

@layus I can't identify any issue with this set system-wide.
Well nothing I didn't have before so 👍

layus · 2017-01-25T00:11:06Z

FYI, ncurses is a shared lib, and share exactly the same process as the shell. Now, if you rely on your shell to setup TERMINFO* vars to get proper curses interaction with your terminal emulator, it is already too late because curses looks at the environment of the shell itself. To make curses pick the right config, you need either to have that config already setup before the shell starts (solutions 1 and 2) or make the shell update its own process environment (solutions 3) and reinitialize curses data structures.

I think 2 is best, even if I started with 3 (I pushed changes to zsh and fish) before realizing that a solution that requires updating any single shell is not viable. 2 is really the way to go, but it can never hurt to have 3, for us and for other distros.

set environment.sessionVariables (this is bad, because it impacts every process, even system daemons. On every change to this attribute, all the services will restart.)
Use the symlink trick
Patch the shell to make TERMINFO* vars special.

layus · 2017-01-25T00:12:38Z

@unode Thanks ! Feel free to submit a PR to the manual before me :-D.

see NixOS/nixpkgs#19785

Ralith · 2017-06-23T21:59:34Z

@layus fwiw, this is still broken on 17.03, which does seem to have zsh 5.3.1.

It should go without saying that you shouldn't need to manually bodge symlinks to get a non-broken login shell.

layus · 2017-06-26T13:31:33Z

It should go without saying that you shouldn't need to manually bodge symlinks to get a non-broken login shell.

Why ? What makes you expect this ? NixOS uses env vars to pass the terminfo database location. These variables are sourced by the shell. These variables are therefore not in the shell environment when it starts. To get this working, you need special handling in the shell, which is not perfect yet.

In your non-workring shell, try to export TERM=$TERM. It works !
There is apparently something special with login that makes zsh fail to load/find the terminfo.
I am trying to debug this, but I can already tell that a symlink is the simplest way to go.

From man terminfo:

If the environment variable TERMINFO is set, it is interpreted as the pathname of a directory containing the compiled description you are working on. Only that directory is searched.

If TERMINFO is not set, ncurses will instead look in the directory $HOME/.terminfo for a compiled description.

Next, if the environment variable TERMINFO_DIRS is set, ncurses will interpret the contents of that variable as a list of colon-separated directories (or database files) to be searched.
An empty directory name (i.e., if the variable begins or ends with a colon, or contains adjacent colons) is interpreted as the system location /nix/store/k0iyljcvsy6q74d575hlx0dc8jv1lvah-ncurses-6.0/share/terminfo.

Finally, ncurses searches these compiled-in locations:

a list of directories (no default value), and

the system terminfo directory, /nix/store/k0iyljcvsy6q74d575hlx0dc8jv1lvah-ncurses-6.0/share/terminfo (the compiled-in default).

Pick your own fix.

Ralith · 2017-06-26T17:39:38Z

Why ? What makes you expect this ?

It's not acceptable for reasonable usage to be broken out of the box.

NixOS uses env vars to pass the terminfo database location. These variables are sourced by the shell. These variables are therefore not in the shell environment when it starts. To get this working, you need special handling in the shell, which is not perfect yet.

Why not set TERMINFO_DIRS in the environment in which an interactive shell is started, like we do for any other program that has some unusual environment requirements?

I can already tell that a symlink is the simplest way to go.

It's a fine workaround, but a workaround is not a fix.

layus · 2017-06-27T14:44:35Z

@Ralith see #26897 for a fix with terminfo packages installed globally. This should work for you.

For the general case, there is a bug/race condition in zsh, which make my above patch not working in practice. See http://www.zsh.org/mla/workers/2017/msg01035.html

layus · 2017-06-30T12:21:11Z

@Ralith After a two-day marathon chasing the bug in Zsh, I found that ncurses has incorrect caching of env vars, making it ignore our export TERMINFO_DIRS during at most one second. This is why calling export TERM=$TERM in your zsh (over ssh) term should work.

I have submitted a patch upstream, and may consider inserting it into nixpkgs directly (see below)
I believe this, together with #26897, ends this long quest of ours 🎉 .
Would you mind testing the commit (layus@277da2a) on my own branch (https://github.com/layus/nixpkgs/tree/ncurses-caching-bug) to see if it works for you ?

You will need to apply the patch only to zsh's ncurses, otherwise you will end up with a mass-rebuild ;-).

Cheers !

diff --git a/ncurses/tinfo/db_iterator.c b/ncurses/tinfo/db_iterator.c
index 94a4082047..0549dae224 100644
--- a/ncurses/tinfo/db_iterator.c
+++ b/ncurses/tinfo/db_iterator.c
@@ -92,33 +92,33 @@ check_existence(const char *name, struct stat *sb)
  * Store the latest value of an environment variable in my_vars[] so we can
  * detect if one changes, invalidating the cached search-list.
  */
 static bool
 update_getenv(const char *name, DBDIRS which)
 {
     bool result = FALSE;
 
     if (which < dbdLAST) {
 	char *value;
+	char *cached_value = my_vars[which].value;
+	bool same_value;
 
-	if ((value = getenv(name)) == 0 || (value = strdup(value)) == 0) {
-	    ;
-	} else if (my_vars[which].name == 0 || strcmp(my_vars[which].name, name)) {
-	    FreeIfNeeded(my_vars[which].value);
-	    my_vars[which].name = name;
-	    my_vars[which].value = value;
-	    result = TRUE;
-	} else if ((my_vars[which].value != 0) ^ (value != 0)) {
-	    FreeIfNeeded(my_vars[which].value);
-	    my_vars[which].value = value;
-	    result = TRUE;
-	} else if (value != 0 && strcmp(value, my_vars[which].value)) {
+	if ((value = getenv(name)) != 0) {
+	    value = strdup(value);
+	}
+	same_value = (value == 0 && cached_value == 0)
+	    || (value != 0 && cached_value != 0 && strcmp(value, cached_value) == 0);
+
+	// Update var name for later checks
+	my_vars[which].name = name;
+
+	if (!same_value) {
 	    FreeIfNeeded(my_vars[which].value);
 	    my_vars[which].value = value;
 	    result = TRUE;
 	} else {
 	    free(value);
 	}
     }
     return result;
 }

Ralith · 2017-06-30T16:44:02Z

That does indeed work! I still feel like having the environment set up before the shell starts is more natural, but I don't feel strongly, especially since it sounds like zsh upstream intends to have this functionality. Thanks for all your efforts!

layus · 2017-08-17T08:53:53Z

@Ralith wrote:

I still feel like having the environment set up before the shell starts is more natural, but I don't feel strongly, especially since it sounds like zsh upstream intends to have this functionality.

Well, there is no other way to make this work for normal users without nixup or something similar. We want users installing terminfo files in their profile to be able to access them. Setting these env vars before the shell starts is tricky, because the shell is the one responsible for setting env vars.

Anyway, the patch landed upstream, and the related PR was opened @ #28334 \o/

Bump version to include a patch that fixes NixOS#19785.

Ralith · 2017-08-18T00:51:03Z

Setting these env vars before the shell starts is tricky, because the shell is the one responsible for setting env vars.

It's worth noting that this is solvable; either exec a fresh shell after setting things, or set the environment variables with a non-shell mechanism, e.g. by calling setenv(3).

nixos-discourse · 2020-01-28T06:31:31Z

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/error-with-terminfo-when-running-bash-with-nix-run-on-nixos/5660/1

NeQuissimus added 0.kind: bug Something is broken 6.topic: user experience 9.needs: community feedback labels Oct 22, 2016

pSub added a commit to pSub/configs that referenced this issue Jan 21, 2017

[nixops] work around NixOS/nixpkgs#19785

481b168

yacinehmito added a commit to yacinehmito/yarn-nix that referenced this issue Jan 24, 2017

Fix for terminfo

ab25c24

see NixOS/nixpkgs#19785

yacinehmito added a commit to yacinehmito/yarn-nix that referenced this issue Jan 30, 2017

Fix for terminfo

50c52fb

see NixOS/nixpkgs#19785

Ralith mentioned this issue Mar 8, 2017

rxvt_unicode.terminfo broken #23402

Closed

layus mentioned this issue Jun 27, 2017

terminfo: symlink terminfo to /etc for ncurses #26897

Merged

layus mentioned this issue Aug 17, 2017

ncurses: 6.0 -> 6.0-20170729 #28334

Merged

layus added a commit to layus/nixpkgs that referenced this issue Aug 17, 2017

ncurses: 6.0 -> 6.0-20170729

7290d4e

Bump version to include a patch that fixes NixOS#19785.

fpletz removed the 9.needs: community feedback label Aug 22, 2017

FRidh closed this as completed in e6661db Aug 30, 2017

layus mentioned this issue Sep 14, 2018

fish: Could not set up terminal #36146

Closed

This was referenced Jul 22, 2024

prompt curruption when sshing into OSX starship/starship#5105

Closed

installed terminfo does not load correctly when logging in to a zsh user over ssh #329268

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

installed terminfo does not load correctly when logging in to a zsh user over ssh #19785

installed terminfo does not load correctly when logging in to a zsh user over ssh #19785

Ralith commented Oct 22, 2016 •

edited

Loading

pschuprikov commented Oct 31, 2016

pschuprikov commented Oct 31, 2016

layus commented Nov 8, 2016

layus commented Nov 8, 2016

layus commented Nov 9, 2016

layus commented Nov 9, 2016

pschuprikov commented Nov 9, 2016

layus commented Nov 10, 2016

unode commented Jan 24, 2017

layus commented Jan 24, 2017

corngood commented Jan 25, 2017

unode commented Jan 25, 2017 •

edited

Loading

layus commented Jan 25, 2017

layus commented Jan 25, 2017

Ralith commented Jun 23, 2017

layus commented Jun 26, 2017 •

edited

Loading

Ralith commented Jun 26, 2017

layus commented Jun 27, 2017

layus commented Jun 30, 2017

Ralith commented Jun 30, 2017

layus commented Aug 17, 2017

Ralith commented Aug 18, 2017

nixos-discourse commented Jan 28, 2020

installed terminfo does not load correctly when logging in to a zsh user over ssh #19785

installed terminfo does not load correctly when logging in to a zsh user over ssh #19785

Comments

Ralith commented Oct 22, 2016 • edited Loading

Issue description

Steps to reproduce

Technical details

pschuprikov commented Oct 31, 2016

pschuprikov commented Oct 31, 2016

layus commented Nov 8, 2016

layus commented Nov 8, 2016

layus commented Nov 9, 2016

layus commented Nov 9, 2016

pschuprikov commented Nov 9, 2016

layus commented Nov 10, 2016

unode commented Jan 24, 2017

layus commented Jan 24, 2017

corngood commented Jan 25, 2017

unode commented Jan 25, 2017 • edited Loading

layus commented Jan 25, 2017

layus commented Jan 25, 2017

Ralith commented Jun 23, 2017

layus commented Jun 26, 2017 • edited Loading

Ralith commented Jun 26, 2017

layus commented Jun 27, 2017

layus commented Jun 30, 2017

Ralith commented Jun 30, 2017

layus commented Aug 17, 2017

Ralith commented Aug 18, 2017

nixos-discourse commented Jan 28, 2020

Ralith commented Oct 22, 2016 •

edited

Loading

unode commented Jan 25, 2017 •

edited

Loading

layus commented Jun 26, 2017 •

edited

Loading