[Nix-dev] weird garbage output inside nix-shell

[Roni Choudhury]

Hi all!

I’m using nix-shell to do some development. I have the environment set up
so I can build my code, but I had a question about invoking packages not in
the shell’s environment. For instance, in that shell, when I run vim, I get
output like this:

vim: àΠbÌ : x #í : Error 224711464
vim: }ÿ : xépOö : Error 1320334120

These two lines came from two different invocations - note how the output
seems to be random.

If I include vim in shell.nix, then I don’t see this problem. (I don’t want
to include vim in shell.nix in this case, because I don’t think the choice
of editor is critical to the build environment, etc.)

My main question is: *what is causing this strange output?*

My secondary question: *how do I invoke nix-shell in such a way as to
instantiate the environment specified in shell.nix, plus a few “custom”
packages (such as vim)?*

I guess I should mention that my shell of choice is zsh. When I run
nix-shell I get dropped into a bash shell. From there, if I invoke zsh I’m
back where I want to be. Could that quirk have something to do with this?

Thanks in advance!

roni
​

[Samuel Leathers]

I use nix-shell --run zsh. I alias that to nshell.

Thanks,

Sam

Post by Roni Choudhury
Hi all!
I’m using nix-shell to do some development. I have the environment set up
so I can build my code, but I had a question about invoking packages not in
the shell’s environment. For instance, in that shell, when I run vim, I
vim: àΠbÌ : x #í : Error 224711464
vim: }ÿ : xépOö : Error 1320334120
These two lines came from two different invocations - note how the output
seems to be random.
If I include vim in shell.nix, then I don’t see this problem. (I don’t
want to include vim in shell.nix in this case, because I don’t think the
choice of editor is critical to the build environment, etc.)
My main question is: *what is causing this strange output?*
My secondary question: *how do I invoke nix-shell in such a way as to
instantiate the environment specified in shell.nix, plus a few “custom”
packages (such as vim)?*
I guess I should mention that my shell of choice is zsh. When I run
nix-shell I get dropped into a bash shell. From there, if I invoke zsh
I’m back where I want to be. Could that quirk have something to do with
this?
Thanks in advance!
roni
​
_______________________________________________
nix-dev mailing list
https://mailman.science.uu.nl/mailman/listinfo/nix-dev

[Roni Choudhury]

Post by Samuel Leathers
I use nix-shell --run zsh. I alias that to nshell.

Thanks, Sam, this is a great idea! On this note, do you know of a good way
to update the prompt to reflect the nix-shell environment I'm in? i.e., is
there a way to *name* the nix-shell env, and get access to that name
through an environment variable?

I wanted to add that even if I don't drop into zsh as my first act in the
nix-shell, I still get that weird output from executables not in the
environment (such as `zsh` and `git`). If anyone has any insight or can
answer my other question, that would be great.

Thanks!

roni

Post by Samuel Leathers

Post by Roni Choudhury
Hi all!
I’m using nix-shell to do some development. I have the environment set
up so I can build my code, but I had a question about invoking packages not
in the shell’s environment. For instance, in that shell, when I run vim,
vim: àΠbÌ : x #í : Error 224711464
vim: }ÿ : xépOö : Error 1320334120
These two lines came from two different invocations - note how the output
seems to be random.
If I include vim in shell.nix, then I don’t see this problem. (I don’t
want to include vim in shell.nix in this case, because I don’t think the
choice of editor is critical to the build environment, etc.)
My main question is: *what is causing this strange output?*
My secondary question: *how do I invoke nix-shell in such a way as to
instantiate the environment specified in shell.nix, plus a few “custom”
packages (such as vim)?*
I guess I should mention that my shell of choice is zsh. When I run
nix-shell I get dropped into a bash shell. From there, if I invoke zsh
I’m back where I want to be. Could that quirk have something to do with
this?
Thanks in advance!
roni
​

_______________________________________________

Post by Roni Choudhury
nix-dev mailing list
https://mailman.science.uu.nl/mailman/listinfo/nix-dev

[Linus Heckemann]

Post by Samuel Leathers
I use nix-shell --run zsh. I alias that to nshell.
Thanks, Sam, this is a great idea! On this note, do you know of a good
way to update the prompt to reflect the nix-shell environment I'm in?
i.e., is there a way to *name* the nix-shell env, and get access to that
name through an environment variable?

Since everything that's passed to stdenv.mkDerivation ends up as an
environment variable, you can use the "name" environment variable. In my
case, my prompt function contains:

if [[ $IN_NIX_SHELL ]]; then
printf "(%s)" "${name:-nix-shell}"
fi

Post by Samuel Leathers
I wanted to add that even if I don't drop into zsh as my first act in
the nix-shell, I still get that weird output from executables not in the
environment (such as `zsh` and `git`). If anyone has any insight or can
answer my other question, that would be great.

This is guesswork, but it could be locale-related. Do you use UTF-8
locales everywhere, or do you use other encodings that might mess up if
interpreted as UTF-8? Does setting LC_ALL=en_US.UTF-8 within the shell help?


Linus

[Sergiu Ivanov]

Hello Roni,

Post by Roni Choudhury

Post by Samuel Leathers
I use nix-shell --run zsh. I alias that to nshell.

Thanks, Sam, this is a great idea! On this note, do you know of a good way
to update the prompt to reflect the nix-shell environment I'm in? i.e., is
there a way to *name* the nix-shell env, and get access to that name
through an environment variable?

nix-shell --run 'MY_SHELL_NAME=foo zsh'

will allow you to tag the shell with a custom environment variable.

If you want to update the prompt, you should probably play around with
shell prompt variables (like PS1 in bash):

http://zsh.sourceforge.net/Doc/Release/Prompt-Expansion.html#Prompt-Expansion

Post by Roni Choudhury
I wanted to add that even if I don't drop into zsh as my first act in the
nix-shell, I still get that weird output from executables not in the
environment (such as `zsh` and `git`). If anyone has any insight or can
answer my other question, that would be great.

My first reaction on seeing your very first mail was that you were
probably having issues with processing of ASCII terminal codes.
Unfortunately, I don't have any clear suggestions.

Have you tried running the commands in different terminal emulators?

--
Sergiu

[Roni Choudhury]

Hi all-

Linus, Sam, and Sergiu - thank you for the suggestions. I did poke around
in the output from `env` and discovered IN_NIX_SHELL, and arranged my
.zshrc to show `${name}` when `$IN_NIX_SHELL` is true. Fantastic!

I'm not sure the other problem has to do with character encodings (though I
don't know too much about how that works). I did notice that `ldd $(which
vim)` shows me different results when I am inside my nix-shell vs. when I'm
not. I kind of thought I shouldn't even be able to launch vim from inside
my shell if it's not listed as a buildDep.

If I send my shell.nix, would someone else mind trying to use it to see if
you can reproduce my strange results?

Thanks!

roni

Post by Sergiu Ivanov
Hello Roni,

Post by Roni Choudhury

Post by Samuel Leathers
I use nix-shell --run zsh. I alias that to nshell.

Thanks, Sam, this is a great idea! On this note, do you know of a good

way

Post by Roni Choudhury
to update the prompt to reflect the nix-shell environment I'm in? i.e.,

is

Post by Roni Choudhury
there a way to *name* the nix-shell env, and get access to that name
through an environment variable?

nix-shell --run 'MY_SHELL_NAME=foo zsh'
will allow you to tag the shell with a custom environment variable.
If you want to update the prompt, you should probably play around with
http://zsh.sourceforge.net/Doc/Release/Prompt-Expansion.html#Prompt-Expansion

Post by Roni Choudhury
I wanted to add that even if I don't drop into zsh as my first act in the
nix-shell, I still get that weird output from executables not in the
environment (such as `zsh` and `git`). If anyone has any insight or can
answer my other question, that would be great.

My first reaction on seeing your very first mail was that you were
probably having issues with processing of ASCII terminal codes.
Unfortunately, I don't have any clear suggestions.
Have you tried running the commands in different terminal emulators?
--
Sergiu

[Sergiu Ivanov]

Hi Roni,

Post by Roni Choudhury
Linus, Sam, and Sergiu - thank you for the suggestions. I did poke
around in the output from `env` and discovered IN_NIX_SHELL, and
arranged my .zshrc to show `${name}` when `$IN_NIX_SHELL` is
true. Fantastic!

Great job! I didn't know about that environment variable.

Post by Roni Choudhury
I'm not sure the other problem has to do with character encodings
(though I don't know too much about how that works).

Ooh, sorry, I messed up in my previous E-mail by saying "ASCII terminal
codes" :-( What I really meant was this:

https://en.wikipedia.org/wiki/ANSI_escape_code

Post by Roni Choudhury
I did notice that `ldd $(which vim)` shows me different results when I
am inside my nix-shell vs. when I'm not.

This is weird. I get exactly the same results (modulo the differences
in the memory map addresses listed in parentheses at the end of each
line). (I don't have vim mentioned in this particular shell.nix.)

Post by Roni Choudhury
I kind of thought I shouldn't even be able to launch vim from inside
my shell if it's not listed as a buildDep.

It looks to me that nix-shell _extends_ the environment it is run in.
So, if you run nix-shell in an environment which contains vim, you have
vim inside nix-shell.

Post by Roni Choudhury
If I send my shell.nix, would someone else mind trying to use it to see if
you can reproduce my strange results?

Go ahead. (Remember to scrub all sensitive information.)

--
Sergiu

[Linus]

Post by Sergiu Ivanov

Post by Roni Choudhury
I kind of thought I shouldn't even be able to launch vim from inside
my shell if it's not listed as a buildDep.

It looks to me that nix-shell _extends_ the environment it is run in.
So, if you run nix-shell in an environment which contains vim, you have
vim inside nix-shell.

Correct — unless you use the --pure option in which case the environment is cleared before adding the build inputs.

[Roni Choudhury]

Post by Sergiu Ivanov

Post by Roni Choudhury
Linus, Sam, and Sergiu - thank you for the suggestions. I did poke
around in the output from `env` and discovered IN_NIX_SHELL, and
arranged my .zshrc to show `${name}` when `$IN_NIX_SHELL` is
true. Fantastic!

Great job! I didn't know about that environment variable.

I think it should be called NIX_IN_SHELL but IN_NIX_SHELL is good too :).

Post by Sergiu Ivanov
I'm not sure the other problem has to do with character encodings

Post by Roni Choudhury
(though I don't know too much about how that works).

Ooh, sorry, I messed up in my previous E-mail by saying "ASCII terminal
https://en.wikipedia.org/wiki/ANSI_escape_code

Ah ok. Well, I use terminal colors in my zsh setup so I can’t imagine it’s
a misinterpreted escape code


Post by Sergiu Ivanov
I did notice that `ldd $(which vim)` shows me different results when I

Post by Roni Choudhury
am inside my nix-shell vs. when I'm not.

This is weird. I get exactly the same results (modulo the differences
in the memory map addresses listed in parentheses at the end of each
line). (I don't have vim mentioned in this particular shell.nix.)

Definitely weird then - is there some way to probe exactly what packages
are in my nix-shell environment? Do I do that just by following symlinks
eventually into the store?

Post by Sergiu Ivanov
I kind of thought I shouldn't even be able to launch vim from inside

Post by Roni Choudhury
my shell if it's not listed as a buildDep.

It looks to me that nix-shell _extends_ the environment it is run in.
So, if you run nix-shell in an environment which contains vim, you have
vim inside nix-shell.

Right, my mistake. Then I’m starting to think it’s that weird shifting of
dynamic libraries that’s causing my shell trouble. In fact, see below


Post by Sergiu Ivanov
If I send my shell.nix, would someone else mind trying to use it to see if

Post by Roni Choudhury
you can reproduce my strange results?

Go ahead. (Remember to scrub all sensitive information.)

Thank you. I’ve attached my shell.nix file to this message.

I did discover that I can “fix” my shell problems by omitting this line
from shell.nix:

LD_LIBRARY_PATH = stdenv.lib.makeLibraryPath libs;

Without that line, my LD_LIBRARY_PATH env var just has
/run/opengl-driver/lib in it, and vim works as expected. I’m not sure why I
did this, because my build seems to work without it. This may explain why
my ldd output differs inside and outside my nix-shell environment.

So I guess my next question is: if I did want to put those packages’
libraries into my library loader path, how am I supposed to do it? I tried
this:

LD_LIBRARY_PATH = "$LD_LIBRARY_PATH:${stdenv.lib.makeLibraryPath libs}";

but that doesn’t work for, perhaps, obvious reasons.

Thanks for the help on this!!

roni
​

[Sergiu Ivanov]

Hey Roni,

Post by Roni Choudhury
Definitely weird then - is there some way to probe exactly what packages
are in my nix-shell environment? Do I do that just by following symlinks
eventually into the store?

There are probably some more intelligent ways of doing that, but I don't
know of them :-(

Post by Roni Choudhury

Post by Roni Choudhury
If I send my shell.nix, would someone else mind trying to use it to see if

Post by Roni Choudhury
you can reproduce my strange results?

Go ahead. (Remember to scrub all sensitive information.)

Thank you. I’ve attached my shell.nix file to this message.

I have no trouble running vim inside the environment provided by your
shell.nix. I get no extra messages.

However, when I try running termite, gedit, or gimp, I get error
messages similar to yours. I suppose your vim command launches gvim,
which uses GTK, just like gedit and gimp. Note that I don't have issues
with qutebrowser, which does not use GTK.

Post by Roni Choudhury
I did discover that I can “fix” my shell problems by omitting this line
LD_LIBRARY_PATH = stdenv.lib.makeLibraryPath libs;
Without that line, my LD_LIBRARY_PATH env var just has
/run/opengl-driver/lib in it, and vim works as expected. I’m not sure why I
did this, because my build seems to work without it.

Omitting this line fixes the issues on my machine as well. Moreover, if
I take your original shell.nix (with redefined LD_LIBRARY_PATH), run
nix-shell and manually set LD_LIBRARY_PATH to /run/opengl-driver/lib, or
even to the _empty string_, things work as expected.

Post by Roni Choudhury
This may explain why my ldd output differs inside and outside my
nix-shell environment.

I didn't observe any differences for any of the applications I tried,
but I didn't look really hard.

Post by Roni Choudhury
So I guess my next question is: if I did want to put those packages’
libraries into my library loader path, how am I supposed to do it? I tried
LD_LIBRARY_PATH = "$LD_LIBRARY_PATH:${stdenv.lib.makeLibraryPath libs}";
but that doesn’t work for, perhaps, obvious reasons.

One way of doing this would be just moving that line into shellHook
(mind the spaces around the equals sign):

shellHook = ''
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${stdenv.lib.makeLibraryPath libs}"
echo "Candela nix dev environment"
'';

The way you tried it originally doesn't work because, apparently, Nix
does not apply shell variable expansion.

If you tried writing something like this, outside of shellHook,

LD_LIBRARY_PATH = "${LD_LIBRARY_PATH}:${stdenv.lib.makeLibraryPath lib}";

you probably saw that LD_LIBRARY_PATH is undefined. I think that's
because the environment variable LD_LIBRARY_PATH is not defined (or is
not accessible via a Nix variable) at this stage of evaluation of
shell.nix (that's a belief I inferred by observation; that is, I don't
know for sure).


Now, if I try launching GTK applications from within the shell.nix
modified in this way, I get the same error message. That may probably
be due to the fact that my GTK applications were built against GTK 3,
while you shell.nix requests GTK 2 (that's a guess, again).

--
Sergiu

[Roni Choudhury]

I use the plain old terminal vim, but you inspired me to start probing
whether one of those libraries was specifically causing the problem.

It turns out to be `glibc` that was causing the trouble. I have no idea why
I thought I needed glibc in my nix-shell environment but there you go. If I
omit that one line from my `shell.nix` then everything seems to work ok.

Sergiu, thanks again for engaging with me and diving into this issue, it's
much appreciated!

roni

Post by Sergiu Ivanov
Hey Roni,

Post by Roni Choudhury
Definitely weird then - is there some way to probe exactly what packages
are in my nix-shell environment? Do I do that just by following symlinks
eventually into the store?

There are probably some more intelligent ways of doing that, but I don't
know of them :-(

Post by Roni Choudhury

Post by Roni Choudhury
If I send my shell.nix, would someone else mind trying to use it to see

if

Post by Roni Choudhury

Post by Roni Choudhury

Post by Roni Choudhury
you can reproduce my strange results?

Go ahead. (Remember to scrub all sensitive information.)

Thank you. I’ve attached my shell.nix file to this message.

I have no trouble running vim inside the environment provided by your
shell.nix. I get no extra messages.
However, when I try running termite, gedit, or gimp, I get error
messages similar to yours. I suppose your vim command launches gvim,
which uses GTK, just like gedit and gimp. Note that I don't have issues
with qutebrowser, which does not use GTK.

Post by Roni Choudhury
I did discover that I can “fix” my shell problems by omitting this line
LD_LIBRARY_PATH = stdenv.lib.makeLibraryPath libs;
Without that line, my LD_LIBRARY_PATH env var just has
/run/opengl-driver/lib in it, and vim works as expected. I’m not sure

why I

Post by Roni Choudhury
did this, because my build seems to work without it.

Omitting this line fixes the issues on my machine as well. Moreover, if
I take your original shell.nix (with redefined LD_LIBRARY_PATH), run
nix-shell and manually set LD_LIBRARY_PATH to /run/opengl-driver/lib, or
even to the _empty string_, things work as expected.

Post by Roni Choudhury
This may explain why my ldd output differs inside and outside my
nix-shell environment.

I didn't observe any differences for any of the applications I tried,
but I didn't look really hard.

Post by Roni Choudhury
So I guess my next question is: if I did want to put those packages’
libraries into my library loader path, how am I supposed to do it? I

tried

Post by Roni Choudhury
LD_LIBRARY_PATH = "$LD_LIBRARY_PATH:${stdenv.lib.makeLibraryPath libs}";
but that doesn’t work for, perhaps, obvious reasons.

One way of doing this would be just moving that line into shellHook
shellHook = ''
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${stdenv.lib.makeLibraryPath libs}"
echo "Candela nix dev environment"
'';
The way you tried it originally doesn't work because, apparently, Nix
does not apply shell variable expansion.
If you tried writing something like this, outside of shellHook,
LD_LIBRARY_PATH = "${LD_LIBRARY_PATH}:${stdenv.lib.makeLibraryPath lib}";
you probably saw that LD_LIBRARY_PATH is undefined. I think that's
because the environment variable LD_LIBRARY_PATH is not defined (or is
not accessible via a Nix variable) at this stage of evaluation of
shell.nix (that's a belief I inferred by observation; that is, I don't
know for sure).
Now, if I try launching GTK applications from within the shell.nix
modified in this way, I get the same error message. That may probably
be due to the fact that my GTK applications were built against GTK 3,
while you shell.nix requests GTK 2 (that's a guess, again).
--
Sergiu

[Sergiu Ivanov]

Post by Roni Choudhury
I use the plain old terminal vim, but you inspired me to start probing
whether one of those libraries was specifically causing the problem.
It turns out to be `glibc` that was causing the trouble. I have no idea why
I thought I needed glibc in my nix-shell environment but there you go. If I
omit that one line from my `shell.nix` then everything seems to work ok.

Oh, all right, I shall accept that as magic :-)

Post by Roni Choudhury
Sergiu, thanks again for engaging with me and diving into this issue, it's
much appreciated!

No problem, glad it helped a bit :-)