[Nix-dev] problem using Haskell libraries that depend on C libraries

[Ganesh Sittampalam]

Hi,

I have a general problem with developing Haskell code in NixOS. It
arises when I directly build a Haskell library that depends on a C
library, rather than installing the Haskell library as a nix package.

The problem is best illustrated with an example using myEnvFun and zlib,
though the problem shows up with many Haskell libraries and in various
places in NixOS:

zlibTest = pkgs.myEnvFun {
name = "zlibTest";
buildInputs = [ stdenv pkgconfig zlib ghc.ghc763
haskellPackages.cabalInstall_1_18_0_3 which ];
};



zlibTest:~$ cabal unpack zlib
[all fine]
zlibTest:~$ cd zlib-0.5.4.1
zlibTest:~/zlib-0.5.4.1$ cabal configure
[all fine]
zlibTest:~/zlib-0.5.4.1$ cabal repl
...
Loading object (dynamic) z ... failed.
<command line>: user specified .o/.so/.DLL could not be loaded
(libz.so: cannot open shared object file: No such file or directory)
Whilst trying to load: (dynamic) z

Similar problems occur when I actually install zlib with cabal install
and try to use it in ghci directly, or try to build something that uses
Template Haskell and depends on it.

I can also reproduce the same problem inside "nix-shell '<nixpkgs>' -A
haskellPackages.zlib".

I think the essential problem is that the configured package doesn't end
up with any reference to the system zlib library, and ghc apparently
doesn't get directly told about the contents of $NIX_LDFLAGS - whereas
gcc is run with a wrapper shell script that does pass these flags in,
which is how cabal configure succeeds.

I'm aware of a couple of workarounds:

(a) Use any Haskell packages of this nature via nix "properly", i.e. put
haskellPackages.zlib into the dependencies.

I'm not too keen on this for a few reasons:

- I might actually be working on such a package.

- I like using nix to easily switch between different GHCs, but I'd
rather do my development in a normal Haskell way that reflects what
users will see.

- Haskell code I work on has a lot of dependencies so it would be a
pain to maintain the nix configuration (presumably cabal2nix could help
me, but it still feels like unwelcome overhead)

(b) Switch the Haskell library to use pkgconfig instead of directly
referring to header and library files

- This seems like generally a good thing but I'm not sure if there are
downsides for the library and I would have to persuade all the relevant
maintainers to accept such a change.

Any suggestions on how I should proceed? I first encountered this
problem quite early on with using NixOS and have since hit it repeatedly.

Cheers,

Ganesh

[Peter Simons]

Hi Ganesh,

Post by Ganesh Sittampalam
zlibTest:~/zlib-0.5.4.1$ cabal repl
...
Loading object (dynamic) z ... failed.
<command line>: user specified .o/.so/.DLL could not be loaded
(libz.so: cannot open shared object file: No such file or directory)
Whilst trying to load: (dynamic) z

you are right: the environment variable $NIX_LDFLAGS ensures that GCC
can find libz (and other systems libraries), which is why "cabal
configure" succeeds. But this won't help GHC locate those libraries,
because it doesn't know about that variable. I guess, we could create a
wrapper around cabal-install to remedy that issue.

Anyway, a simple work-around for this problem is to call:

$ cabal configure ${NIX_LDFLAGS//-L/--extra-lib-dirs=}

Alternatively, you can add appropriate environment variables to the
derivation by means of the extraCmds attribute, i.e. define in
~/.nixpkgs/config.nix:

{
packageOverrides = super: let self = super.pkgs; in
{
zlibTest = self.myEnvFun {
name = "zlibTest";
buildInputs = with self; [
stdenv pkgconfig zlib which ghc.ghc763
haskellPackages.cabalInstall_1_18_0_3
];
extraCmds = ''
export LD_LIBRARY_PATH+=:${self.zlib}/lib
'';
};
};
}

Then "cabal configure && cabal repl" should succeed, too.

Best regards,
Peter

[Ganesh Sittampalam]

Hi Peter,

Post by Peter Simons
you are right: the environment variable $NIX_LDFLAGS ensures that GCC
can find libz (and other systems libraries), which is why "cabal
configure" succeeds. But this won't help GHC locate those libraries,
because it doesn't know about that variable. I guess, we could create a
wrapper around cabal-install to remedy that issue.

Right - or could it be added to the ghc wrapper? It seems logical for it
to get the same behaviour as gcc, but I still haven't got a good
intuition for how this information naturally flows in nix.

Post by Peter Simons
[...]

Thanks for the workarounds, I'll have a play with them and see which one
works best for me.

Post by Peter Simons
export LD_LIBRARY_PATH+=:${self.zlib}/lib

Out of interest, why doesn't nix/nixpkgs set LD_LIBRARY_PATH in the
first place?

Cheers,

Ganesh

[Peter Simons]

Hi Ganesh,

I guess, we could create a wrapper around cabal-install to remedy
that issue.

Could [$NIX_LDFLAGS] be added to the ghc wrapper?

that would probably work, too. Adding the proper flags to "cabal configure" has
the advantage, IMHO, that those paths will be recorded in the package database;
so GHC knows about them. I'm not sure how relevant that difference is, though.

Personally, to me this whole wrapper business feels like a dead-end. I always
prefer ghcWithPackages to build a GHC environment.

Best regards,
Peter