[Nix-dev] Expression for run time for build environment closures?

[Tyson Whitehead]

Hi All,

Working on merging the cuda and non-cuda expression from the python package theano as was wondering if there is an expression for making a gcc wrapped with certain libraries (like a build environment). That is, theano creates C++ code on the fly, compiles it up, and imports it into your python session. I want to make theano do this compilation inside a nix build environment I setup.

Ideally I would like to set the command theano uses to launch g++ to a script that then runs g++ as if it was in a nix-shell session for a derivation I specify in my theano nix expression. Specifically I want to add all the dependencies that the theano generated code will need in it to ensure they are available and compatible with what theano was built using as otherwise the import explodes.

Is there any easy way to do this sort of thing already? Generating and compiling code on the fly seems to be a popular thing to do at the moment (firedrake is another python example) so I can see this sort of thing would be useful beyond just theano.

Thanks! -Tyson

[Tyson Whitehead]

Post by Tyson Whitehead
Is there any easy way to do this sort of thing already? Generating and compiling code on the fly seems to be a popular thing to do at the moment (firedrake is another python example) so I can see this sort of thing would be useful beyond just theano.

I came up with following

let
wrapped = command: buildInputs:
runCommandCC "${command}-wrapped" { inherit buildInputs; } ''
cat > "$out" <<EOF
#!$(type -p bash)
$(declare -xp | sed -e '/^[^=]\+="\('"''${NIX_STORE//\//\\/}"'\|[^\/]\)/!d')
$(type -p ${command}) "\$@"
EOF
chmod +x "$out"
'';
in
wrapped "gcc" [ ncurses ]

This creates the following sort of shell script file

#!/nix/store/010yd8jls8w4vcnql4zhjbnyp2yay5pl-bash-4.4-p5/bin/bash
declare -x CC="gcc"
declare -x CONFIG_SHELL="/nix/store/010yd8jls8w4vcnql4zhjbnyp2yay5pl-bash-4.4-p5/bin/bash"
declare -x CXX="g++"
declare -x NIX_BUILD_CORES="8"
declare -x NIX_CC="/nix/store/wgvrkijby0awhqkyfkj0qp9cc4921hd9-gcc-wrapper-5.4.0"
declare -x NIX_CFLAGS_COMPILE=" -isystem /nix/store/zsf6vd6q048116rypzi121kmx1565g92-ncurses-6.0-dev/include"
declare -x NIX_ENFORCE_NO_NATIVE="1"
declare -x NIX_ENFORCE_PURITY="1"
declare -x NIX_INDENT_MAKE="1"
declare -x NIX_LDFLAGS="-rpath /nix/store/7fr5dwl7pmbqjbq4286y6xmwx2bs6pbd-g++-wrapped/lib64 -rpath /nix/store/7fr5dwl7pmbqjbq4286y6xmwx2bs6pbd-g++-wrapped/lib -L/nix/store/zsf6vd6q048116$
declare -x NIX_STORE="/nix/store"
declare -x PATH="/nix/store/zsf6vd6q048116rypzi121kmx1565g92-ncurses-6.0-dev/bin:/nix/store/adcj4hwyqnzri00vzjvm9npxncjjmxa2-ncurses-6.0/bin:/nix/store/xyw9r56nwy463549dl7l9bwlvqii5ysn-pat$
declare -x SHELL="/nix/store/010yd8jls8w4vcnql4zhjbnyp2yay5pl-bash-4.4-p5/bin/bash"
declare -x SHLVL="1"
declare -x SOURCE_DATE_EPOCH="1"
declare -x TZ="UTC"
declare -x _PATH="/nix/store/zsf6vd6q048116rypzi121kmx1565g92-ncurses-6.0-dev/bin:/nix/store/adcj4hwyqnzri00vzjvm9npxncjjmxa2-ncurses-6.0/bin:/nix/store/xyw9r56nwy463549dl7l9bwlvqii5ysn-pa$
declare -x buildInputs=""
declare -x builder="/nix/store/010yd8jls8w4vcnql4zhjbnyp2yay5pl-bash-4.4-p5/bin/bash"
declare -x name="g++-wrapped"
declare -x nativeBuildInputs="/nix/store/zsf6vd6q048116rypzi121kmx1565g92-ncurses-6.0-dev"
declare -x out="/nix/store/7fr5dwl7pmbqjbq4286y6xmwx2bs6pbd-g++-wrapped"
declare -x passAsFile="buildCommand"
declare -x propagatedBuildInputs=""
declare -x propagatedNativeBuildInputs=""
declare -x shell="/nix/store/010yd8jls8w4vcnql4zhjbnyp2yay5pl-bash-4.4-p5/bin/bash"
declare -x stdenv="/nix/store/nmzimmm6awzjl8czr17c8fmrdikhw0ma-stdenv"
declare -x system="x86_64-linux"
/nix/store/wgvrkijby0awhqkyfkj0qp9cc4921hd9-gcc-wrapper-5.4.0/bin/g++ "$@"

Cheers! -Tyson