[Nix-dev] nix-daemon and private git repos

[Harmen]

Hi all,

I'm struggling to get fetchgitPrivate to work on nix-daemon installation (no
NixOS, these are Ubuntu machines with nix).
I can make it work on my dev machine, with is a non-daemon install, by setting

NIX_PATH=ssh-config-file=/my/ssh/config:$NIX_PATH

But that doesn't work in sandboxed daemon mode, because the nixbld* users can't
read that file (both because of access rights, and because of the sandbox).

Note that the config file and any keys it points to must be readable
by the build user, which depending on your nix configuration means making it
readable by the build-users-group, the user of the running nix-daemon, or the
user calling the nix command which started the build. Similarly, if using an
ssh agent ssh-auth-sock must point to a socket the build user can access.
You may need StrictHostKeyChecking=no in the config file. Since ssh
will refuse to use a group-readable private key, if using build-users you will
likely want to use something like IdentityFile /some/directory/%u/key and have
a directory for each build user accessible to that user.

from
https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchgit/private.nix

which sounds reasonable, but it I don't see how to do that since I don't know
exactly which user will run the build. Also because of the sandbox the build
can't read the ssh config file at all.


So next option is to generate the configfile in my expression, a-la
https://www.mpscholten.de/nixos/2016/07/07/private-github-repositories-and-nixos.html
but I don't know how to set nix.path from inside an expression. It would also
require bundling the key with the expression, but if that works...


I can't be the first to want to use fetchgitPrivate with a sandboxed
nix-daemon. Any experiences or tips?
Thanks!
Harmen

[Tomasz Czyż]

You don't need to know the user itself I think, you could share it with
nixbld group (probably).
Also, I think this way recommends to use ssh-agent as far as I remember
looking at that (but could changed).

What I did for one project was:

337 fetchgitPrivate = (args: derivation ((nixpkgs.fetchgit

args).drvAttrs // {
338 # This function is wrapper around nix fetch git function
to use SSH key.
339 # It can be useful when fetching from private
repositories.
340 #
341 # Original function is in
https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchgit/default.nix
342 #
343 GIT_SSH = rr.lib.writeShellScript "git-ssh" ''#!
${nixpkgs.bash}/bin/bash -eu
344 if ! [[ -r '${repositories-ssh-private-key-path}'
]];then
345 echo "ERROR:" >&2
346 echo "ERROR:" >&2
347 echo "ERROR: ssh key
'${repositories-ssh-private-key-path}' is not readable by build user
($(id))" >&2
348 echo "ERROR:" >&2
349 echo "ERROR:" >&2
350 exit 5
351 fi
352 exec ${nixpkgs.openssh}/bin/ssh -o
353 '';

But probably is way better to go with agent.

Hi all,
I'm struggling to get fetchgitPrivate to work on nix-daemon installation (no
NixOS, these are Ubuntu machines with nix).
I can make it work on my dev machine, with is a non-daemon install, by setting
NIX_PATH=ssh-config-file=/my/ssh/config:$NIX_PATH
But that doesn't work in sandboxed daemon mode, because the nixbld* users can't
read that file (both because of access rights, and because of the sandbox).

Note that the config file and any keys it points to must be readable
by the build user, which depending on your nix configuration means

making it

readable by the build-users-group, the user of the running nix-daemon,

or the

user calling the nix command which started the build. Similarly, if

using an

ssh agent ssh-auth-sock must point to a socket the build user can access.
You may need StrictHostKeyChecking=no in the config file. Since ssh
will refuse to use a group-readable private key, if using build-users

you will

likely want to use something like IdentityFile /some/directory/%u/key

and have

a directory for each build user accessible to that user.

from
https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchgit/
private.nix
which sounds reasonable, but it I don't see how to do that since I don't know
exactly which user will run the build. Also because of the sandbox the build
can't read the ssh config file at all.
So next option is to generate the configfile in my expression, a-la
https://www.mpscholten.de/nixos/2016/07/07/private-
github-repositories-and-nixos.html
but I don't know how to set nix.path from inside an expression. It would also
require bundling the key with the expression, but if that works...
I can't be the first to want to use fetchgitPrivate with a sandboxed
nix-daemon. Any experiences or tips?
Thanks!
Harmen
_______________________________________________
nix-dev mailing list
https://mailman.science.uu.nl/mailman/listinfo/nix-dev

[Harmen]

Hi Harmen,

Post by Harmen
I can't be the first to want to use fetchgitPrivate with a sandboxed
nix-daemon. Any experiences or tips?

I had it working but there are several cases which needs extra setup
that I recommend to avoid fetchgitPrivate and use source variables,
e.g. src = <proprietarySrc>; This works the same in any context, e.g. in
hydra, without hydra, etc. You just need to set the proprietarySrc
accordingly.

Thanks for the help Tomas,

could you be a bit more specific what you mean?

Thanks!
Harmen

[Tomas Hlavaty]

Post by Harmen

Hi Harmen,

Post by Harmen
I can't be the first to want to use fetchgitPrivate with a sandboxed
nix-daemon. Any experiences or tips?

I had it working but there are several cases which needs extra setup
that I recommend to avoid fetchgitPrivate and use source variables,
e.g. src = <proprietarySrc>; This works the same in any context, e.g. in
hydra, without hydra, etc. You just need to set the proprietarySrc
accordingly.

could you be a bit more specific what you mean?

Have a look at NIX_PATH: https://nixos.org/nix/manual/#env-NIX_PATH

You can configure the NIX_PATH locally, e.g.

export NIX_PATH=proprietarySrc=$HOME/src/proprietary:$NIX_PATH

or explicitly via command-line arg to nix-build, nix-shell etc.

Then instead of writing your package as:

stdenv.mkDerivation rec {
src = fetchgitPrivate ...;
...
}

write

stdenv.mkDerivation rec {
src = <proprietarySrc>;
...
}

If used in the context of hydra, hydra configuration says, where and how
to obtain that software, then hydra downloads it and passes it further
for build. E.g. in
<https://hydra.nixos.org/jobset/nix/master#tabs-configuration>, there
are paths nix and nixpkgs, which are then written in nix as <nix> and
<nixpkgs> respectively. In this example, no fetchgitPrivate is used but
you can see those paths. In your case of proprietary package, you don't
have to do anything special except allowing hydra to download those
proprietary repos. We have a read-only access to our git repo with ssh
key for hydra.

If you go with the fetchgitPrivate option, you have to grant access in
too many places, i.e. in every context where your package can be
evaluated.

[Alexander V. Nikolaev]

On Mon, Jul 03, 2017 at 03:19:31PM +0200, Harmen wrote:

I have `fetchgitCustom` expression, which can use pre-seeded "deploy"
keys (but with some security implications -- because key is
world-readable). It works with sandbox builds, and should work with
hydra as well.

If anyone interesting in this solution, I'll prepare PR soon.

Post by Harmen
I'm struggling to get fetchgitPrivate to work on nix-daemon installation (no
NixOS, these are Ubuntu machines with nix).
I can make it work on my dev machine, with is a non-daemon install, by setting
NIX_PATH=ssh-config-file=/my/ssh/config:$NIX_PATH
But that doesn't work in sandboxed daemon mode, because the nixbld* users can't
read that file (both because of access rights, and because of the sandbox).

Note that the config file and any keys it points to must be readable
by the build user, which depending on your nix configuration means making it
readable by the build-users-group, the user of the running nix-daemon, or the
user calling the nix command which started the build. Similarly, if using an
ssh agent ssh-auth-sock must point to a socket the build user can access.
You may need StrictHostKeyChecking=no in the config file. Since ssh
will refuse to use a group-readable private key, if using build-users you will
likely want to use something like IdentityFile /some/directory/%u/key and have
a directory for each build user accessible to that user.

from
https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchgit/private.nix
which sounds reasonable, but it I don't see how to do that since I don't know
exactly which user will run the build. Also because of the sandbox the build
can't read the ssh config file at all.
So next option is to generate the configfile in my expression, a-la
https://www.mpscholten.de/nixos/2016/07/07/private-github-repositories-and-nixos.html
but I don't know how to set nix.path from inside an expression. It would also
require bundling the key with the expression, but if that works...
I can't be the first to want to use fetchgitPrivate with a sandboxed
nix-daemon. Any experiences or tips?
Thanks!
Harmen
_______________________________________________
nix-dev mailing list
https://mailman.science.uu.nl/mailman/listinfo/nix-dev

[Tomas Hlavaty]

Post by Alexander V. Nikolaev
I have `fetchgitCustom` expression, which can use pre-seeded "deploy"
keys (but with some security implications -- because key is
world-readable). It works with sandbox builds, and should work with
hydra as well.

And does it work if the package is build by remote build slaves?

[zimbatm]

Yes, the source is part of the build input and is uploaded to the worker to
run the build.

Post by Tomas Hlavaty

Post by Alexander V. Nikolaev
I have `fetchgitCustom` expression, which can use pre-seeded "deploy"
keys (but with some security implications -- because key is
world-readable). It works with sandbox builds, and should work with
hydra as well.

And does it work if the package is build by remote build slaves?
_______________________________________________
nix-dev mailing list
https://mailman.science.uu.nl/mailman/listinfo/nix-dev

[Harmen via nix-dev]

On Tue, Jul 04, 2017 at 08:10:09PM +0000, zimbatm wrote:

Thanks for the suggestions.

I took away from this that it's best to not have Nix deal with the checkouts.
For now I'll keep things as separate repos, to keep things easier.

Eventually I would like to go to one pkgs tree for all repos, but I'll think
about how to do that best. It also ties into integration and system testing,
which is not just Nix related.

thanks!
Harmen