Discussion:
nix-daemon and private git repos
(too old to reply)
Harmen
2017-07-03 13:19:31 UTC
Permalink
Hi all,

I'm struggling to get fetchgitPrivate to work on nix-daemon installation (no
NixOS, these are Ubuntu machines with nix).
I can make it work on my dev machine, with is a non-daemon install, by setting

NIX_PATH=ssh-config-file=/my/ssh/config:$NIX_PATH

But that doesn't work in sandboxed daemon mode, because the nixbld* users can't
read that file (both because of access rights, and because of the sandbox).
Note that the config file and any keys it points to must be readable
by the build user, which depending on your nix configuration means making it
readable by the build-users-group, the user of the running nix-daemon, or the
user calling the nix command which started the build. Similarly, if using an
ssh agent ssh-auth-sock must point to a socket the build user can access.
You may need StrictHostKeyChecking=no in the config file. Since ssh
will refuse to use a group-readable private key, if using build-users you will
likely want to use something like IdentityFile /some/directory/%u/key and have
a directory for each build user accessible to that user.
from
https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchgit/private.nix

which sounds reasonable, but it I don't see how to do that since I don't know
exactly which user will run the build. Also because of the sandbox the build
can't read the ssh config file at all.


So next option is to generate the configfile in my expression, a-la
https://www.mpscholten.de/nixos/2016/07/07/private-github-repositories-and-nixos.html
but I don't know how to set nix.path from inside an expression. It would also
require bundling the key with the expression, but if that works...


I can't be the first to want to use fetchgitPrivate with a sandboxed
nix-daemon. Any experiences or tips?
Thanks!
Harmen
Tomasz Czyż
2017-07-03 13:35:18 UTC
Permalink
You don't need to know the user itself I think, you could share it with
nixbld group (probably).
Also, I think this way recommends to use ssh-agent as far as I remember
looking at that (but could changed).

What I did for one project was:

337 fetchgitPrivate = (args: derivation ((nixpkgs.fetchgit
args).drvAttrs // {
338 # This function is wrapper around nix fetch git function
to use SSH key.
339 # It can be useful when fetching from private
repositories.
340 #
341 # Original function is in
https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchgit/default.nix
342 #
343 GIT_SSH = rr.lib.writeShellScript "git-ssh" ''#!
${nixpkgs.bash}/bin/bash -eu
344 if ! [[ -r '${repositories-ssh-private-key-path}'
]];then
345 echo "ERROR:" >&2
346 echo "ERROR:" >&2
347 echo "ERROR: ssh key
'${repositories-ssh-private-key-path}' is not readable by build user
($(id))" >&2
348 echo "ERROR:" >&2
349 echo "ERROR:" >&2
350 exit 5
351 fi
352 exec ${nixpkgs.openssh}/bin/ssh -o
353 '';
But probably is way better to go with agent.
Hi all,
I'm struggling to get fetchgitPrivate to work on nix-daemon installation (no
NixOS, these are Ubuntu machines with nix).
I can make it work on my dev machine, with is a non-daemon install, by setting
NIX_PATH=ssh-config-file=/my/ssh/config:$NIX_PATH
But that doesn't work in sandboxed daemon mode, because the nixbld* users can't
read that file (both because of access rights, and because of the sandbox).
Note that the config file and any keys it points to must be readable
by the build user, which depending on your nix configuration means
making it
readable by the build-users-group, the user of the running nix-daemon,
or the
user calling the nix command which started the build. Similarly, if
using an
ssh agent ssh-auth-sock must point to a socket the build user can access.
You may need StrictHostKeyChecking=no in the config file. Since ssh
will refuse to use a group-readable private key, if using build-users
you will
likely want to use something like IdentityFile /some/directory/%u/key
and have
a directory for each build user accessible to that user.
from
https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchgit/
private.nix
which sounds reasonable, but it I don't see how to do that since I don't know
exactly which user will run the build. Also because of the sandbox the build
can't read the ssh config file at all.
So next option is to generate the configfile in my expression, a-la
https://www.mpscholten.de/nixos/2016/07/07/private-
github-repositories-and-nixos.html
but I don't know how to set nix.path from inside an expression. It would also
require bundling the key with the expression, but if that works...
I can't be the first to want to use fetchgitPrivate with a sandboxed
nix-daemon. Any experiences or tips?
Thanks!
Harmen
_______________________________________________
nix-dev mailing list
https://mailman.science.uu.nl/mailman/listinfo/nix-dev
--
Tomasz CzyŌ
Harmen
2017-07-03 13:38:43 UTC
Permalink
Hi Harmen,
Post by Harmen
I can't be the first to want to use fetchgitPrivate with a sandboxed
nix-daemon. Any experiences or tips?
I had it working but there are several cases which needs extra setup
that I recommend to avoid fetchgitPrivate and use source variables,
e.g. src = <proprietarySrc>; This works the same in any context, e.g. in
hydra, without hydra, etc. You just need to set the proprietarySrc
accordingly.
Thanks for the help Tomas,

could you be a bit more specific what you mean?

Thanks!
Harmen
Tomas Hlavaty
2017-07-03 15:11:34 UTC
Permalink
Post by Harmen
Hi Harmen,
Post by Harmen
I can't be the first to want to use fetchgitPrivate with a sandboxed
nix-daemon. Any experiences or tips?
I had it working but there are several cases which needs extra setup
that I recommend to avoid fetchgitPrivate and use source variables,
e.g. src = <proprietarySrc>; This works the same in any context, e.g. in
hydra, without hydra, etc. You just need to set the proprietarySrc
accordingly.
could you be a bit more specific what you mean?
Have a look at NIX_PATH: https://nixos.org/nix/manual/#env-NIX_PATH

You can configure the NIX_PATH locally, e.g.

export NIX_PATH=proprietarySrc=$HOME/src/proprietary:$NIX_PATH

or explicitly via command-line arg to nix-build, nix-shell etc.

Then instead of writing your package as:

stdenv.mkDerivation rec {
src = fetchgitPrivate ...;
...
}

write

stdenv.mkDerivation rec {
src = <proprietarySrc>;
...
}

If used in the context of hydra, hydra configuration says, where and how
to obtain that software, then hydra downloads it and passes it further
for build. E.g. in
<https://hydra.nixos.org/jobset/nix/master#tabs-configuration>, there
are paths nix and nixpkgs, which are then written in nix as <nix> and
<nixpkgs> respectively. In this example, no fetchgitPrivate is used but
you can see those paths. In your case of proprietary package, you don't
have to do anything special except allowing hydra to download those
proprietary repos. We have a read-only access to our git repo with ssh
key for hydra.

If you go with the fetchgitPrivate option, you have to grant access in
too many places, i.e. in every context where your package can be
evaluated.
Alexander V. Nikolaev
2017-07-04 10:49:50 UTC
Permalink
On Mon, Jul 03, 2017 at 03:19:31PM +0200, Harmen wrote:

I have `fetchgitCustom` expression, which can use pre-seeded "deploy"
keys (but with some security implications -- because key is
world-readable). It works with sandbox builds, and should work with
hydra as well.

If anyone interesting in this solution, I'll prepare PR soon.
Post by Harmen
I'm struggling to get fetchgitPrivate to work on nix-daemon installation (no
NixOS, these are Ubuntu machines with nix).
I can make it work on my dev machine, with is a non-daemon install, by setting
NIX_PATH=ssh-config-file=/my/ssh/config:$NIX_PATH
But that doesn't work in sandboxed daemon mode, because the nixbld* users can't
read that file (both because of access rights, and because of the sandbox).
Note that the config file and any keys it points to must be readable
by the build user, which depending on your nix configuration means making it
readable by the build-users-group, the user of the running nix-daemon, or the
user calling the nix command which started the build. Similarly, if using an
ssh agent ssh-auth-sock must point to a socket the build user can access.
You may need StrictHostKeyChecking=no in the config file. Since ssh
will refuse to use a group-readable private key, if using build-users you will
likely want to use something like IdentityFile /some/directory/%u/key and have
a directory for each build user accessible to that user.
from
https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/fetchgit/private.nix
which sounds reasonable, but it I don't see how to do that since I don't know
exactly which user will run the build. Also because of the sandbox the build
can't read the ssh config file at all.
So next option is to generate the configfile in my expression, a-la
https://www.mpscholten.de/nixos/2016/07/07/private-github-repositories-and-nixos.html
but I don't know how to set nix.path from inside an expression. It would also
require bundling the key with the expression, but if that works...
I can't be the first to want to use fetchgitPrivate with a sandboxed
nix-daemon. Any experiences or tips?
Thanks!
Harmen
_______________________________________________
nix-dev mailing list
https://mailman.science.uu.nl/mailman/listinfo/nix-dev
Tomas Hlavaty
2017-07-04 16:46:07 UTC
Permalink
Post by Alexander V. Nikolaev
I have `fetchgitCustom` expression, which can use pre-seeded "deploy"
keys (but with some security implications -- because key is
world-readable). It works with sandbox builds, and should work with
hydra as well.
And does it work if the package is build by remote build slaves?
zimbatm
2017-07-04 20:10:09 UTC
Permalink
Yes, the source is part of the build input and is uploaded to the worker to
run the build.
Post by Tomas Hlavaty
Post by Alexander V. Nikolaev
I have `fetchgitCustom` expression, which can use pre-seeded "deploy"
keys (but with some security implications -- because key is
world-readable). It works with sandbox builds, and should work with
hydra as well.
And does it work if the package is build by remote build slaves?
_______________________________________________
nix-dev mailing list
https://mailman.science.uu.nl/mailman/listinfo/nix-dev
Harmen via nix-dev
2017-07-06 08:49:56 UTC
Permalink
On Tue, Jul 04, 2017 at 08:10:09PM +0000, zimbatm wrote:

Thanks for the suggestions.

I took away from this that it's best to not have Nix deal with the checkouts.
For now I'll keep things as separate repos, to keep things easier.

Eventually I would like to go to one pkgs tree for all repos, but I'll think
about how to do that best. It also ties into integration and system testing,
which is not just Nix related.

thanks!
Harmen

Loading...