Abysmal performance on basic commands involving nixpkgs

[eclairevoyant]

Commands like

nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz hello

or

nix profile install nixpkgs#hello

gets stuck for tens of minutes (!) after unzipping, sometimes never completing. SIGINT does nothing, have to force stop the app.

[expenses]

The evaluation speed (in a proot at least) is very slow. Once a specific version of nixpkgs has been evaluation cached then things get a lot faster. Running just nixpkgs#<whatever> is generally a bad idea as nixpkgs updates too quickly. Using a specific revision is faster as is just adding/removing packages from a flake.

[eclairevoyant]

taking hours to eval makes this basically unusable 🤷
and why would proot be related? eval happens in-memory.
and proot claims to "not cause overhead".

[expenses]

Running nix-on-droid switch takes about a minute for me once evaluation caching is done, which for me is very usable. Perhaps you could try and profile things to see where time is being spent?

[eclairevoyant]

once evaluation caching is done

I feel like this is missing the point - if eval takes over an hour with no guarantee of completion whenever the revision needs updating, I'd have to sit there waiting and hoping it ever completes. There's obviously a design flaw and I'd hope it can be addressed. Or at least, part of the social contract of using open-source software is reporting issues as they are encountered, if no one has done so prior.

[t184256]

OK, my stance on this.

1. IDK who claims "proot doesn't cause overhead", proot tanks IO real hard, and nixpkgs eval is very IO-heavy.
2. proot is the project's chosen way to run unprivileged without recompiling nixpkgs. I have neither the bandwidth to implement our ways of relocating /nix/store, nor the resources to recompile nixpkgs to a new location. Whoever has either of the two, reach out to me and we'll figure something out.
3. Hour-long eval is horrible UX and entirely unreasonable.
4. on a 5 year old Samsung S10e 6GB with LineageOS 19 and wakelock held, nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz hello (i.e., download + unpack + eval) takes less than 8 minutes for me.
5. rm -rf ~/.cache/nix/eval-cache-v5; nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz gnumake immediately after this evaluates in 3.2 seconds
6. time nix profile install nixpkgs#ncdu downloads/unpacks/evals in under 8 minutes.
7. That being said, if you claim that eval takes an hour for you, I'm sorry, but I can neither reproduce that nor advise what should you do about it, besides maybe suggesting you acquire a wakelock and/or follow guidance from https://dontkillmyapp.com

[eclairevoyant]

1. IDK who claims "proot doesn't cause overhead",

Apologies, I misread the comment here: https://wiki.termux.com/wiki/PRoot
What it actually said was chroot has no overhead, which makes more sense WRT the performance seen here.
If proot fundamentally tanks IO, then fair enough, strace indicates most of the time spent on newfsastat calls.

6. time nix profile install nixpkgs#ncdu downloads/unpacks/evals in under 8 minutes.

8 min is quite reasonable and closer to what I'd expect on a phone. If I can get it there I'd be happy with that.

7. if you claim that eval takes an hour for you, I'm sorry, but I can neither reproduce that nor advise what should you do about it, besides maybe suggesting you acquire a wakelock and/or follow guidance from https://dontkillmyapp.com

Battery saver is off, and "allow background usage" is on. Though, I'd be surprised if doze mode was involved for a foreground app while the screen is on?

Also, I think the link provided is outdated, as I don't see any option for "battery optimization" as mentioned in https://dontkillmyapp.com/google

FWIW I use a Pixel 7 with the current (June) release of android 14.

Other helpful info might be that, the initial bootstrap of nix-on-droid takes about 3 min. I'll run the time command on the fresh install for comparison.

[t184256]

Initial bootstrap definitely involves more than eval'ing a hello, so something is killing the performance afterwards. IDK whether an app can run out of Doze credits while being in foreground, but try (quoting from same device):

• App info - Battery usage - Unrestricted
• Notifications - Nix - Expand - Acquire wakelock

[Arian04]

I'm not super knowledgeable about Nix yet, but I'll try to provide useful and relevant information, let me know if I miss anything and I'll follow up with that info.

Device Info + App info

I'm using a Pixel 5 on LineageOS 21, rooted with KernelSU (probably irrelevant but thought I should mention just in case).

The app's battery restriction setting thing is set to Unrestricted and I always acquire a wakelock via the notification when using it. I'm running it with the screen on, app in the foreground, and charger plugged in. Of course that's not how I always use it, but for right now, since I've been configuring things while switching over from Termux, this is how I've been using it

I didn't time it perfectly, but seem to have encountered the same behavior as the person who created this issue. SIGINT did not work for me either, I assume since it was waiting on IO or something along those lines? However, I wanted to add that I don't experience the problem anymore now that I've copied over my dotfiles from my PC. Here is the (I think) relevant part of my config:

nixOnDroidConfigurations.default = nix-on-droid.lib.nixOnDroidConfiguration {
    modules = [
        {nix.registry.nixpkgs.flake = nixpkgs;}
        ...
    ];
};

I put ... to denote that I omitted the rest of the stuff in that block, because I don't think it matters, but if it does, let me know and I'll add it back. I use this on my PC because I use nixpkgs unstable and don't wanna have to redownload ~40MB every now and then if I run a quick nix shell nixpkgs#foo. As I said before, I'm not super knowledgeable about Nix yet, but I think this is what was mentioned in this comment:

Once a specific version of nixpkgs has been evaluation cached then things get a lot faster. Running just nixpkgs#<whatever> is generally a bad idea as nixpkgs updates too quickly. Using a specific revision is faster as is just adding/removing packages from a flake.

Basically this comment is a "I also experience this issue, here's the config that seemingly makes the issue disappear", and hopefully the info helps. Again, let me know if I should add any more details.

[t184256]

@Arian04 Could you time the commands from #374 (comment) and post more objective results?

[Arian04]

TL;DR: I timed those commands with my current config, then commented out the lines that I felt had "fixed" the issue, then switch'd into the new config, then re-timed those commands, and got perfectly reasonable results both times, so sorry about the noise. Either I was super impatient before and interpreted the times as like 5x longer than they were, or I'm just no longer able to reproduce the issue :(

Testing details

Testing with current config

nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz hello

• took 3:17, was just showing [41.7 MiB DL] for a little over 2 minutes of that time I believe

rm -rf ~/.cache/nix/eval-cache-v5; nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz gnumake

• running this immediately afterwards evaluates in a little under a second

time nix profile install nixpkgs#ncdu

• takes 4 seconds

Testing with modified config

• After running the previous test I ran nix-collect-garbage, then nix-on-droid switch --flake'd my config with {nix.registry.nixpkgs.flake = nixpkgs;} commented out. Then re-did those tests with the following results.
• also I removed a line setting NIX_PATH in my home-manager config that I think was doing a related thing maybe?
• I actually had to nix profile rollback twice because of weird errors while trying to switch, but yea.

nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz hello

• same as before

rm -rf ~/.cache/nix/eval-cache-v5; nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz gnumake

• same as before

time nix profile install nixpkgs#ncdu

• takes 3:29

[eclairevoyant]

Sorry, I missed responding to this earlier.

• App info - Battery usage - Unrestricted

EDIT: this is indeed set.

Original comment:
As mentioned, I still don't see this option in battery settings:

• Notifications - Nix - Expand - Acquire wakelock

I did try this and it says "wakelock held" in the notification, but the screen does still shut off after some time (1 min in my case, which is my display timeout) - is that expected?

Beyond this, I've run time nix build nixpkgs#hello; the vast majority of the nearly 15 min it appeared to be paused on [41.9 MiB DL] evaluating derivation 'flake:nixpkgs#hello' due to the background I/O. The following was the output:

time nix build nixpkgs#hello

real    13m45.525s
user    0m32.591s
sys     4m43.273s

Leaving the phone plugged in vs unplugged did not appear to significantly impact the time spent here, over multiple runs this was the best time I saw.

I'm not sure if this time can be improved, if this is fundamentally a proot limitation then I'll close this out.

[Arian04]

As mentioned, I still don't see this option in battery settings:

I noticed the option got moved around a bit in an Android update. you can find it if you click on the preference item itself, but not the toggle switch on the right. so basically click around where the actual "Allow background usage" text is, and it'll show you the submenu "underneath" it, if that makes sense

[eclairevoyant]

Ah, yes it's set to unrestricted already.

[bohanubis]

I have a question why can't NIX_PATH can't be generated