NixOS exporter host does not find `usb` module

[AiyionPrime]

@Emantor I'm currently trying to install your nix package for python 3.13.

After the backport of a fix yesterday I can install it, but have trouble bringing a usb relay board to work.

The exporter runs NixOS and has this minimal? configuration:

leinestelle-gruppe:
  location: leinestelle
  HIDRelay:
    index: 1
    invert: false
    match:
      ID_PATH: 'pci-0000:00:14.0-usb-0:2'

I invoke it from the shell:

labgrid-exporter configuration.yaml

In another terminal (on the same nix machine) I have a coordinator running:

labgrid-coordinator

and in the third I invoke the client.
When I ask it to show power devices, it find the usb module lacking.

labgrid-client -c remote.yaml --place leinestelle acquire
labgrid-client -c remote.yaml power get

which produces a stacktrace:

Selected role main and place leinestelle from configuration file
Traceback (most recent call last):
  File "/nix/store/mw3m9qrwd8d1sniswd95yqpqima3329x-python3.13-labgrid-25.0.1/lib/python3.13/site-packages/labgrid/remote/client.py", line 2156, in main
    args.func(session)
    ~~~~~~~~~^^^^^^^^^
  File "/nix/store/mw3m9qrwd8d1sniswd95yqpqima3329x-python3.13-labgrid-25.0.1/lib/python3.13/site-packages/labgrid/remote/client.py", line 876, in power
    drv = target.get_driver("PowerProtocol", name=name)
  File "/nix/store/mw3m9qrwd8d1sniswd95yqpqima3329x-python3.13-labgrid-25.0.1/lib/python3.13/site-packages/labgrid/target.py", line 235, in get_driver
    return self._get_driver(cls, name=name, resource=resource, activate=activate)
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/mw3m9qrwd8d1sniswd95yqpqima3329x-python3.13-labgrid-25.0.1/lib/python3.13/site-packages/labgrid/target.py", line 209, in _get_driver
    self.activate(found[0])
    ~~~~~~~~~~~~~^^^^^^^^^^
  File "/nix/store/mw3m9qrwd8d1sniswd95yqpqima3329x-python3.13-labgrid-25.0.1/lib/python3.13/site-packages/labgrid/target.py", line 468, in activate
    self.activate(supplier)
    ~~~~~~~~~~~~~^^^^^^^^^^
  File "/nix/store/mw3m9qrwd8d1sniswd95yqpqima3329x-python3.13-labgrid-25.0.1/lib/python3.13/site-packages/labgrid/target.py", line 472, in activate
    client.on_activate()
    ~~~~~~~~~~~~~~~~~~^^
  File "/nix/store/mw3m9qrwd8d1sniswd95yqpqima3329x-python3.13-labgrid-25.0.1/lib/python3.13/site-packages/labgrid/driver/usbhidrelay.py", line 28, in on_activate
    self.proxy = self.wrapper.load('usb_hid_relay')
                 ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/nix/store/mw3m9qrwd8d1sniswd95yqpqima3329x-python3.13-labgrid-25.0.1/lib/python3.13/site-packages/labgrid/util/agentwrapper.py", line 126, in load
    self.call('load', name, source)
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/mw3m9qrwd8d1sniswd95yqpqima3329x-python3.13-labgrid-25.0.1/lib/python3.13/site-packages/labgrid/util/agentwrapper.py", line 106, in call
    raise AgentException(e)
labgrid.util.agentwrapper.AgentException: ModuleNotFoundError("No module named 'usb'")

I'm unsure why this fails but expect the load at runtime pattern to have clashed with nixos isolation.
Whatever the python env of the invocation is, it apparently has no access to the usb module, despite it being installed by the nix package.
Do you have a suggestion for me?

[Bastian-Krause]

I'm not familiar with NixOS, but the agent code copied to the exporter (which is the same system in your case) is not executed within a virtualenv. That means the python-usb dependency you're stumbling upon needs to be installed in the system-wide Python installation (managed by the distro)..

[Bastian-Krause]

See also #1782.

[AiyionPrime]

You're not at the Congress, are you? 😁

[AiyionPrime]

I'm not familiar with NixOS, but the agent code copied to the exporter (which is the same system in your case) is not executed within a virtualenv. That means the python-usb dependency you're stumbling upon needs to be installed in the system-wide Python installation (managed by the distro)..

This info was quite important, thanks for it.

Where does the design decision to scp the agent file onto the exporter host stem from?
Wouldn't it be more straight forward to ship the file as labgrid-agent binary (possibly managed by the system pkg manager) and just invoke it via ssh instead?

[Emantor]

Where does the design decision to scp the agent file onto the exporter host stem from? Wouldn't it be more straight forward to ship the file as labgrid-agent binary (possibly managed by the system pkg manager) and just invoke it via ssh instead?

I think this is an artifact from development which carried over into production since it lets the developer diverge versions between exporter and client. So the client can have a newer version of labgrid with a more up to date agent interface which can be modified locally instead of having to possibly go through the IT hoops to get the exporter machines updated.

Thinking about this from a nix perspective: Can you check whether you can inject you own virtual environment for the exporter via something similar to:

systemd.services.labgrid-exporter = let 
  python-with-my-packages = pkgs.python3.withPackages (p: with p; [
    pyusb
    # other python packages you want
  ]);
in {
[…]
  path = [ python-with-my-packages ];
[…]

Edit: This does not work, but adding a similar python environment to system packages should work.

[AiyionPrime]

While manual additions to system package do work, they are considered an antipattern in nixos, as they taint other environments as well.

Would something like #1791 be considerable?