Skip to content
This repository was archived by the owner on Sep 1, 2025. It is now read-only.
This repository was archived by the owner on Sep 1, 2025. It is now read-only.

Implement chunks builtin for makeDerivationParallel #995

@dsalaza4

Description

@dsalaza4

makeDerivationParallel is great for local parallelization in just one machine.

Many times, the next step for improving performance is distributing workloads into several machines.

A chunks builtin would be great for this:

{
  chunks,
  inputs,
  makeDerivation,
  makeDerivationParallel,
}: let
numbers = [0 1 2 3 4 5 6 7 8 9];
myChunks =  chunks numbers 3; # [[0 1 2 3] [4 5 6] [7 8 9]]

buildNumber = n: makeDerivation {
  name = "build-number-${n}";
  env.envNumber = n;
  builder = ''
    echo "$envNumber"
    touch "$out"
  '';
};
in
  makeDerivationParallel {
    dependencies = builtins.map buildNumber (inputs.nixpkgs.lib.lists.elemAt myChunks 0);
    name = "build-numbers-0";
  }

With this logic one would be able to easily distribute workloads into several CI jobs, achieving distributed parallelization.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions