Parallel builds broken

[qknight]

problem

• nix build has such a minor cpu utilization, i only see a load of 25% at max
• cargo build basically goes to 100%

it seems in this video it was doing much more parallel builds: https://asciinema.org/a/742433

observation (parallelism)

nix build --file target/debug/nix/cargo_build_caller.nix deps.adler2-2_0_0-115180b36279fc7c deps.anstyle-1_0_10-bf6d032cb7d79be1 deps.allocator-api2-0_2_21-bd3713078dfee01f -L

shows that these 3 are build in parallel!

experiment

created passthru-test to experiment and it shows parallel builds work there!

findings

in the generated nix build system i use passthru in this line:

${fn.rustc_propagated_arguments passthru.rust_crate_libraries}

and if i remove it, it builds in parallel!

so what is the cause? this is the implementation:

1. rustc_propagated_arguments = rust_crate_libraries: lib.replaceStrings ["\n"] [""] (builtins.concatStringsSep " " (map (crate:
2.   if builtins.pathExists "${crate}/nix/rustc-propagated-arguments" then
3.    builtins.readFile "${crate}/nix/rustc-propagated-arguments"
4.  else
5.    ""
6.  ) (allCollectedInputs rust_crate_libraries)));

caused by line 2+3, so it blocks because of the pathExists/readFile since that requires the file be there during nix evaluation time and not during build time. (not sure if the terms were used correctly).

solution ideas

idea1

• replace builtins.pathExists/builtins.readFile and built the rustc command from bash. i tried this for a while but ran in into the problem with empty spaces ' ':

        proc-macro2> Compiling proc-macro2-1_0_97-script_build-b0e673af67401d13
        proc-macro2> @cargo { "type":0, "crate_name":"proc-macro2", "crate_type":"(build.rs build)", "id":"proc-macro2-1_0_97-script_build-b0e673af67401d13" }
        proc-macro2> +++ /nix/store/h1c2imj0dpfyyfrd0i6195xznqxcar8x-rust-stable-2025-08-07/bin/rustc --crate-name build_script_build --edition=2021 build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no --cfg 'feature="default"' --cfg 'feature="proc-macro"' --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values("default", "nightly", "proc-macro", "span-locations"))' -C metadata=800301207300bbac -C extra-filename=-b0e673af67401d13 --out-dir /nix/store/rdjqd65zqgxkydyb4nl217423539nqn2-proc-macro2-1_0_97-script_build-b0e673af67401d13 -L dependency=/nix/store/46wmzlzs30vcq83wbm1q516cpd0p0zqx-rustc-linker-arguments-dir/deps collect_rustc_propagated_arguments ' ' --cap-lints allow
        proc-macro2> +++ rustc_exit_value=1
        proc-macro2> +++ set +x -e
        proc-macro2> error: multiple input filenames provided (first two filenames are `build.rs` and `collect_rustc_propagated_arguments`)
        proc-macro2> 

doing this with bash is hard, i had this before and it forced me to do the code in the way it is now.

idea 2

maybe using nushell instead of bash could be a thing.

let binary = 'my_binary'
let args = ['arg1', 'arg2', '--flag']

# Example: variable with 20 paths
let paths = [
    "/path/a", "/path/b", "/path/c", "/path/d", "/path/e",
    "/path/f", "/path/g", "/path/h", "/path/i", "/path/j",
    "/path/k", "/path/l", "/path/m", "/path/n", "/path/o",
    "/path/p", "/path/q", "/path/r", "/path/s", "/path/t"
]

# Read "mycommand" file from each path, concatenate their contents with a space
let extra_arg = ($paths | each { |p| open $"($p)/mycommand" | str trim } | str join ' ')

# Extend args with the concatenated extra argument
let args = $args ++ [$extra_arg]

let command = [$binary] ++ $args

do $command