Skip to content

Comments

NMake (Windows): robust CUDA lib path resolution for Conda & system installs#705

Merged
neworderofjamie merged 2 commits intogenn-team:nmakefrom
Agrim-P777:nmake/modular_cuda_path_fix
Aug 27, 2025
Merged

NMake (Windows): robust CUDA lib path resolution for Conda & system installs#705
neworderofjamie merged 2 commits intogenn-team:nmakefrom
Agrim-P777:nmake/modular_cuda_path_fix

Conversation

@Agrim-P777
Copy link
Contributor

Summary

On Windows NMake builds, the generated link step hardcoded /LIBPATH:"$(CUDA_PATH)\lib\x64" and/or referenced $(CudaLibraryPath). In practice:

  • Conda modular CUDA stores libs in %CONDA_PREFIX%\Library\lib
  • NMake uppercases imported env vars, so CudaLibraryPathCUDALIBRARYPATH
  • In a normal runtime shell, %LIB% may not include %CONDA_PREFIX%\Library\lib

This PR makes the NMake preamble define a $(LIBCUDA) variable with sensible fallbacks, and uses it in the link rule:

Priority:

  1. CUDALIBRARYPATH (Conda/runtime)
  2. $(CUDA_PATH)\lib\x64 (system CUDA)
  3. $(CUDA_PATH)\lib (alternate layout)
  4. empty (let MSVC’s LIB search resolve)

Changes

  • src/genn/backends/cuda/backend.cc
    • Update genNMakefilePreamble to emit $(LIBCUDA) with the above fallback logic.
    • Update genNMakefileLinkRule to use $(LIBCUDA) and link dynamically to CUDA (no cudart_static.lib dependence).
  • Mirrored the same LIBCUDA logic for HIP backend NMake preamble/link rule in src/genn/backends/hip/backend.cc.

Why

  • Works out-of-the-box with Conda modular CUDA and with system CUDA.
  • Avoids fragile mixed-case env var references (CudaLibraryPath) that NMake won’t populate.
  • Avoids requiring cuda-cudart-static at runtime; dynamic linking via cudart.lib is sufficient (and consistent with MSBuild configs in the repo).

Backwards compatibility

  • System CUDA installs continue to work (fallback to $(CUDA_PATH)\lib\x64 / \lib).
  • Environments where the toolchain sets LIB will still link (empty $(LIBCUDA)).
  • No behavior change on Linux/macOS.

@neworderofjamie
Copy link
Contributor

I didn't realise how weirdly case-sensitive NMake was when I originally implemented the CudaLibraryPath functionality so, at the expense of one extra line of MSBuild, I have updated the nmake branch to look at CUDA_LIBRARY_PATH instead. Hopefully this will tidy up a couple of things. Also, when does the $(CUDA_PATH)\lib alternate layout occur?

@Agrim-P777
Copy link
Contributor Author

Totally agree it’s cleaner to standardize on CUDA_LIBRARY_PATH, (since NMake imports env vars upper-cased and is picky about macro names), but it’s not fully “sufficient” on Windows by itself. In the wild:

  • Conda modular CUDA typically won’t set CUDA_LIBRARY_PATH in a user shell.
  • Some CI scripts already export CUDALIBRARYPATH (no underscore) for MSBuild/NMake convenience.
  • The most robust NMake preamble is to try both env vars (in all-caps, as NMake will have them), then fall back to system CUDA:
// Prefer explicit CUDALIBRARYPATH (Conda/runtime) then CUDA_LIBRARY_PATH, then manual CUDA installs, else nothing (fall back to LIB)
!IF DEFINED(CUDALIBRARYPATH)            // uses defined and doesn't rely on empty string comparisons
LIBCUDA=/LIBPATH:"$(CUDALIBRARYPATH)"
!ELSEIF DEFINED(CUDA_LIBRARY_PATH)
LIBCUDA=/LIBPATH:"$(CUDA_LIBRARY_PATH)"
!ELSEIF EXIST("$(CUDA_PATH)\lib\x64\cudart.lib")
LIBCUDA=/LIBPATH:"$(CUDA_PATH)\lib\x64"
!ELSEIF EXIST("$(CUDA_PATH)\lib\cudart.lib")
LIBCUDA=/LIBPATH:"$(CUDA_PATH)\lib"

// Nothing – rely on LIB if it’s set by the environment/toolchain
!ELSE
LIBCUDA=
!ENDIF

If we standardize our docs and CI to always set CUDA_LIBRARY_PATH, only that’s sufficient and clean. If we want to remain compatible with existing Conda packaging (which often used CUDALIBRARYPATH for NMake), we should check both. Happy to simplify to just CUDA_LIBRARY_PATH if you prefer it that way

  1. A bare \lib (without \x64) is uncommon today but can appear if:
    • Someone uses a re-packaged or minimal SDK that flattens libs under lib (seen in a few enterprise/internal images).
    • A custom toolchain bundle deposits import libs directly into lib (no x64 subdir).
    • Very old/legacy or nonstandard installs.
  • In short, \lib\x64 is the standard; \lib is a defensive last-ditch fallback. keeping it can save odd setups.

That will make NMake builds work out-of-the-box across Conda modular CUDA, system CUDA, and quirky environments—without relying on fragile mixed-case names.

If you’re happy with the dual-variable check, I can update the PR to prefer CUDALIBRARYPATH, then CUDA_LIBRARY_PATH, then the CUDA_PATH fallbacks, if you prefer we can drop the final \lib fallback; it just helps with nonstandard/legacy layouts.

@neworderofjamie
Copy link
Contributor

Some CI scripts already export CUDALIBRARYPATH (no underscore) for MSBuild/NMake convenience

What scripts do this? I'm pretty surprised that anyone cares about this arcane combination apart from us 🤣 If it's even semi-standard then I'll just change everything to use CUDALIBRARYPATH instead of CUDA_LIBRARY_PATH and save one layer of complexity.

@Agrim-P777
Copy link
Contributor Author

Ah sorry, I should’ve been clearer — I didn’t mean there’s some wider “semi-standard.” I was referring to our Conda packaging CI scripts, where we exported CUDALIBRARYPATH specifically so NMake would pick it up (since it uppercases env vars). That’s what I had in mind when I mentioned “some scripts.” (all the draft scripts of our conda recipe)

So yeah, it’s totally internal/our setup, not anything broader in the wild. If you’d prefer to standardize on CUDALIBRARYPATH in the repo and I just adapt our Conda recipes/CI to set that instead, that works fine too 👍

@neworderofjamie
Copy link
Contributor

Ahh ok, that makes more sense. In that case, can you rebase onto my nmake branch and update all our scripts to use CUDA_LIBRARY_PATH as that's a more normal naming convention for environment variables.

…llback to CUDA_PATH\lib\x64 for manual installs, otherwise let linker use LIB if present
@Agrim-P777 Agrim-P777 force-pushed the nmake/modular_cuda_path_fix branch from 6957b44 to d3996b5 Compare August 26, 2025 19:00
@neworderofjamie neworderofjamie merged commit 412f2c8 into genn-team:nmake Aug 27, 2025
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants