Switch to uv build frontend and add top-level uv workspace#1615
Switch to uv build frontend and add top-level uv workspace#1615kkraus14 wants to merge 18 commits intoNVIDIA:mainfrom
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
0e6820a to
3505cf1
Compare
|
/ok to test |
- Add root pyproject.toml as virtual uv workspace covering cuda_bindings, cuda_core, cuda_pathfinder, cuda_python, and cuda_python_test_helpers. - Declare workspace-level cu12/cu13 conflicts so uv lock can resolve the mutually-exclusive cuda-toolkit version requirements. - Add [tool.cibuildwheel] build-frontend = "build[uv]" to cuda_bindings and cuda_core. - Add [tool.uv.sources] workspace references to cuda_bindings and cuda_core so transitive workspace deps resolve during lock. - Add cuda-pathfinder to cuda_core build-system.requires so uv sees the transitive build dependency chain. - Make cuda_python dependencies static in pyproject.toml. - Replace pip with uv across all CI workflows (build-wheel, test-wheel-linux, test-wheel-windows, build-docs, coverage) and ci/tools/run-tests. - Install uv via astral-sh/setup-uv@v5 in every workflow. - Use uv build --wheel, uv pip install, and uv sync --only-group for dependency group installation. - Exclude uv.lock from SPDX and large-file pre-commit checks. - Generate uv.lock for the workspace. Co-authored-by: Cursor <cursoragent@cursor.com>
- Revert cuda_python dependencies and optional-dependencies back to dynamic so setup.py controls the precise version pin at wheel-build time. - Pin astral-sh/setup-uv to commit SHA d4b2f3b6 (v5) in all workflows. - Remove twine install and twine check steps (uv build validates wheel metadata). - Regenerate uv.lock. Co-authored-by: Cursor <cursoragent@cursor.com>
- Replace duplicated test deps in build-wheel.yml with uv sync --package <pkg> --only-group test --no-install-project. - Improve conflict comments to clarify they are about cuda-toolkit version constraints, not cuda-bindings compatibility. Co-authored-by: Cursor <cursoragent@cursor.com>
uv build does not validate long_description rendering or metadata completeness the way twine check --strict does. Restore the twine install and all four twine check steps (pathfinder, bindings, cuda-python, core). Co-authored-by: Cursor <cursoragent@cursor.com>
3e1070a to
36cf441
Compare
|
/ok to test |
CI workflows run without a virtual environment, so uv pip requires --system to install into the system Python. Also rebase on latest main to resolve merge conflicts (.gitattributes, .pre-commit-config). Co-authored-by: Cursor <cursoragent@cursor.com>
36cf441 to
9c9757f
Compare
|
/ok to test |
1 similar comment
|
/ok to test |
uv build from a workspace member directory outputs to the workspace root's dist/, not the member's dist/. Add --output-dir dist to both cuda_pathfinder and cuda_python uv build calls so wheels land where subsequent steps expect them. Co-authored-by: Cursor <cursoragent@cursor.com>
b5aaea0 to
a736c3d
Compare
|
/ok to test |
uv sync always creates/uses a virtual environment and doesn't support --system. Since CI installs everything into the system Python, use uv export to extract dependency group requirements and pipe them to uv pip install --system instead. coverage.yml is left using uv sync since it intentionally creates and targets a .venv. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
|
Without --frozen, uv export tries to update the lockfile which triggers setuptools-scm metadata extraction. This fails in CI because the git checkout doesn't have the context setuptools-scm needs. --frozen tells uv to use the committed lockfile as-is. Also switch coverage.yml from uv sync to the same uv export pattern for consistency. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
| git_describe_command = ["git", "describe", "--dirty", "--tags", "--long", "--match", "cuda-core-v*[0-9]*"] | ||
|
|
||
| [tool.uv] | ||
| conflicts = [[{ extra = "cu12" }, { extra = "cu13" }]] |
The build[uv] frontend inside cibuildwheel creates build isolation environments with paths derived from the uv cache directory. On Windows, the cumulative PATH length pushes past cmd.exe's ~8KB limit when running vcvarsall.bat, specifically for free-threaded Python (py3.14t) which has additional path entries. Set UV_CACHE_DIR=C:\uvc in CIBW_ENVIRONMENT_WINDOWS for all three cibuildwheel steps to keep temp paths short. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
| # Version from git tags; setup.py still overrides install_requires at wheel build | ||
| # for exact version pinning (~= for release, == for dev). | ||
| [tool.setuptools_scm] | ||
| root = ".." | ||
| version_file = "cuda_python/_version.py" | ||
| tag_regex = "^(?P<version>v\\d+\\.\\d+\\.\\d+)" | ||
| git_describe_command = ["git", "describe", "--dirty", "--tags", "--long", "--match", "v*[0-9]*"] |
| # Workspace-level conflicts: cuda-toolkit can only be one major version at a | ||
| # time, so every cu12 extra/group is incompatible with every cu13 one. | ||
| conflicts = [ |
There was a problem hiding this comment.
Q: It seems we have to list all possible combinations of conflicts? Is this the only way at the workspace level? Maybe uv could honor the conflicts in each individual pyproject.toml? We currently only add a conflict for cuda-core.
There was a problem hiding this comment.
This is on my list to get cursor to see if it can simplify 😄. I agree this is quite annoying.
There was a problem hiding this comment.
@leofang see the updated comment. Unfortunately uv doesn't automatically propagate the information upwards so we need to keep this here in addition to defining it at the package level.
I'll open an issue upstream and link it here.
cibuildwheel creates build isolation venvs under TEMP, and on Windows the default TEMP path (C:\Users\runneradmin\AppData\Local\Temp\...) combined with cibuildwheel's own subdirectories produces paths long enough to exceed cmd.exe's ~8KB environment limit when setuptools invokes vcvarsall.bat. This only manifests on py3.14t (free-threaded) which has additional path entries. Fix by shortening TEMP/TMP to C:\t on Windows before cibuildwheel runs, keeping build-frontend = "build[uv]" consistently on all platforms. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
…licts - Comment uv export | uv pip install pattern in build-wheel.yml, test-wheel-linux.yml, and test-wheel-windows.yml explaining what the command does (installs a dependency group from pyproject.toml). - Add comment to root pyproject.toml explaining why all conflict combinations must be listed at the workspace root (uv limitation). Co-authored-by: Cursor <cursoragent@cursor.com>
|
There are build frontends? |
Create a venv (uv venv) early in each workflow so uv auto-detects it. This enables: - uv sync --frozen --only-group for dependency groups (replaces the uv export | uv pip install -r - workaround) - uv tool run twine for one-shot tool invocations (no install needed) - Dropping --system from all uv pip install calls build-docs.yml keeps --system since it installs into a conda env. coverage.yml already had a venv and now uses uv sync too. Co-authored-by: Cursor <cursoragent@cursor.com>
- Upgrade astral-sh/setup-uv from v5 to v7.3.0 (SHA eac588ad) in all workflows. - Use activate-environment: true to auto-create and activate a venv, removing manual uv venv steps. - Replace twine install + twine check with uv tool run twine check (no install needed). - Remove --python .venv/bin/python from coverage.yml since the venv is now auto-activated. - Keep build-docs.yml on --system since it uses a conda env. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
build-wheel.yml uses setup-python twice: 3.12 for cibuildwheel and the matrix version for Cython test compilation. Cannot use setup-uv activate-environment since the matrix Python is not available when setup-uv runs. Instead, create a fresh venv manually after the second setup-python step so the venv uses the matrix Python. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
uv run executes commands in the venv context, ensuring tools like cythonize are on PATH. This avoids needing to manually add the venv bin directory to GITHUB_PATH. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
uv run --no-project runs the command in the existing venv without trying to sync the workspace project. This lets cythonize and other venv-installed tools be found on PATH without needing to manually activate the venv or modify GITHUB_PATH. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
Cython test build scripts need cythonize on PATH. Prepend both .venv/bin (Linux) and .venv/Scripts (Windows) to PATH within the run block so the venv-installed tools are found. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
uv sync manages the full venv state and removes packages not in the lockfile, including pre-built wheels installed via uv pip install. Swap the order: run uv sync first (installs test deps), then uv pip install the pre-built wheels. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
Summary
pyproject.tomlas a virtual uv workspace coveringcuda_bindings,cuda_core,cuda_pathfinder,cuda_python, andcuda_python_test_helpers.build[uv]backend forcuda_bindingsandcuda_core.pipusage withuvacross CI workflows (build-wheel,test-wheel-linux,test-wheel-windows,build-docs,coverage) andci/tools/run-tests.astral-sh/setup-uv@v5in every workflow.uv build --wheel,uv pip install, anduv sync --only-groupfor dependency group installation.uv lockcan resolve the mutually-exclusivecuda-toolkitversion requirements.uv.lockfor the workspace.Test plan
uv lockreproduces the same lockfile from a clean checkout with CUDA_HOME setMade with Cursor