Add code coverage support for Windows and linux platform#1584
Add code coverage support for Windows and linux platform#1584rluo8 wants to merge 10 commits intoNVIDIA:mainfrom
Conversation
|
Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
Hi @mdboom , |
|
In order to test this before we merge it, I
The job is running here: https://github.com/NVIDIA/cuda-python/actions/runs/21754529128 |
mdboom
left a comment
mdboom
left a comment
There was a problem hiding this comment.
This is really great. A lot of hard problems solved here.
I just had the one question about whether we need to do a full clone of the repo. Other than resolving that question, LGTM.
| - name: Checkout ${{ github.event.repository.name }} | ||
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||
| with: | ||
| fetch-depth: 0 |
There was a problem hiding this comment.
Do we need to fetch the whole history?
There was a problem hiding this comment.
Yes, since cuda_bindings and cuda_core both use setuptools_scm to get their version from git history, fetch-depth: 0 is required. Without full history, setuptools_scm may compute an incorrect version, which causes cuda.core.init.py's import-time check to fail with ImportError: cuda.bindings 12.x or 13.x must be installed.
Error like this without fetch-depth:0:
- cd /__w/cuda-python/cuda-python/.venv/lib/python3.14/site-packages
- /__w/cuda-python/cuda-python/.venv/bin/pytest -v --cov=./cuda --cov-append --cov-context=test --cov-config=/__w/cuda-python/cuda-python/.coveragerc /__w/cuda-python/cuda-python/cuda_core/tests
ImportError while loading conftest '/__w/cuda-python/cuda-python/cuda_core/tests/conftest.py'.
../../../../cuda_core/tests/conftest.py:7: in
import helpers
../../../../cuda_core/tests/helpers/init.py:10: in
from cuda.core._utils.cuda_utils import handle_return
cuda/core/init.py:14: in
raise ImportError("cuda.bindings 12.x or 13.x must be installed")
E ImportError: cuda.bindings 12.x or 13.x must be installed
Error: Process completed with exit code 4.
|
/ok to test |
|
| with: | ||
| name: coverage | ||
| name: coverage-combined | ||
| path: docs/coverage/ |
There was a problem hiding this comment.
Q: Does each PR (and that built from main) overwrite with each other?
There was a problem hiding this comment.
Yes, they would overwrite with each other and only keep the latest report in docs/coverage.
The coverage workflow needs to be manually triggered for PR. And for main branch, it was scheduled to be nightly triggered. Current github page report was from this PR because the nightly main coverage run failed and couldn't generate a report.
Historical coverage data is available as artifacts uploaded by each coverage run, with a retention period of 7 days (configurable via retention-days in the workflow).
Do we need to keep historical coverage reports in github pages?
| # Patch cuda_core pyproject.toml - add after cuda.core._cpp line | ||
| sed -i '/\"cuda\.core\._cpp\" = \[\".*\"\]/a \"*\" = [\"*.cpp\"]' cuda_core/pyproject.toml | ||
|
|
||
| echo "Applied coverage patches to cuda_core" |
There was a problem hiding this comment.
Rather than doing these patches I think it would be better to add build commands into the projects that need it and pass the options through via pip install --config-settings or using an environment variable.
There was a problem hiding this comment.
Yes, using an environment variable would be better and more reliable. There is already a CUDA_PYTHON_COVERAGE variable in the project which enables Cython linetrace. However, due to a recent change in wheel packaging, extra changes are needed in the project setup to properly support the coverage build. @mdboom , would it be all right to make changes to the project setup files for coverage support?
There was a problem hiding this comment.
Yes, it's fine to update them for what we need. As Keith suggested, if there is a way to just pass in configuration options rather than patching it, that would be preferable.
|
Also, should we consider using something like https://codecov.io instead of manually managing all of the coverage reports? |
Codecov could be a good option. It's a third-party service that might need to be enabled for the project first if using it. |
4 participants
This is to add code coverage support for Windows platform.
Here are the main steps: