Add Cobalt SVE microbenchmarks run to runtime-perf-jobs#5127
Add Cobalt SVE microbenchmarks run to runtime-perf-jobs#5127DrewScoggins wants to merge 1 commit intodotnet:mainfrom
Conversation
Add a new coreclr micro run targeting the Azure Linux 3 Cobalt ARM64 queue with runCategories set to SVE instead of the default Runtime and Libraries categories. Also adds the arm64 build job dependency. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
LoopedBard3
left a comment
There was a problem hiding this comment.
Looks like this error was hit:
[2026/02/27 00:02:41][INFO] $ dotnet run --project /home/helixbot/work/AF7E09A3/w/AB5F09BE/e/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net11.0 --no-restore --no-build -- --anyCategories SVE --logBuildOutput --generateBinLog --corerun /home/helixbot/work/AF7E09A3/p/Core_Root/corerun --partition-count 15 --partition-index 3 --artifacts /home/helixbot/work/AF7E09A3/w/AB5F09BE/uploads/BenchmarkDotNet.Artifacts --packages /home/helixbot/work/AF7E09A3/w/AB5F09BE/e/performance/artifacts/packages --buildTimeout 1200
[2026/02/27 00:02:44][INFO] // Validating benchmarks:
[2026/02/27 00:02:44][INFO] // * SveBenchmarks.AddReduction.Vector128AddReduction does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO]
[2026/02/27 00:02:44][INFO] // * SveBenchmarks.Partition.SveTail does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO]
[2026/02/27 00:02:44][INFO] // * SveBenchmarks.Partition.Scalar does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO]
[2026/02/27 00:02:44][INFO] // * SveBenchmarks.Partition.SvePartition does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO]
[2026/02/27 00:02:44][INFO] // * SveBenchmarks.SquareRoot.Vector128SquareRoot does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO]
[2026/02/27 00:02:44][INFO] // * SveBenchmarks.StrCmp.Vector128StrCmp does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO]
[2026/02/27 00:02:44][INFO] // * SveBenchmarks.StrCmp.SveTail does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO]
[2026/02/27 00:02:44][INFO] // * SveBenchmarks.StrCmp.Scalar does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO]
[2026/02/27 00:02:44][INFO] $ popd
There was a problem hiding this comment.
Pull request overview
Adds a new CoreCLR microbenchmarks run targeting the Azure Linux 3 Cobalt ARM64 queue, intended to run only the SVE-focused benchmarks rather than the default Runtime/Libraries set. This extends the runtime perf pipeline to cover Cobalt+SVE and adds the needed ARM64 build job dependency.
Changes:
- Adds an ARM64 CoreCLR build job template dependency to the runtime perf pipeline.
- Adds a new CoreCLR
linux_arm64microbenchmarks job targetingperfcobaltwithrunCategoriesset for SVE.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| runCategories: 'SVE' | ||
| additionalJobIdentifier: 'SVE' |
There was a problem hiding this comment.
runCategories is set to 'SVE', but the microbenchmarks category defined in src/benchmarks/micro/Categories.cs is Categories.Sve with the value "Sve" (mixed case). Since runCategories is passed through to BenchmarkDotNet as --anyCategories (string match), using 'SVE' will likely select zero benchmarks. Update runCategories (and ideally additionalJobIdentifier for consistency) to use 'Sve' to match the actual category string.
| runCategories: 'SVE' | |
| additionalJobIdentifier: 'SVE' | |
| runCategories: 'Sve' | |
| additionalJobIdentifier: 'Sve' |
1dda1b7 to
c3d8c4b
Compare
LoopedBard3
left a comment
There was a problem hiding this comment.
Noticed another approach already pushed but not tested so undid my quick fix. Do we want to add Categories Sve to the mandatory category list or do we want to add Categories Runtime to the Sve tests? Just another option.
Add a new coreclr micro run targeting the Azure Linux 3 Cobalt ARM64 queue with runCategories set to SVE instead of the default Runtime and Libraries categories. Also adds the arm64 build job dependency.