Skip to content

Add Cobalt SVE microbenchmarks run to runtime-perf-jobs#5127

Open
DrewScoggins wants to merge 1 commit intodotnet:mainfrom
DrewScoggins:AddSveRuns
Open

Add Cobalt SVE microbenchmarks run to runtime-perf-jobs#5127
DrewScoggins wants to merge 1 commit intodotnet:mainfrom
DrewScoggins:AddSveRuns

Conversation

@DrewScoggins
Copy link
Member

Add a new coreclr micro run targeting the Azure Linux 3 Cobalt ARM64 queue with runCategories set to SVE instead of the default Runtime and Libraries categories. Also adds the arm64 build job dependency.

Add a new coreclr micro run targeting the Azure Linux 3 Cobalt ARM64
queue with runCategories set to SVE instead of the default Runtime and
Libraries categories. Also adds the arm64 build job dependency.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@DrewScoggins
Copy link
Member Author

Copy link
Member

@LoopedBard3 LoopedBard3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@LoopedBard3 LoopedBard3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this error was hit:

[2026/02/27 00:02:41][INFO] $ dotnet run --project /home/helixbot/work/AF7E09A3/w/AB5F09BE/e/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net11.0 --no-restore --no-build -- --anyCategories SVE --logBuildOutput --generateBinLog --corerun /home/helixbot/work/AF7E09A3/p/Core_Root/corerun --partition-count 15 --partition-index 3 --artifacts /home/helixbot/work/AF7E09A3/w/AB5F09BE/uploads/BenchmarkDotNet.Artifacts --packages /home/helixbot/work/AF7E09A3/w/AB5F09BE/e/performance/artifacts/packages --buildTimeout 1200
[2026/02/27 00:02:44][INFO] // Validating benchmarks:
[2026/02/27 00:02:44][INFO] //    * SveBenchmarks.AddReduction.Vector128AddReduction does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO] 
[2026/02/27 00:02:44][INFO] //    * SveBenchmarks.Partition.SveTail does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO] 
[2026/02/27 00:02:44][INFO] //    * SveBenchmarks.Partition.Scalar does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO] 
[2026/02/27 00:02:44][INFO] //    * SveBenchmarks.Partition.SvePartition does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO] 
[2026/02/27 00:02:44][INFO] //    * SveBenchmarks.SquareRoot.Vector128SquareRoot does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO] 
[2026/02/27 00:02:44][INFO] //    * SveBenchmarks.StrCmp.Vector128StrCmp does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO] 
[2026/02/27 00:02:44][INFO] //    * SveBenchmarks.StrCmp.SveTail does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO] 
[2026/02/27 00:02:44][INFO] //    * SveBenchmarks.StrCmp.Scalar does not belong to one of the mandatory categories: Runtime, ThirdParty, Libraries. Use [BenchmarkCategory(Categories.$)]
[2026/02/27 00:02:44][INFO] 
[2026/02/27 00:02:44][INFO] $ popd

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new CoreCLR microbenchmarks run targeting the Azure Linux 3 Cobalt ARM64 queue, intended to run only the SVE-focused benchmarks rather than the default Runtime/Libraries set. This extends the runtime perf pipeline to cover Cobalt+SVE and adds the needed ARM64 build job dependency.

Changes:

  • Adds an ARM64 CoreCLR build job template dependency to the runtime perf pipeline.
  • Adds a new CoreCLR linux_arm64 microbenchmarks job targeting perfcobalt with runCategories set for SVE.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +283 to +284
runCategories: 'SVE'
additionalJobIdentifier: 'SVE'
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runCategories is set to 'SVE', but the microbenchmarks category defined in src/benchmarks/micro/Categories.cs is Categories.Sve with the value "Sve" (mixed case). Since runCategories is passed through to BenchmarkDotNet as --anyCategories (string match), using 'SVE' will likely select zero benchmarks. Update runCategories (and ideally additionalJobIdentifier for consistency) to use 'Sve' to match the actual category string.

Suggested change
runCategories: 'SVE'
additionalJobIdentifier: 'SVE'
runCategories: 'Sve'
additionalJobIdentifier: 'Sve'

Copilot uses AI. Check for mistakes.
LoopedBard3
LoopedBard3 previously approved these changes Feb 27, 2026
Copy link
Member

@LoopedBard3 LoopedBard3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noticed another approach already pushed but not tested so undid my quick fix. Do we want to add Categories Sve to the mandatory category list or do we want to add Categories Runtime to the Sve tests? Just another option.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants