Skip to content

Fix parser comparison mismatches#4220

Open
ericsciple wants to merge 1 commit intomainfrom
users/ericsciple/26-02-compare
Open

Fix parser comparison mismatches#4220
ericsciple wants to merge 1 commit intomainfrom
users/ericsciple/26-02-compare

Conversation

@ericsciple
Copy link
Collaborator

@ericsciple ericsciple commented Feb 2, 2026

Fixes telemetry noise from parser comparison mismatches between legacy and new workflow parsers.

Fixes

  • ActionManifestMismatch: Load - Fix ConvertToLegacySteps() to use explicit property mapping instead of broken JSON round-trip
  • EvaluateJobContainer / EvaluateJobServiceContainers - Only emit error for empty container image during early validation, not at runtime
  • TemplateEvaluatorMismatch (fromJSON) - Treat JSON parse errors as matching even when error messages differ
  • Cancellation race conditions - Skip mismatch recording when cancellation state changes during the evaluation window

@ericsciple ericsciple changed the title Fix ConvertToLegacySteps to use explicit property mapping Fix parser comparison mismatches Feb 2, 2026
Func<TNew> newEvaluator,
Func<TLegacy, TNew, bool> resultComparer)
{
// Capture cancellation state before evaluation
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This helps eliminate noise due to a cancellation race condition — i.e. cancellation fires after legacy evaluation, before new evaluation

@ericsciple ericsciple force-pushed the users/ericsciple/26-02-compare branch from b2ef7f3 to 53c6839 Compare February 3, 2026 06:44
@ericsciple ericsciple marked this pull request as ready for review February 3, 2026 06:45
@ericsciple ericsciple requested a review from a team as a code owner February 3, 2026 06:45
Copilot AI review requested due to automatic review settings February 3, 2026 06:45
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request tightens the comparison behavior between legacy and new workflow/action parsers and addresses several sources of noisy or misleading telemetry. It adds targeted tests around parser comparison, fixes composite action step conversion, refines container image validation semantics, and makes exception comparison more tolerant of semantically equivalent JSON parse errors while handling cancellation races.

Changes:

  • Add ActionManifestParserComparisonL0 tests to validate ActionManifestManagerWrapper step conversion, job container evaluation, JSON parse error equivalence, and cancellation-aware mismatch recording.
  • Update WorkflowTemplateConverter.ConvertToJobContainer to only emit an error for empty container images during early validation, not during runtime evaluation, and clean up minor formatting issues.
  • Enhance PipelineTemplateEvaluatorWrapper.EvaluateAndCompare to avoid recording mismatches when cancellation is requested mid-evaluation and to treat known JSON parse error patterns as equivalent, plus expose internals of Runner.Worker to the Test assembly and implement an explicit step conversion path in ActionManifestManagerWrapper.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/Test/L0/Worker/ActionManifestParserComparisonL0.cs New L0 tests covering composite step conversion, empty container handling, JSON parse error equivalence, and cancellation-aware mismatch behavior between legacy and new parsers.
src/Sdk/WorkflowParser/Conversion/WorkflowTemplateConverter.cs Adjust container image validation to only log errors during early validation, maintain consistent formatting, and keep job/service container conversion behavior aligned with expectations.
src/Runner.Worker/PipelineTemplateEvaluatorWrapper.cs Wrap legacy/new evaluations with cancellation-state tracking, gate mismatch recording on absence of cancellation races, and add JSON-parse-aware exception comparison helpers.
src/Runner.Worker/InternalsVisibleTo.cs Allow the Test assembly to access internal members of Runner.Worker to support the new comparison tests.
src/Runner.Worker/ActionManifestManagerWrapper.cs Replace composite step JSON round-tripping with explicit property mapping for RunStep/ActionStep, add helpers to reconstruct conditions and inputs, parse uses references into legacy reference types, and slightly improve comparison logging.

Fixes telemetry noise from parser comparison mismatches between legacy and new
workflow parsers.

## Fixes

- **ActionManifestMismatch: Load** - Fix `ConvertToLegacySteps()` to use explicit property mapping instead of broken JSON round-trip
- **EvaluateJobContainer / EvaluateJobServiceContainers** - Only emit error for empty container image during early validation, not at runtime
- **TemplateEvaluatorMismatch (fromJSON)** - Treat JSON parse errors as matching even when error messages differ
- **Cancellation race conditions** - Skip mismatch recording when cancellation state changes during the evaluation window
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant