[CI Failure Doctor] CI Job 'logs-token-check' Disabled with `if: false` in PR #13756

[github-actions]

Investigation Summary

Workflow Run Details

• Workflow Run ID: 21682685660
• Branch: main
• Commit: a2dfbe9 (Add AWF --skip-pull flag and pre-download agent/squid images Add AWF --skip-pull flag and pre-download agent/squid images #13756)
• Status: Failed (overall workflow conclusion)
• Run Time: 2026-02-04 18:04:24 to 18:08:09 UTC (~4 minutes)

Job Analysis

Job Status Summary:

• Total Jobs: 53 (but only 30 completed)
• Completed: 30 jobs
• Successful: 29 jobs ✅
• Skipped: 1 job ⏭️ (logs-token-check)
• Failed: 0 jobs

Root Cause: logs-token-check Job Disabled

Location: .github/workflows/ci.yml, lines 1352-1450

Finding: The logs-token-check job is intentionally disabled using if: false:

logs-token-check:
  if: false  # ← Job is explicitly disabled
  runs-on: ubuntu-latest
  permissions:
    contents: read
    actions: read
  # ... rest of job configuration ...

Job Purpose: The disabled job is a comprehensive validation job that:

1. Builds the gh-aw binary (make build)
2. Runs the gh aw logs command on the smoke-copilot workflow
3. Validates the JSON output structure, checking for:
   • summary.total_tokens field (token count > 0)
   • runs array (with expected schema)
   • agent field (engine identification)
   • workflow_path field (workflow location)
   • workflow_name field (workflow identifier)

Impact Assessment

Downstream Effects:

1. Missing Test Coverage: The logs-token-check job provides validation for the gh aw logs command's JSON output format
2. Token Parsing Not Verified: No automated validation that token counts are correctly parsed and included in logs output
3. Workflow Metadata Not Validated: No checks that workflow names, paths, and agent information are properly resolved in log JSON
4. Silent Regression Risk: Changes to the gh aw logs command output format won't be caught by CI without enabling this job

Severity: MEDIUM

• Current workflow still completes successfully (29/30 completed jobs pass)
• No immediate failure, but important validation is skipped
• Future regressions in logs command output structure won't be caught

Historical Context

PR #13756 Analysis:

• This is the first and only commit to .github/workflows/ci.yml (new file)
• The job was added with if: false from the start (not a regression)
• The commit includes significant changes to AWF (Agentic Workflows Framework)
• The commit also made 396 additions and 357 deletions across the codebase

Possible Reasons for Disabling:

1. Incomplete Implementation: The job may require features not yet implemented (e.g., gh aw logs command)
2. External Dependencies: May depend on GitHub Actions logs that aren't available during testing
3. Intentional Deferral: Disabled to get PR Add AWF --skip-pull flag and pre-download agent/squid images #13756 merged, with plans to enable later
4. Missing Environment Setup: May require special configuration (e.g., GitHub token, specific artifacts)

Recommended Actions

IMMEDIATE:

1. ✅ Document the Skip Reason: Add a comment explaining why if: false is set

   logs-token-check:
     # TODO: Enable after gh aw logs command is fully implemented
     # or after GitHub Actions log artifacts are available in CI
     if: false

2. 🔍 Verify Job Readiness: Check if the job can actually run by temporarily enabling it:

   • Does gh aw logs smoke-copilot work in CI?
   • Are the smoke-copilot workflow artifacts available?
   • Can the JSON validation succeed?

SHORT-TERM:
3. 📋 Create Tracking Issue: Link this to PR #13756 review comments or create a follow-up issue
4. 🧪 Conditional Enable: Consider conditional logic instead of if: false:

if: github.event.pull_request.draft == false  # Only on non-draft PRs
# OR
if: contains(github.event.pull_request.labels.*.name, 'test-logs-command')

LONG-TERM:
5. 📅 Re-enable With Plan: Create a specific tracking issue to re-enable this job with:

• Clear acceptance criteria
• Dependencies that must be resolved
• Timeline for enabling

Code Quality Notes

The job is well-structured with comprehensive validations:

• ✅ Proper error handling (set -e flag)
• ✅ Clear diagnostic output with emoji-based status indicators
• ✅ Defensive JSON parsing with jq error checking
• ✅ Multi-level validation (structure + content + field resolution)
• ⚠️ Could benefit from a comments explaining why smoke-copilot was chosen as test subject

Related Files

• PR: Add AWF --skip-pull flag and pre-download agent/squid images #13756 (Add AWF --skip-pull flag and pre-download agent/squid images)
• Commit: a2dfbe9
• Workflow File: .github/workflows/ci.yml (lines 1352-1450)

---

Status: 🔴 Action Required - Job disabled without documented reason; needs follow-up plan

AI generated by CI Failure Doctor

To add this workflow in your repository, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. See usage guide.

• expires on Feb 5, 2026, 6:15 PM UTC