feat: hierarchical dispatch model for meta orchestrator

.tc-output.txt

@@ -0,0 +1,5 @@
+npm warn Unknown env config "devdir". This will stop working in the next major version of npm.
+src/types/workflow.ts(33,59): error TS2305: Module '"../schema/workflow.schema.js"' has no exported member 'AgentRole'.
+src/types/workflow.ts(33,70): error TS2305: Module '"../schema/workflow.schema.js"' has no exported member 'ExecutionModel'.
+src/types/workflow.ts(34,48): error TS2305: Module '"../schema/workflow.schema.js"' has no exported member 'AgentRoleSchema'.
+src/types/workflow.ts(34,65): error TS2305: Module '"../schema/workflow.schema.js"' has no exported member 'ExecutionModelSchema'.

README.md

@@ -9,7 +9,7 @@ A [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) server for AI
 
 ---
 
-**[Quick Start](#-quick-start)** • **[Schema Guide](schemas/README.md)** • **[API Reference](docs/api-reference.md)** • **[Workflow Fidelity](docs/workflow-fidelity.md)** • **[Development](docs/development.md)** • **[Workflows](https://github.com/m2ux/workflow-server/tree/workflows)** • **[Engineering](https://github.com/m2ux/workflow-server/tree/engineering)**
+**[Quick Start](#-quick-start)** • **[Architecture Overview](docs/architecture.md)** • **[Schema Guide](schemas/README.md)** • **[API Reference](docs/api-reference.md)** • **[Workflow Fidelity](docs/workflow-fidelity.md)** • **[Development](docs/development.md)** • **[Workflows](https://github.com/m2ux/workflow-server/tree/workflows)** • **[Engineering](https://github.com/m2ux/workflow-server/tree/engineering)**
 
 ---
 

docs/api-reference.md

@@ -6,62 +6,66 @@
 
 No session token required.
 
-| Tool | Parameters | Description |
-|------|------------|-------------|
-| `discover` | - | Entry point. Returns server name, version, available workflows, and the bootstrap procedure |
-| `list_workflows` | - | List all available workflow definitions with full metadata |
-| `health_check` | - | Server health, version, workflow count, and uptime |
+| Tool | Parameters | Returns | Description |
+|------|------------|---------|-------------|
+| `discover` | - | Server info and `discovery` instructions | Entry point. Returns server name, version, and the bootstrap procedure. Use list_workflows to list workflows. |
+| `list_workflows` | - | Array of workflow definitions | List all available workflow definitions with full metadata |
+| `health_check` | - | Server status and stats | Server health, version, workflow count, and uptime |
 
 ### Session Tools
 
-| Tool | Parameters | Description |
-|------|------------|-------------|
-| `start_session` | `workflow_id`, `session_token?`, `agent_id?` | Start a new session or inherit an existing one. For fresh sessions, provide only `workflow_id`. For worker dispatch or resume, also provide `session_token` — the returned token inherits all state (`sid`, `act`, `pcp`, `pcpt`, `cond`, `v`) with `agent_id` stamped into the signed `aid` field. `workflow_id` is validated against the token's workflow. |
+| Tool | Parameters | Returns | Description |
+|------|------------|---------|-------------|
+| `start_session` | `workflow_id`, `agent_id`, `session_token?` | `session_token`, workflow info, and `inherited` flag (if resuming) | Start a new session or inherit an existing one. For fresh sessions, provide `workflow_id` and `agent_id`. For worker dispatch or resume, also provide `session_token` — the returned token inherits all state (`sid`, `act`, `pcp`, `pcpt`, `cond`, `v`) with `agent_id` stamped into the signed `aid` field. `workflow_id` is validated against the token's workflow. |
+| `dispatch_workflow` | `workflow_id`, `parent_session_token`, `variables?` | `client_session_token`, `client_session_id`, `parent_session_id`, workflow metadata, `initial_activity`, and a pre-composed `client_prompt` (to be passed to the client/sub-agent) | Create a client session for a target workflow and return a dispatch package for a sub-agent. The client session is independent and does NOT inherit state from the parent session. |
+| `get_workflow_status` | `client_session_token?`, `client_session_id?`, `parent_session_token?` | `status` (active/blocked/completed), `current_activity`, `completed_activities`, `last_checkpoint` info, and variables | Check the status of a dispatched client workflow session. Requires either `client_session_token`, or `client_session_id` + `parent_session_token` for authorization. |
 
 ### Workflow Tools
 
 All require `session_token`. The workflow is determined from the session token (set at `start_session`). Each response includes an updated token and validation result in `_meta`.
 
-| Tool | Parameters | Description |
-|------|------------|-------------|
-| `get_workflow` | `session_token`, `summary?` | Load the workflow definition for the current session. `summary=true` (default) returns rules, variables, execution model, `initialActivity`, and activity stubs. `summary=false` returns the full definition |
-| `next_activity` | `session_token`, `activity_id`, `transition_condition?`, `step_manifest?`, `activity_manifest?` | Load and transition to an activity. Returns the complete activity definition (steps, checkpoints, transitions, mode overrides, rules, skill references). Also returns a trace token in `_meta.trace_token`. **Embeds required checkpoint IDs in the token — hard-rejects transition to a different activity until all are resolved via `respond_checkpoint`** |
-| `get_checkpoint` | `session_token`, `activity_id`, `checkpoint_id` | Load full checkpoint details (message, options with effects, blocking/auto-advance config) for presentation |
-| `respond_checkpoint` | `session_token`, `checkpoint_id`, `option_id?`, `auto_advance?`, `condition_not_met?` | Resolve a pending checkpoint. Exactly one of: `option_id` (user's selection), `auto_advance` (use `defaultOption` after `autoAdvanceMs` elapses, non-blocking only), or `condition_not_met` (dismiss conditional checkpoint). Returns effects and updated token with the checkpoint removed from `pcp` |
+| Tool | Parameters | Returns | Description |
+|------|------------|---------|-------------|
+| `get_workflow` | `session_token`, `summary?` | Complete workflow definition or summary metadata | Load the workflow definition for the current session. `summary=true` (default) returns rules, variables, execution model, `initialActivity`, and activity stubs. `summary=false` returns the full definition |
+| `next_activity` | `session_token`, `activity_id`, `transition_condition?`, `step_manifest?`, `activity_manifest?` | Complete activity definition and trace token in `_meta` | Load and transition to an activity. Also advances the session token to track the current activity. |
+| `yield_checkpoint` | `session_token`, `checkpoint_id` | Status, `checkpoint_handle`, and instructions | Yield execution to the orchestrator at a checkpoint step. Returns a `checkpoint_handle` that the worker must yield to the orchestrator via a `<checkpoint_yield>` block. |
+| `resume_checkpoint` | `session_token` | Status and instructions | Resume execution after the orchestrator resolves a checkpoint. Validates the checkpoint was resolved and advances the token sequence. |
+| `present_checkpoint` | `checkpoint_handle` | Full checkpoint definition | Used by the orchestrator. Load full checkpoint details (message, options with effects, blocking/auto-advance config) from a worker's yielded `checkpoint_handle` for presentation to the user. |
+| `respond_checkpoint` | `checkpoint_handle`, `option_id?`, `auto_advance?`, `condition_not_met?` | Resolution status and any defined `effect` | Used by the orchestrator. Resolve a yielded checkpoint. Exactly one of: `option_id` (user's selection), `auto_advance` (use `defaultOption` after `autoAdvanceMs` elapses, non-blocking only), or `condition_not_met` (dismiss conditional checkpoint). Unblocks the worker's token. |
 
 ### Skill Tools
 
 All require `session_token`. The workflow is determined from the session token.
 
-| Tool | Parameters | Description |
-|------|------------|-------------|
-| `get_skills` | `session_token` | Load all workflow-level skills (behavioral protocols). Returns a map of skill objects with `_resources` containing lightweight references (index, id, version — no content) |
-| `get_skill` | `session_token`, `step_id` | Load the skill for a specific step within the current activity. Requires `next_activity` to have been called first |
-| `get_resource` | `session_token`, `resource_index` | Load a resource's full content by index. Bare indices resolve within the session workflow; prefixed refs (e.g., `meta/04`) resolve from the named workflow |
+| Tool | Parameters | Returns | Description |
+|------|------------|---------|-------------|
+| `get_skills` | `session_token` | Map of skill objects with lightweight `_resources` references | Load all workflow-level skills (behavioral protocols). |
+| `get_skill` | `session_token`, `step_id?` | Skill definition object | Load the skill for a specific step within the current activity. If `step_id` is omitted, loads the primary skill for the activity. Requires `next_activity` to have been called first |
+| `get_resource` | `session_token`, `resource_index` | Resource content, id, and version | Load a resource's full content by index. Bare indices resolve within the session workflow; prefixed refs (e.g., `meta/04`) resolve from the named workflow |
 
 ### Trace Tools
 
-| Tool | Parameters | Description |
-|------|------------|-------------|
-| `get_trace` | `session_token`, `trace_tokens?` | Resolve accumulated trace tokens into full event data. Without tokens, returns the in-memory trace for the current session |
+| Tool | Parameters | Returns | Description |
+|------|------------|---------|-------------|
+| `get_trace` | `session_token`, `trace_tokens?` | Trace source, event count, and array of events | Resolve accumulated trace tokens into full event data. Without tokens, returns the in-memory trace for the current session |
 
 
 ## Session Token
 
 The session token is an opaque string returned by `start_session`. It captures the context of each call (workflow, activity, skill) so the server can validate subsequent calls for consistency.
 
-The token payload carries: `wf` (workflow ID), `act` (current activity), `skill` (last loaded skill), `cond` (last transition condition), `v` (workflow version), `seq` (sequence counter), `ts` (creation timestamp), `sid` (session UUID), `aid` (agent ID — set via `start_session(agent_id)`), `pcp` (pending checkpoint IDs), and `pcpt` (checkpoint issuance timestamp). When `start_session` is called with an existing `session_token`, all fields are inherited (including `sid`, `pcp`, `act`) and `aid` is stamped with the new agent identity.
+The token payload carries: `wf` (workflow ID), `act` (current activity), `skill` (last loaded skill), `cond` (last transition condition), `v` (workflow version), `seq` (sequence counter), `ts` (creation timestamp), `sid` (session UUID), `aid` (agent ID — set via `start_session`'s `agent_id` parameter), and `bcp` (active blocking checkpoint ID, if any). When `start_session` is called with an existing `session_token`, all fields are inherited (including `sid`, `act`) and `aid` is stamped with the new agent identity.
 
 ### Lifecycle
 
 1. Call `discover` to learn the bootstrap procedure and available workflows
 2. Call `list_workflows` to match the user's goal to a workflow
-3. Call `start_session(workflow_id)` to get a session token (workflow is bound to the session from this point)
+3. Call `start_session(workflow_id, agent_id)` to get a session token (workflow is bound to the session from this point)
 4. Call `get_skills` to load behavioral protocols
 5. Call `get_workflow(summary=true)` to get the activity list and `initialActivity`
 6. Call `next_activity(initialActivity)` to load the first activity
-7. For each step with a skill, call `get_skill(step_id)` then `get_resource` for each `_resources` entry
-8. Call `respond_checkpoint` for each required checkpoint before transitioning
+7. For each step with a `skill` property, call `get_skill(step_id)` then `get_resource` for each `_resources` entry. Do NOT call `get_skill` for steps without a skill.
+8. When encountering a checkpoint step, call `yield_checkpoint`, yield to the orchestrator, and wait to be resumed via `resume_checkpoint`.
 9. Read `transitions` from the activity response; call `next_activity` with a `step_manifest` to advance
 10. Accumulate `_meta.trace_token` from each `next_activity` call for post-execution trace resolution
 
@@ -93,12 +97,12 @@ Warnings do not block execution — the tool still returns its result. They enab
 
 ### Checkpoint Enforcement
 
-When `next_activity` loads an activity with required checkpoints, those checkpoint IDs are embedded in the token's `pcp` field. **Calling `next_activity` for a different activity while `pcp` is non-empty produces a hard error** (not a warning).
+When a worker encounters a checkpoint step during activity execution, it calls `yield_checkpoint`. This sets the `bcp` (blocking checkpoint) field in the token and returns a `checkpoint_handle`. **Calling `next_activity` while `bcp` is set produces a hard error** (not a warning).
 
-To clear the gate, call `respond_checkpoint` for each pending checkpoint:
+The worker yields the `checkpoint_handle` to the orchestrator. To clear the gate, the orchestrator calls `respond_checkpoint` using the handle:
 
 ```json
-{ "session_token": "...", "checkpoint_id": "confirm-implementation", "option_id": "proceed" }
+{ "checkpoint_handle": "...", "option_id": "proceed" }
 ```
 
 Three resolution modes:
@@ -107,7 +111,7 @@ Three resolution modes:
 - **`auto_advance: true`** — use the checkpoint's `defaultOption`. Only valid for non-blocking checkpoints (`blocking: false`). The server enforces the full `autoAdvanceMs` timer.
 - **`condition_not_met: true`** — dismiss a conditional checkpoint whose condition evaluated to false. Only valid when the checkpoint has a `condition` field.
 
-The response includes any effects from the selected option (`setVariable`, `transitionTo`, `skipActivities`), the remaining pending checkpoints, and an updated token.
+The response includes any effects from the selected option (`setVariable`, `transitionTo`, `skipActivities`). The orchestrator relays these updates back to the worker, which then calls `resume_checkpoint` to proceed.
 
 ### Step Completion Manifest
 
@@ -162,14 +166,13 @@ When calling `get_skill { step_id }`:
 #### session-protocol (universal)
 
 Session lifecycle protocol:
-- **Bootstrap**: `start_session(workflow_id)` → `get_skills` → `get_workflow` → `next_activity(initialActivity)`
+- **Bootstrap**: `start_session(workflow_id, agent_id)` → `get_skills` → `get_workflow` → `next_activity(initialActivity)`
 - **Per-step**: `get_skill(step_id)` → `get_resource(resource_index)` for each `_resources` entry
 - **Transitions**: Read `transitions` from activity response → `next_activity(activity_id)` with `step_manifest`
 
-#### execute-activity (universal)
+#### 11-activity-worker (universal)
 
 Activity execution protocol for workers:
-- **Goal resolution**: `discover` → `list_workflows` → match user goal
 - **Bootstrap**: `start_session` → `get_skills` → `next_activity`
 - **Execution**: Steps → checkpoints (yield to orchestrator) → artifacts → structured result
 

docs/architecture.md

@@ -0,0 +1,23 @@
+# System Architecture Overview
+
+The Workflow Server is designed to orchestrate complex software engineering tasks using an autonomous, multi-agent framework. To handle ambiguity, long-running processes, and safe state transitions, the system's architecture is strictly modularized across several key paradigms.
+
+Below is the collection of architectural models that govern how the server and its agents interact.
+
+## 1. [Hierarchical Dispatch Model](dispatch_model.md)
+To prevent prompt degradation and ensure safe boundaries, the server uses a multi-layered agent model. The Meta Orchestrator (Level 0) handles high-level user interaction and workflow dispatch. The Workflow Orchestrator (Level 1) manages the state machine for a specific workflow. The Activity Worker (Level 2) executes the actual domain tasks (coding, reviewing, etc.). This document details how these agents are spawned, how they share session state, and how they resume each other.
+
+## 2. [Just-In-Time (JIT) Checkpoint Model](checkpoint_model.md)
+The JIT Checkpoint Model handles execution pauses. When a worker agent needs human input (like confirming a target directory or approving a PR), it cannot ask the user directly. Instead, it yields a `checkpoint_handle` up the chain to the Meta Orchestrator. This document explains how the handle bubbles up, how the server resolves it, and how the resulting variable updates are passed back down the chain via conversational resumes to unblock the worker.
+
+## 3. [State Management & Deterministic Transitions](state_management_model.md)
+The orchestration engine strictly enforces deterministic state transitions. Rather than asking an LLM to guess what to do next based on conversation history, the Workflow Orchestrator evaluates a rigid set of JSON variables against predefined condition strings in the workflow definition. This document covers how variables are initialized, how checkpoint effects and worker outputs mutate them, and how that state is persisted to `workflow-state.json`.
+
+## 4. [Artifact & Workspace Isolation](artifact_management_model.md)
+The system enforces strict boundaries between orchestration metadata (the workflow engine, plans, and state) and the user's domain codebase. This document outlines the purpose of the `.engineering` directory, the mandatory updates to the planning folder's `README.md`, the `artifactPrefix` naming conventions, and the specific Git submodule procedures agents must follow when committing orchestration artifacts.
+
+## 5. [Skill & Resource Resolution Architecture](resource_resolution_model.md)
+To preserve the precious context windows of Large Language Models, the system uses a lazy-loading resource architecture. Instead of injecting massive prompts, the server utilizes Canonical IDs and a Universal Skill Fallback mechanism. This document explains how `.toon` skill files declare lightweight `_resources` arrays, prompting agents to call the `get_resource` tool only when they need to read large, specialized markdown guides (like Git or Atlassian API tutorials).
+
+## 6. [Workflow Fidelity](workflow-fidelity.md)
+Because agents are autonomous, they must be audited to ensure they actually followed the instructions defined in the workflow's TOON files. This document details the Step Completion Manifest (a structured summary of what the agent did) and the cryptographic Trace Tokens. It explains how mechanical and semantic traces are recorded, validated against the workflow schema, and appended to the permanent audit log.