Added low-level workflow protocol documentation#5024
Added low-level workflow protocol documentation#5024WhitWaldo wants to merge 15 commits intodapr:v1.17from
Conversation
…ved via LLM (and heavily edited) from the dapr/dapr repository (favoring code implementation over available protobuf resources). Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
|
I think the contributing folder for this type of content makes sense. 👍 |
marcduiker
left a comment
There was a problem hiding this comment.
I think this is valuable to have, and I encourage other Dapr maintainers to provide more docs similar to this. I can imagine the overall process goes from Proposal -> Protocol (Docs)-> Development -> Release + Docs.
daprdocs/content/en/contributing/protocol-reference/workflow-protocol/_index.md
Show resolved
Hide resolved
|
|
||
| ### Deterministic Orchestrators | ||
| Orchestrator code must be side-effect free except via engine-mediated effects. Control flow must be reproducible | ||
| during replay (e.g., no unguarded access to wall-clock APIs). |
There was a problem hiding this comment.
Could be a language thing, but I've never heard of wall-clock APIs 😬 . Can this described differently?
There was a problem hiding this comment.
I think that's an autocorrect fail. I'll review and update.
|
@JoshVanL can you give this a review? |
|
|
||
| There are two protocol surfaces: | ||
| 1. Management API (standard Dapr gRPC): | ||
| - Start, terminate, pause, resume, and query workflow instances. |
There was a problem hiding this comment.
Pull request overview
This pull request adds comprehensive low-level protocol documentation for the Dapr Workflow building block. The documentation is aimed at SDK contributors, maintainers, and runtime developers who need to understand the internal mechanics of how Dapr's workflow capability operates at the protocol level, independent of SDK implementations.
Changes:
- Adds a new "Protocol Reference" section under contributing documentation
- Creates detailed workflow protocol documentation covering management APIs, execution APIs, orchestration lifecycle, activity lifecycle, state management, and versioning
- Includes 8 new documentation files with technical implementation details
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
daprdocs/content/en/contributing/protocol-reference/_index.md |
Creates the top-level protocol reference section index |
daprdocs/content/en/contributing/protocol-reference/workflow-protocol/_index.md |
Provides workflow protocol overview, architecture, and documentation map |
daprdocs/content/en/contributing/protocol-reference/workflow-protocol/workflow-protocol-management-api.md |
Documents the management API for controlling workflow instances |
daprdocs/content/en/contributing/protocol-reference/workflow-protocol/workflow-protocol-execution-api.md |
Details the Task Hub Protocol used by SDK workers |
daprdocs/content/en/contributing/protocol-reference/workflow-protocol/workflow-protocol-orchestration-lifecycle.md |
Explains replay-based orchestration execution and lifecycle |
daprdocs/content/en/contributing/protocol-reference/workflow-protocol/workflow-protocol-activity-lifecycle.md |
Describes activity execution flow and related concepts |
daprdocs/content/en/contributing/protocol-reference/workflow-protocol/workflow-protocol-state-and-history.md |
Documents state storage schema and history management |
daprdocs/content/en/contributing/protocol-reference/workflow-protocol/workflow-protocol-versioning.md |
Explains workflow versioning support |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - ** Orchestration ** | ||
|
|
||
| The deterministic coordinator that defines the workflow. The engine drives orchestrations via history replay to | ||
| rebuild state and schedule outbound tasks (activities, sub-orchestrations, timers, external events). | ||
|
|
||
| - ** Activity ** | ||
|
|
||
| The atomic unit of work. Activities are executed **at-least-once** and report results or failures back to the engine. | ||
| Idempotency is recommended. | ||
|
|
||
| - ** State Store & Backend ** |
There was a problem hiding this comment.
The markdown formatting for this list item is inconsistent with the others. It should use the same bullet marker (-) and indentation as the other items in the list.
| - ** Orchestration ** | |
| The deterministic coordinator that defines the workflow. The engine drives orchestrations via history replay to | |
| rebuild state and schedule outbound tasks (activities, sub-orchestrations, timers, external events). | |
| - ** Activity ** | |
| The atomic unit of work. Activities are executed **at-least-once** and report results or failures back to the engine. | |
| Idempotency is recommended. | |
| - ** State Store & Backend ** | |
| - **Orchestration** | |
| The deterministic coordinator that defines the workflow. The engine drives orchestrations via history replay to | |
| rebuild state and schedule outbound tasks (activities, sub-orchestrations, timers, external events). | |
| - **Activity** | |
| The atomic unit of work. Activities are executed **at-least-once** and report results or failures back to the engine. | |
| Idempotency is recommended. | |
| - **State Store & Backend** |
| The atomic unit of work. Activities are executed **at-least-once** and report results or failures back to the engine. | ||
| Idempotency is recommended. | ||
|
|
||
| - ** State Store & Backend ** |
There was a problem hiding this comment.
The markdown formatting for this list item is inconsistent with the others. It should use the same bullet marker (-) and indentation as the other items in the list.
| - ** State Store & Backend ** | |
| - **State Store & Backend** |
| - The engine safeguards idempotence and exactly-once commit of orchestration state. | ||
| 6. Completion | ||
| - Orchestration returns an output (success) or a failure (exception details). | ||
| - Final state and output are persisted; status queries refelct the terminal state. |
There was a problem hiding this comment.
There's a spelling error in "refelct" which should be "reflect".
| - Final state and output are persisted; status queries refelct the terminal state. | |
| - Final state and output are persisted; status queries reflect the terminal state. |
| @@ -0,0 +1,23 @@ | |||
| --- | |||
There was a problem hiding this comment.
The file starts with a UTF-8 BOM (Byte Order Mark) character "" which should be removed. BOM characters can cause issues with some parsers and are not necessary for UTF-8 encoded files.
| --- | |
| --- |
| @@ -0,0 +1,137 @@ | |||
| --- | |||
There was a problem hiding this comment.
The file starts with a UTF-8 BOM (Byte Order Mark) character "" which should be removed. BOM characters can cause issues with some parsers and are not necessary for UTF-8 encoded files.
| --- | |
| --- |
| - ** Orchestration ** | ||
|
|
||
| The deterministic coordinator that defines the workflow. The engine drives orchestrations via history replay to | ||
| rebuild state and schedule outbound tasks (activities, sub-orchestrations, timers, external events). | ||
|
|
||
| - ** Activity ** | ||
|
|
||
| The atomic unit of work. Activities are executed **at-least-once** and report results or failures back to the engine. | ||
| Idempotency is recommended. | ||
|
|
||
| - ** State Store & Backend ** |
There was a problem hiding this comment.
The markdown formatting for this list item is inconsistent with the others. It should use the same bullet marker (-) and indentation as the other items in the list.
| - ** Orchestration ** | |
| The deterministic coordinator that defines the workflow. The engine drives orchestrations via history replay to | |
| rebuild state and schedule outbound tasks (activities, sub-orchestrations, timers, external events). | |
| - ** Activity ** | |
| The atomic unit of work. Activities are executed **at-least-once** and report results or failures back to the engine. | |
| Idempotency is recommended. | |
| - ** State Store & Backend ** | |
| - **Orchestration** | |
| The deterministic coordinator that defines the workflow. The engine drives orchestrations via history replay to | |
| rebuild state and schedule outbound tasks (activities, sub-orchestrations, timers, external events). | |
| - **Activity** | |
| The atomic unit of work. Activities are executed **at-least-once** and report results or failures back to the engine. | |
| Idempotency is recommended. | |
| - **State Store & Backend** |
| @@ -0,0 +1,109 @@ | |||
| --- | |||
There was a problem hiding this comment.
The file starts with a UTF-8 BOM (Byte Order Mark) character "" which should be removed. BOM characters can cause issues with some parsers and are not necessary for UTF-8 encoded files.
| --- | |
| --- |
| @@ -0,0 +1,118 @@ | |||
| --- | |||
There was a problem hiding this comment.
The file starts with a UTF-8 BOM (Byte Order Mark) character "" which should be removed. BOM characters can cause issues with some parsers and are not necessary for UTF-8 encoded files.
| --- | |
| --- |
| @@ -0,0 +1,46 @@ | |||
| --- | |||
There was a problem hiding this comment.
The file starts with a UTF-8 BOM (Byte Order Mark) character "" which should be removed. BOM characters can cause issues with some parsers and are not necessary for UTF-8 encoded files.
| --- | |
| --- |
| - Start, terminate, pause, resume, and query workflow instances. | ||
| 2. Execution API (Task Hub Protocol): | ||
| - Worker facing, used to receive orchestration/activity work items and to report completion (e.g., via | ||
| `TaskHubSidecarService`). |
There was a problem hiding this comment.
The indentation is inconsistent here. Line 21 should use 4 spaces to align with line 20, matching the indentation of the bullet point above (line 18).
| `TaskHubSidecarService`). | |
| `TaskHubSidecarService`). |
Well, I thought about that, but it's reference to the low-level mechanism of how a core piece of Dapr works. Contributing is often more the community or rules-based processes around which someone can jump in and help out, but as this is technical reference documentation, hiding it in "protocol reference" just made more sense. |
Thank you for helping make the Dapr documentation better!
Please follow this checklist before submitting:
In addition, please fill out the following to help reviewers understand this pull request:
Description
Added low-level protocol implementation details about how the Workflow capability works from the runtime perspective independent of any SDK implementations. This will facilitate SDK contributors and maintainers and runtime maintainers to get up to speed of how Dapr's unique implementation works without having to independently parse the source code in the dapr/dapr runtime themselves.
This PR was largely generated by LLM atop the dapr/dapr repository with an emphasis on how the runtime logic works. While the generation was to be influenced by the shape of the protobuf definitions, it was not to rely on them because there are so many unused methods and properties in the existing durable task prototypes that Dapr doesn't use itself, further complicating developer understanding of what Dapr actually does here. I've also heavily edited it to correct hallucinations based on my own understanding and experience of and with the protocol.
I put this in the Contribution section instead of alongside the rest of the workflow documentation because:
This should be lightly reviewed by the @dapr/maintainers-dapr team for accuracy before merging.
It does reference versioning functionality coming with 1.17, so it is not compatible with the 1.16 branch.
Issue reference