.NET: [Feature Branch] Add basic durable workflow support

[kshyju]

Motivation and Context

This PR adds durable workflow support to the Agent Framework, enabling workflows to run as Durable Task orchestrations. This addresses the need for long-running, reliable workflow execution that can survive process restarts, handle failures gracefully, and supports distributed workflow execution across multiple nodes.

Note: This is the first in a series of PRs for durable workflow support. This PR establishes the foundational APIs and infrastructure. Follow-up PRs will add additional features such as events, yield output, human-in-the-loop patterns, and sub-workflow support.

Key scenarios enabled:

• Running workflows as durable orchestrations with automatic state persistence
• Sequential and concurrent workflow execution patterns
• Fan-out/fan-in workflow patterns
• Conditional routing based on runtime conditions
• Integration with existing Durable Task infrastructure

Description

This PR introduces a new set of public APIs for running workflows as Durable Task orchestrations.

New Public APIs

Workflow Execution Interfaces

API	Description
IWorkflowRun	Represents a running workflow instance with event tracking
IAwaitableWorkflowRun	Extends IWorkflowRun with completion awaiting capability
IWorkflowClient	Client interface for starting and managing workflow executions

public interface IWorkflowRun
{
    string RunId { get; }
    IEnumerable<WorkflowEvent> OutgoingEvents { get; }
    IEnumerable<WorkflowEvent> NewEvents { get; }
    int NewEventCount { get; }
}

public interface IAwaitableWorkflowRun : IWorkflowRun
{
    ValueTask<TResult?> WaitForCompletionAsync<TResult>(CancellationToken cancellationToken = default);
}

public interface IWorkflowClient
{
    ValueTask<IWorkflowRun> RunAsync<TInput>(Workflow workflow, TInput input, string? runId = null, CancellationToken cancellationToken = default) where TInput : notnull;
    ValueTask<IWorkflowRun> RunAsync(Workflow workflow, string input, string? runId = null, CancellationToken cancellationToken = default);
}

Configuration Options

API	Description
DurableOptions	Root configuration container for durable agents and workflows
DurableWorkflowOptions	Configuration for registering and managing durable workflows

public sealed class DurableOptions
{
    public DurableAgentsOptions Agents { get; }
    public DurableWorkflowOptions Workflows { get; }
}

public sealed class DurableWorkflowOptions
{
    public IReadOnlyDictionary<string, Workflow> Workflows { get; }
    public void AddWorkflow(Workflow workflow);
    public void AddWorkflows(IEnumerable<Workflow> workflows);
}

Service Collection Extensions

API	Description
ConfigureDurableOptions	Configures both durable agents and workflows
ConfigureDurableWorkflows	Configures durable workflows only

public static IServiceCollection ConfigureDurableOptions(
    this IServiceCollection services,
    Action<DurableOptions> configure,
    Action<IDurableTaskWorkerBuilder>? workerBuilder = null,
    Action<IDurableTaskClientBuilder>? clientBuilder = null);

public static IServiceCollection ConfigureDurableWorkflows(
    this IServiceCollection services,
    Action<DurableWorkflowOptions> configure,
    Action<IDurableTaskWorkerBuilder>? workerBuilder = null,
    Action<IDurableTaskClientBuilder>? clientBuilder = null);

Usage Examples

Sequential Workflow (Sample: 01_SequentialWorkflow)

// Define executors for the workflow
OrderLookup orderLookup = new();
OrderCancel orderCancel = new();
SendEmail sendEmail = new();

// Build the CancelOrder workflow: OrderLookup -> OrderCancel -> SendEmail
Workflow cancelOrder = new WorkflowBuilder(orderLookup)
    .WithName("CancelOrder")
    .WithDescription("Cancel an order and notify the customer")
    .AddEdge(orderLookup, orderCancel)
    .AddEdge(orderCancel, sendEmail)
    .Build();

// Register durable workflows
services.ConfigureDurableWorkflows(
    workflowOptions => workflowOptions.AddWorkflow(cancelOrder),
    workerBuilder: builder => builder.UseDurableTaskScheduler(dtsConnectionString),
    clientBuilder: builder => builder.UseDurableTaskScheduler(dtsConnectionString));

// Run a workflow
IWorkflowClient client = serviceProvider.GetRequiredService<IWorkflowClient>();
IAwaitableWorkflowRun run = (IAwaitableWorkflowRun)await client.RunAsync(cancelOrder, orderId);
string? result = await run.WaitForCompletionAsync<string>();

Fan-out/Fan-in Workflow (Sample: 02_ConcurrentWorkflow)

// Define executors: 2 class-based executors and 2 AI agents
ParseQuestionExecutor parseQuestion = new();
AIAgent physicist = chatClient.AsAIAgent("You are a physics expert.", "Physicist");
AIAgent chemist = chatClient.AsAIAgent("You are a chemistry expert.", "Chemist");
AggregatorExecutor aggregator = new();

// Build workflow: ParseQuestion -> [Physicist, Chemist] (parallel) -> Aggregator
Workflow workflow = new WorkflowBuilder(parseQuestion)
    .WithName("ExpertReview")
    .AddFanOutEdge(parseQuestion, [physicist, chemist])
    .AddFanInEdge([physicist, chemist], aggregator)
    .Build();

// Register using ConfigureDurableOptions (supports both agents and workflows)
services.ConfigureDurableOptions(
    options => options.Workflows.AddWorkflow(workflow),
    workerBuilder: builder => builder.UseDurableTaskScheduler(dtsConnectionString),
    clientBuilder: builder => builder.UseDurableTaskScheduler(dtsConnectionString));

Conditional Edges Workflow (Sample: 03_ConditionalEdges)

// Define executors for the workflow
OrderIdParser orderParser = new();
OrderEnrich orderEnrich = new();
PaymentProcessor paymentProcessor = new();
NotifyFraud notifyFraud = new();

// Build workflow with conditional routing based on customer status
// OrderIdParser -> OrderEnrich --[IsBlocked]--> NotifyFraud
//                             |--[NotBlocked]--> PaymentProcessor
Workflow orderAudit = new WorkflowBuilder(orderParser)
    .WithName("OrderAudit")
    .WithDescription("Audit order and route based on customer status")
    .AddEdge(orderParser, orderEnrich)
    .AddEdge(orderEnrich, notifyFraud, condition: OrderRouteConditions.WhenBlocked())
    .AddEdge(orderEnrich, paymentProcessor, condition: OrderRouteConditions.WhenNotBlocked())
    .Build();

// Condition functions for routing logic
internal static class OrderRouteConditions
{
    internal static Func<Order?, bool> WhenBlocked() => 
        order => order?.Customer?.IsBlocked == true;

    internal static Func<Order?, bool> WhenNotBlocked() => 
        order => order?.Customer?.IsBlocked == false;
}

Validation/Testing

Samples Added:

Sample	Description
samples/Durable/Workflow/ConsoleApps/01_SequentialWorkflow	Demonstrates a simple sequential workflow pattern
samples/Durable/Workflow/ConsoleApps/02_ConcurrentWorkflow	Demonstrates fan-out/fan-in pattern with AI agents
samples/Durable/Workflow/ConsoleApps/03_ConditionalEdges	Demonstrates conditional routing based on runtime conditions

Integration Tests: Integration tests have been added to run and validate the sample workflows, ensuring end-to-end functionality works as expected.

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the Contribution Guidelines
• All unit tests pass, and I have added new tests where possible
• Is this a breaking change? No

[kshyju]

The change here is to move common logic to SamplesValidationBase so that both this and the workflow's console app samples validation IT can use same code.

[kshyju]

This class was moved to DurableDataConverter (renamed)

[kshyju]

This was the previously called DefaultDataConverter. I moved it to this file and renamed.

[kshyju]

This was added after talking to MAF folks. we are still discussing the proposal about the api change which may go in the core library. In that case, this will be removed.

[kshyju]

Code here is moved from the existing agents console app samples validation file.

[Copilot]

Pull request overview

This PR introduces the initial infrastructure for running Microsoft.Agents.AI.Workflows.Workflow graphs as Durable Task orchestrations, plus samples and tests that exercise sequential and concurrent patterns.

Changes:

• Add durable workflow runtime components (options, DI extensions, orchestration runner, edge routing, executor dispatch, and JSON context) under Microsoft.Agents.AI.DurableTask.
• Add public workflow execution APIs (IWorkflowClient, IWorkflowRun, IAwaitableWorkflowRun) and wire them into the Durable Task registration pipeline.
• Add two durable workflow console samples (sequential and fan-out/fan-in with AI agents) and corresponding integration tests that validate them end-to-end, including shared DTS/Redis test infrastructure.

Reviewed changes

Copilot reviewed 46 out of 46 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
dotnet/tests/Microsoft.Agents.AI.DurableTask.UnitTests/Workflows/WorkflowNamingHelperTests.cs	Adds unit tests for workflow/orchestration name conversion and executor ID parsing.
dotnet/tests/Microsoft.Agents.AI.DurableTask.UnitTests/Microsoft.Agents.AI.DurableTask.UnitTests.csproj	References the Workflows project so workflow-related tests can compile.
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/WorkflowConsoleAppSamplesValidation.cs	Adds integration tests that drive the new durable workflow console samples and assert expected log lines and completion.
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/SamplesValidationBase.cs	Introduces shared infra for console-sample integration tests (DTS/Redis bootstrapping, process I/O, log streaming).
dotnet/tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/ConsoleAppSamplesValidation.cs	Refactors durable agent sample tests to reuse SamplesValidationBase and rely on overridden env-var wiring (e.g., Redis) for agents.
dotnet/src/Microsoft.Agents.AI.Workflows/Microsoft.Agents.AI.Workflows.csproj	Grants internals visibility to Microsoft.Agents.AI.DurableTask so workflows internals can be analyzed.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/WorkflowNamingHelper.cs	Centralizes naming conventions for orchestration function names and executor IDs, including stripping GUID suffixes.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/WorkflowGraphInfo.cs	Defines the minimal graph representation (successors, predecessors, conditions, output types) used for message-driven superstep execution.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/WorkflowAnalyzer.cs	Analyzes Workflow instances into WorkflowExecutorInfo and WorkflowGraphInfo, identifies agent executors, and extracts executor output types.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/SentMessageInfo.cs	Models messages emitted from executors via IWorkflowContext.SendMessageAsync for durable transport.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/IWorkflowRun.cs	Public interface describing a workflow run by ID and exposing outgoing/new WorkflowEvent streams.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/IWorkflowClient.cs	Public client interface for starting workflows with typed or string inputs and obtaining IWorkflowRun handles.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/IAwaitableWorkflowRun.cs	Public interface extending IWorkflowRun with WaitForCompletionAsync<TResult> for runs that support awaiting completion.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/ExecutorRegistry.cs	Internal registry mapping logical executor names to bindings for later instantiation, and a helper ExecutorRegistration record.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/EdgeRouters/IDurableEdgeRouter.cs	Defines the routing contract for delivering messages between executors in a durable workflow.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/EdgeRouters/DurableFanOutEdgeRouter.cs	Implements fan-out routing from a single source to multiple target edge routers.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/EdgeRouters/DurableEdgeMap.cs	Builds per-executor routing tables and predecessor counts from WorkflowGraphInfo and manages message queues and fan-in detection.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/EdgeRouters/DurableDirectEdgeRouter.cs	Implements direct edge routing with optional conditional evaluation and JSON-based deserialization of predecessor outputs.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/DurableWorkflowServiceCollectionExtensions.cs	Adds ConfigureDurableWorkflows DI extension that configures only workflows via the underlying ConfigureDurableOptions.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/DurableWorkflowRunner.cs	Orchestration runner that executes workflows via message-driven supersteps, dispatching executors and propagating outputs/messages between steps.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/DurableWorkflowRun.cs	Implements IAwaitableWorkflowRun for durable orchestrations using DurableTaskClient and exposes an event sink (not yet wired) for workflow events.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/DurableWorkflowOptions.cs	Adds workflow-specific options: registering named workflows, tracking executors, and auto-registering any referenced agents into DurableAgentsOptions.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/DurableWorkflowJsonContext.cs	Source-generated JSON serialization context for durable workflow payload types (activity input/output, sent messages, state dictionary).
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/DurableWorkflowClient.cs	Implements IWorkflowClient using DurableTaskClient to schedule new orchestration instances and wrap them in DurableWorkflowRun.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/DurableMessageEnvelope.cs	Wraps serialized message payloads with type-name and source-executor metadata for durable routing.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/DurableExecutorDispatcher.cs	Chooses between dispatching an executor as a Durable activity or AI agent, including entity-based execution for agents.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/DurableActivityOutput.cs	Defines the serialized output format from an activity, including result and any sent messages.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/DurableActivityInput.cs	Defines the serialized input format into an activity, including executor input, input type name, and shared state.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/DurableActivityExecutor.cs	Executes a bound executor from serialized durable input, invokes Executor.ExecuteAsync, and returns serialized output plus sent messages.
dotnet/src/Microsoft.Agents.AI.DurableTask/Workflows/DurableActivityContext.cs	Provides an IWorkflowContext implementation for activities, capturing sent messages but currently no-ops for state/events/halt APIs.
dotnet/src/Microsoft.Agents.AI.DurableTask/ServiceCollectionExtensions.cs	Switches the durable task data converter registration to the new shared DurableDataConverter.
dotnet/src/Microsoft.Agents.AI.DurableTask/Microsoft.Agents.AI.DurableTask.csproj	References the Workflows project, enabling durable workflows to use workflow primitives.
dotnet/src/Microsoft.Agents.AI.DurableTask/Logs.cs	Adds logging helpers for workflow lifecycle, supersteps, fan-in aggregation, executor results, and agent lookup failures.
dotnet/src/Microsoft.Agents.AI.DurableTask/DurableServiceCollectionExtensions.cs	New DI extension entry point for configuring durable agents and workflows, registering orchestrators, activities, entities, and agent services.
dotnet/src/Microsoft.Agents.AI.DurableTask/DurableOptions.cs	Introduces root options object combining durable agents and workflows, used by ConfigureDurableOptions.
dotnet/src/Microsoft.Agents.AI.DurableTask/DurableDataConverter.cs	Shared DataConverter handling durable agent state via source-generated context and camel-cased JSON for other payloads.
dotnet/src/Microsoft.Agents.AI.DurableTask/DurableAgentsOptions.cs	Adds ContainsAgent helper to check whether an agent has already been registered.
dotnet/samples/Durable/Workflow/ConsoleApps/02_ConcurrentWorkflow/README.md	Documents the concurrent workflow sample, its fan-out/fan-in pattern, environment configuration, and example output.
dotnet/samples/Durable/Workflow/ConsoleApps/02_ConcurrentWorkflow/Program.cs	Sample host wiring a fan-out/fan-in Workflow with 2 class executors and 2 AI agents into Durable Task via ConfigureDurableOptions.
dotnet/samples/Durable/Workflow/ConsoleApps/02_ConcurrentWorkflow/ExpertExecutors.cs	Implements the ParseQuestionExecutor and AggregatorExecutor used in the concurrent workflow sample.
dotnet/samples/Durable/Workflow/ConsoleApps/02_ConcurrentWorkflow/02_ConcurrentWorkflow.csproj	Project file for the concurrent workflow console sample, referencing core durable/AI projects.
dotnet/samples/Durable/Workflow/ConsoleApps/01_SequentialWorkflow/README.md	Documents the sequential cancellation workflow sample and how to demonstrate durability across restarts.
dotnet/samples/Durable/Workflow/ConsoleApps/01_SequentialWorkflow/Program.cs	Sample host wiring a 3-step order-cancellation Workflow into Durable Task via ConfigureDurableWorkflows.
dotnet/samples/Durable/Workflow/ConsoleApps/01_SequentialWorkflow/OrderCancelExecutors.cs	Defines the OrderLookup, OrderCancel, and SendEmail executors used by the sequential workflow sample.
dotnet/samples/Durable/Workflow/ConsoleApps/01_SequentialWorkflow/01_SequentialWorkflow.csproj	Project file for the sequential workflow console sample, referencing durable task and agent projects.
dotnet/agent-framework-dotnet.slnx	Adds the new durable workflow sample projects to the .NET solution.

Additional note: Because this PR changes dotnet/src/Microsoft.Agents.AI.DurableTask/**, it likely needs a new bullet under the [Unreleased] section of dotnet/src/Microsoft.Agents.AI.DurableTask/CHANGELOG.md per the repo’s durable-task instructions.

[cgillum]

Some initial comments. I'm not yet done reviewing.

[cgillum]

Great start on this! I finished my first full round. Added a few comments below. Overall, I'm good with the direction.

[cgillum]

Does the core workflow MAF library not have an abstraction for workflow runs?

[kshyju]

No. It returns a concrete Run type.

https://github.com/microsoft/agent-framework/blob/907654a489a4f41bd4b44618ce5daa994383cb3c/dotnet/src/Microsoft.Agents.AI.Workflows/InProcessExecution.cs#L65C29-L65C32

This is one of the API change proposals we shared with them.