-
Notifications
You must be signed in to change notification settings - Fork 1.1k
.NET: [Feature Branch] Add basic durable workflow support #3648
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kshyju
wants to merge
6
commits into
microsoft:feat/durable_task
Choose a base branch
from
kshyju:dwf_console_1
base: feat/durable_task
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
29 changes: 29 additions & 0 deletions
29
...t/samples/Durable/Workflow/ConsoleApps/01_SequentialWorkflow/01_SequentialWorkflow.csproj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
| <PropertyGroup> | ||
| <TargetFrameworks>net10.0</TargetFrameworks> | ||
| <OutputType>Exe</OutputType> | ||
| <ImplicitUsings>enable</ImplicitUsings> | ||
| <Nullable>enable</Nullable> | ||
| <AssemblyName>SequentialWorkflow</AssemblyName> | ||
| <RootNamespace>SequentialWorkflow</RootNamespace> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PackageReference Include="Azure.Identity" /> | ||
| <PackageReference Include="Microsoft.DurableTask.Client.AzureManaged" /> | ||
| <PackageReference Include="Microsoft.DurableTask.Worker.AzureManaged" /> | ||
| <PackageReference Include="Microsoft.Extensions.Hosting" /> | ||
| </ItemGroup> | ||
|
|
||
| <!-- Local projects that should be switched to package references when using the sample outside of this MAF repo --> | ||
| <!-- | ||
| <ItemGroup> | ||
| <PackageReference Include="Microsoft.Agents.AI.DurableTask" /> | ||
| <PackageReference Include="Microsoft.Agents.AI.Workflows" /> | ||
| </ItemGroup> | ||
| --> | ||
| <ItemGroup> | ||
| <ProjectReference Include="..\..\..\..\..\src\Microsoft.Agents.AI.DurableTask\Microsoft.Agents.AI.DurableTask.csproj" /> | ||
| <ProjectReference Include="..\..\..\..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" /> | ||
| </ItemGroup> | ||
| </Project> |
107 changes: 107 additions & 0 deletions
107
dotnet/samples/Durable/Workflow/ConsoleApps/01_SequentialWorkflow/OrderCancelExecutors.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| // Copyright (c) Microsoft. All rights reserved. | ||
|
|
||
| using Microsoft.Agents.AI.Workflows; | ||
|
|
||
| namespace SequentialWorkflow; | ||
|
|
||
| /// <summary> | ||
| /// Looks up an order by its ID and return an Order object. | ||
| /// </summary> | ||
| internal sealed class OrderLookup() : Executor<string, Order>("OrderLookup") | ||
| { | ||
| public override async ValueTask<Order> HandleAsync( | ||
| string message, | ||
| IWorkflowContext context, | ||
| CancellationToken cancellationToken = default) | ||
| { | ||
| Console.WriteLine(); | ||
| Console.ForegroundColor = ConsoleColor.Magenta; | ||
| Console.WriteLine("┌─────────────────────────────────────────────────────────────────┐"); | ||
| Console.WriteLine($"│ [Activity] OrderLookup: Starting lookup for order '{message}'"); | ||
| Console.ResetColor(); | ||
|
|
||
| // Simulate database lookup with delay | ||
| await Task.Delay(TimeSpan.FromMicroseconds(100), cancellationToken); | ||
|
|
||
| Order order = new( | ||
| Id: message, | ||
| OrderDate: DateTime.UtcNow.AddDays(-1), | ||
| IsCancelled: false, | ||
| Customer: new Customer(Name: "Jerry", Email: "jerry@example.com")); | ||
|
|
||
| Console.ForegroundColor = ConsoleColor.Magenta; | ||
| Console.WriteLine($"│ [Activity] OrderLookup: Found order '{message}' for customer '{order.Customer.Name}'"); | ||
| Console.WriteLine("└─────────────────────────────────────────────────────────────────┘"); | ||
| Console.ResetColor(); | ||
|
|
||
| return order; | ||
| } | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Cancels an order. | ||
| /// </summary> | ||
| internal sealed class OrderCancel() : Executor<Order, Order>("OrderCancel") | ||
| { | ||
| public override async ValueTask<Order> HandleAsync( | ||
| Order message, | ||
| IWorkflowContext context, | ||
| CancellationToken cancellationToken = default) | ||
| { | ||
| // Log that this activity is executing (not replaying) | ||
| Console.WriteLine(); | ||
| Console.ForegroundColor = ConsoleColor.Yellow; | ||
| Console.WriteLine("┌─────────────────────────────────────────────────────────────────┐"); | ||
| Console.WriteLine($"│ [Activity] OrderCancel: Starting cancellation for order '{message.Id}'"); | ||
| Console.ResetColor(); | ||
|
|
||
| // Simulate a slow cancellation process (e.g., calling external payment system) | ||
| for (int i = 1; i <= 3; i++) | ||
| { | ||
| await Task.Delay(TimeSpan.FromMilliseconds(100), cancellationToken); | ||
| Console.ForegroundColor = ConsoleColor.DarkYellow; | ||
| Console.WriteLine("│ [Activity] OrderCancel: Processing..."); | ||
| Console.ResetColor(); | ||
| } | ||
|
|
||
| Order cancelledOrder = message with { IsCancelled = true }; | ||
|
|
||
| Console.ForegroundColor = ConsoleColor.Yellow; | ||
| Console.WriteLine($"│ [Activity] OrderCancel: ✓ Order '{cancelledOrder.Id}' has been cancelled"); | ||
| Console.WriteLine("└─────────────────────────────────────────────────────────────────┘"); | ||
| Console.ResetColor(); | ||
|
|
||
| return cancelledOrder; | ||
| } | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Sends a cancellation confirmation email to the customer. | ||
| /// </summary> | ||
| internal sealed class SendEmail() : Executor<Order, string>("SendEmail") | ||
| { | ||
| public override ValueTask<string> HandleAsync( | ||
| Order message, | ||
| IWorkflowContext context, | ||
| CancellationToken cancellationToken = default) | ||
| { | ||
| Console.WriteLine(); | ||
| Console.ForegroundColor = ConsoleColor.Cyan; | ||
| Console.WriteLine("┌─────────────────────────────────────────────────────────────────┐"); | ||
| Console.WriteLine($"│ [Activity] SendEmail: Sending email to '{message.Customer.Email}'..."); | ||
| Console.ResetColor(); | ||
|
|
||
| string result = $"Cancellation email sent for order {message.Id} to {message.Customer.Email}."; | ||
|
|
||
| Console.ForegroundColor = ConsoleColor.Cyan; | ||
| Console.WriteLine("│ [Activity] SendEmail: ✓ Email sent successfully!"); | ||
| Console.WriteLine("└─────────────────────────────────────────────────────────────────┘"); | ||
| Console.ResetColor(); | ||
|
|
||
| return ValueTask.FromResult(result); | ||
| } | ||
| } | ||
|
|
||
| internal sealed record Order(string Id, DateTime OrderDate, bool IsCancelled, Customer Customer); | ||
|
|
||
| internal sealed record Customer(string Name, string Email); | ||
91 changes: 91 additions & 0 deletions
91
dotnet/samples/Durable/Workflow/ConsoleApps/01_SequentialWorkflow/Program.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| // Copyright (c) Microsoft. All rights reserved. | ||
|
|
||
| using Microsoft.Agents.AI.DurableTask.Workflows; | ||
| using Microsoft.Agents.AI.Workflows; | ||
| using Microsoft.DurableTask.Client.AzureManaged; | ||
| using Microsoft.DurableTask.Worker.AzureManaged; | ||
| using Microsoft.Extensions.DependencyInjection; | ||
| using Microsoft.Extensions.Hosting; | ||
| using Microsoft.Extensions.Logging; | ||
| using SequentialWorkflow; | ||
|
|
||
| // Get DTS connection string from environment variable | ||
| string dtsConnectionString = Environment.GetEnvironmentVariable("DURABLE_TASK_SCHEDULER_CONNECTION_STRING") | ||
| ?? "Endpoint=http://localhost:8080;TaskHub=default;Authentication=None"; | ||
|
|
||
| // Define executors for the workflow | ||
| OrderLookup orderLookup = new(); | ||
| OrderCancel orderCancel = new(); | ||
| SendEmail sendEmail = new(); | ||
|
|
||
| // Build the CancelOrder workflow: OrderLookup -> OrderCancel -> SendEmail | ||
| Workflow cancelOrder = new WorkflowBuilder(orderLookup) | ||
| .WithName("CancelOrder") | ||
| .WithDescription("Cancel an order and notify the customer") | ||
| .AddEdge(orderLookup, orderCancel) | ||
| .AddEdge(orderCancel, sendEmail) | ||
| .Build(); | ||
|
|
||
| IHost host = Host.CreateDefaultBuilder(args) | ||
| .ConfigureLogging(logging => logging.SetMinimumLevel(LogLevel.Warning)) | ||
| .ConfigureServices(services => | ||
| { | ||
| services.ConfigureDurableWorkflows( | ||
| workflowOptions => workflowOptions.AddWorkflow(cancelOrder), | ||
| workerBuilder: builder => builder.UseDurableTaskScheduler(dtsConnectionString), | ||
| clientBuilder: builder => builder.UseDurableTaskScheduler(dtsConnectionString)); | ||
| }) | ||
| .Build(); | ||
|
|
||
| await host.StartAsync(); | ||
|
|
||
| IWorkflowClient workflowClient = host.Services.GetRequiredService<IWorkflowClient>(); | ||
|
|
||
| Console.WriteLine("Durable Workflow Sample"); | ||
| Console.WriteLine("Workflow: OrderLookup -> OrderCancel -> SendEmail"); | ||
| Console.WriteLine(); | ||
| Console.WriteLine("Enter an order ID (or 'exit'):"); | ||
|
|
||
| while (true) | ||
| { | ||
| Console.Write("> "); | ||
| string? input = Console.ReadLine(); | ||
| if (string.IsNullOrWhiteSpace(input) || input.Equals("exit", StringComparison.OrdinalIgnoreCase)) | ||
| { | ||
| break; | ||
| } | ||
|
|
||
| try | ||
| { | ||
| await StartNewWorkflowAsync(input, cancelOrder, workflowClient); | ||
| } | ||
| catch (Exception ex) | ||
| { | ||
| Console.WriteLine($"Error: {ex.Message}"); | ||
| } | ||
|
|
||
| Console.WriteLine(); | ||
| } | ||
|
|
||
| await host.StopAsync(); | ||
|
|
||
| // Start a new workflow using IWorkflowClient (no DurableTaskClient needed) | ||
| static async Task StartNewWorkflowAsync(string orderId, Workflow workflow, IWorkflowClient client) | ||
| { | ||
| Console.WriteLine($"Starting workflow for order '{orderId}'..."); | ||
|
|
||
| // RunAsync returns IWorkflowRun, cast to IAwaitableWorkflowRun for completion waiting | ||
| IAwaitableWorkflowRun run = (IAwaitableWorkflowRun)await client.RunAsync(workflow, orderId); | ||
| Console.WriteLine($"Run ID: {run.RunId}"); | ||
|
|
||
| try | ||
| { | ||
| Console.WriteLine("Waiting for workflow to complete..."); | ||
| string? result = await run.WaitForCompletionAsync<string>(); | ||
| Console.WriteLine($"Workflow completed. {result}"); | ||
| } | ||
| catch (InvalidOperationException ex) | ||
| { | ||
| Console.WriteLine($"Failed: {ex.Message}"); | ||
| } | ||
| } |
83 changes: 83 additions & 0 deletions
83
dotnet/samples/Durable/Workflow/ConsoleApps/01_SequentialWorkflow/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| # Sequential Workflow Sample | ||
|
|
||
| This sample demonstrates how to run a sequential workflow as a durable orchestration from a console application using the Durable Task Framework. It showcases the **durability** aspect - if the process crashes mid-execution, the workflow automatically resumes without re-executing completed activities. | ||
|
|
||
| ## Key Concepts Demonstrated | ||
|
|
||
| - Building a sequential workflow with the `WorkflowBuilder` API | ||
| - Using `ConfigureDurableWorkflows` to register workflows with dependency injection | ||
| - Running workflows with `IWorkflowClient` | ||
| - **Durability**: Automatic resume of interrupted workflows | ||
| - **Activity caching**: Completed activities are not re-executed on replay | ||
|
|
||
| ## Overview | ||
|
|
||
| The sample implements an order cancellation workflow with three executors: | ||
|
|
||
| ``` | ||
| OrderLookup --> OrderCancel --> SendEmail | ||
| ``` | ||
|
|
||
| | Executor | Description | | ||
| |----------|-------------| | ||
| | OrderLookup | Looks up an order by ID | | ||
| | OrderCancel | Marks the order as cancelled | | ||
| | SendEmail | Sends a cancellation confirmation email | | ||
|
|
||
| ## Durability Demonstration | ||
|
|
||
| The key feature of Durable Task Framework is **durability**: | ||
|
|
||
| - **Activity results are persisted**: When an activity completes, its result is saved | ||
| - **Orchestrations replay**: On restart, the orchestration replays from the beginning | ||
| - **Completed activities skip execution**: The framework uses cached results | ||
| - **Automatic resume**: The worker automatically picks up pending work on startup | ||
|
|
||
| ### Try It Yourself | ||
|
|
||
| > **Tip:** To give yourself more time to stop the application during `OrderCancel`, consider increasing the loop iteration count or `Task.Delay` duration in the `OrderCancel` executor in `OrderCancelExecutors.cs`. | ||
|
|
||
| 1. Start the application and enter an order ID (e.g., `12345`) | ||
| 2. Wait for `OrderLookup` to complete, then stop the app (Ctrl+C) during `OrderCancel` | ||
| 3. Restart the application | ||
| 4. Observe: | ||
| - `OrderLookup` is **NOT** re-executed (result was cached) | ||
| - `OrderCancel` **restarts** (it didn't complete before the interruption) | ||
| - `SendEmail` runs after `OrderCancel` completes | ||
|
|
||
| ## Environment Setup | ||
|
|
||
| See the [README.md](../README.md) file in the parent directory for information on configuring the environment, including how to install and run the Durable Task Scheduler. | ||
|
|
||
| ## Running the Sample | ||
|
|
||
| ```bash | ||
| cd dotnet/samples/Durable/Workflow/ConsoleApps/01_SequentialWorkflow | ||
| dotnet run --framework net10.0 | ||
| ``` | ||
|
|
||
| ### Sample Output | ||
|
|
||
| ```text | ||
| Durable Workflow Sample | ||
| Workflow: OrderLookup -> OrderCancel -> SendEmail | ||
|
|
||
| Enter an order ID (or 'exit'): | ||
| > 12345 | ||
| Starting workflow for order: 12345 | ||
| Run ID: abc123... | ||
|
|
||
| [OrderLookup] Looking up order '12345'... | ||
| [OrderLookup] Found order for customer 'Jerry' | ||
|
|
||
| [OrderCancel] Cancelling order '12345'... | ||
| [OrderCancel] Order cancelled successfully | ||
|
|
||
| [SendEmail] Sending email to 'jerry@example.com'... | ||
| [SendEmail] Email sent successfully | ||
|
|
||
| Workflow completed! | ||
|
|
||
| > exit | ||
| ``` | ||
|
|
30 changes: 30 additions & 0 deletions
30
...t/samples/Durable/Workflow/ConsoleApps/02_ConcurrentWorkflow/02_ConcurrentWorkflow.csproj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
| <PropertyGroup> | ||
| <TargetFrameworks>net10.0</TargetFrameworks> | ||
| <OutputType>Exe</OutputType> | ||
| <ImplicitUsings>enable</ImplicitUsings> | ||
| <Nullable>enable</Nullable> | ||
| <AssemblyName>WorkflowConcurrency</AssemblyName> | ||
| <RootNamespace>WorkflowConcurrency</RootNamespace> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PackageReference Include="Azure.Identity" /> | ||
| <PackageReference Include="Microsoft.DurableTask.Client.AzureManaged" /> | ||
| <PackageReference Include="Microsoft.DurableTask.Worker.AzureManaged" /> | ||
| <PackageReference Include="Microsoft.Extensions.Hosting" /> | ||
| <PackageReference Include="Azure.AI.OpenAI" /> | ||
| </ItemGroup> | ||
|
|
||
| <!-- Local projects that should be switched to package references when using the sample outside of this MAF repo --> | ||
| <!-- | ||
| <ItemGroup> | ||
| <PackageReference Include="Microsoft.Agents.AI.DurableTask" /> | ||
| <PackageReference Include="Microsoft.Agents.AI.Workflows" /> | ||
| </ItemGroup> | ||
| --> | ||
| <ItemGroup> | ||
| <ProjectReference Include="..\..\..\..\..\src\Microsoft.Agents.AI.DurableTask\Microsoft.Agents.AI.DurableTask.csproj" /> | ||
| <ProjectReference Include="..\..\..\..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" /> | ||
| </ItemGroup> | ||
| </Project> |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.