Python: Integrate directly with Foundry Tools from agents using any chat clients

[rogerbarreto]

Part of #3440

Integrate directly with Foundry Tools from agents that use any chat clients. Meeting to discuss this will be help this week.

[rogerbarreto]

Subplan: #3674 - Integrate Foundry Tools from agents using any chat clients

Issue: #3674
Parent Issue: #3440
Scope: .NET only - MAF.AI.AzureAI package (FoundryAgent)
Deliverable: Sample files demonstrating Foundry Tool usage

---

Objective

Create .NET samples demonstrating how to use Azure AI Foundry Tools directly from FoundryAgent within the MAF.AI.AzureAI package. All samples should follow existing patterns from dotnet/samples/GettingStarted/FoundryAgents/.

---

Foundry Tools to Cover

Based on main plan analysis, these are the tools available in Python that need .NET samples:

Tool	Python Sample Reference	Priority	Complexity
Bing Grounding	azure_ai_with_bing_grounding.py	🔴 High	Low
Bing Custom Search	azure_ai_with_bing_custom_search.py	🟡 Medium	Low
Azure AI Search	azure_ai_with_azure_ai_search.py	🔴 High	Medium
File Search	azure_ai_with_file_search.py	🔴 High	Medium
SharePoint Grounding	azure_ai_with_sharepoint.py	🟡 Medium	Medium
Microsoft Fabric	azure_ai_with_microsoft_fabric.py	🟡 Medium	Medium
Memory Search	azure_ai_with_memory_search.py	🟡 Medium	High
Browser Automation	azure_ai_with_browser_automation.py	🟡 Medium	Medium
Web Search	azure_ai_with_web_search.py	🟡 Medium	Low
OpenAPI Tools	azure_ai_with_openapi.py	🔴 High	Medium
Content Filtering (RAI)	azure_ai_with_content_filtering.py	🔴 High	Low
Local MCP	azure_ai_with_local_mcp.py	🟢 Low	Medium

---

Workplan

Phase 1: Foundation & High Priority Tools

• Task 1.1: Review existing .NET Step14 (Code Interpreter) pattern for tool integration approach
• Task 1.2: Create Step16_BingGrounding sample
  • File: FoundryAgents_Step16_BingGrounding/Program.cs
  • Reference: azure_ai_with_bing_grounding.py
  • Demonstrates: Basic Bing grounding tool configuration with FoundryAgentDefinition
• Task 1.3: Create Step17_AzureAISearch sample
  • File: FoundryAgents_Step17_AzureAISearch/Program.cs
  • Reference: azure_ai_with_azure_ai_search.py
  • Demonstrates: Azure AI Search integration for RAG scenarios
• Task 1.4: Create Step18_FileSearch sample
  • File: FoundryAgents_Step18_FileSearch/Program.cs
  • Reference: azure_ai_with_file_search.py
  • Demonstrates: File search tool with vector stores
• Task 1.5: Create Step19_OpenAPITools sample
  • File: FoundryAgents_Step19_OpenAPITools/Program.cs
  • Reference: azure_ai_with_openapi.py
  • Demonstrates: OpenAPI spec-based external API integration
• Task 1.6: Create Step20_ContentFiltering sample
  • File: FoundryAgents_Step20_ContentFiltering/Program.cs
  • Reference: azure_ai_with_content_filtering.py
  • Demonstrates: RAI content filtering configuration

Phase 2: Medium Priority Tools

• Task 2.1: Create Step21_BingCustomSearch sample
  • Reference: azure_ai_with_bing_custom_search.py
• Task 2.2: Create Step22_SharePointGrounding sample
  • Reference: azure_ai_with_sharepoint.py
• Task 2.3: Create Step23_MicrosoftFabric sample
  • Reference: azure_ai_with_microsoft_fabric.py
• Task 2.4: Create Step24_BrowserAutomation sample
  • Reference: azure_ai_with_browser_automation.py
• Task 2.5: Create Step25_WebSearch sample
  • Reference: azure_ai_with_web_search.py
• Task 2.6: Create Step26_MemorySearch sample
  • Reference: azure_ai_with_memory_search.py

Phase 3: Low Priority & Documentation

• Task 3.1: Create Step27_LocalMCP sample
  • Reference: azure_ai_with_local_mcp.py
• Task 3.2: Update FoundryAgents/README.md with new samples
• Task 3.3: Verify all samples build and run successfully

---

Sample Structure Template

Each sample should follow this pattern (based on existing .NET samples):

FoundryAgents_StepXX_{FeatureName}/
├── FoundryAgents_StepXX_{FeatureName}.csproj
├── Program.cs
└── README.md (optional)

Code Pattern Reference

// Pattern from Step14_CodeInterpreter
var agentDefinition = new FoundryAgentDefinition
{
    Name = "...",
    Instructions = "...",
    Model = modelId,
    Tools = new List<ToolDefinition>
    {
        // Tool-specific definition
    }
};

---

Dependencies & Prerequisites

• Verify Azure SDK has .NET bindings for each tool type
• Check if ToolDefinition classes exist for:
  • BingGroundingToolDefinition
  • AzureAISearchToolDefinition
  • FileSearchToolDefinition
  • OpenAPIToolDefinition
  • etc.
• Identify any gaps requiring Azure SDK updates

---

Python Sample References

Sample	GitHub Link
Bing Grounding	azure_ai_with_bing_grounding.py
Azure AI Search	azure_ai_with_azure_ai_search.py
File Search	azure_ai_with_file_search.py
OpenAPI	azure_ai_with_openapi.py
Content Filtering	azure_ai_with_content_filtering.py
SharePoint	azure_ai_with_sharepoint.py
Microsoft Fabric	azure_ai_with_microsoft_fabric.py
Memory Search	azure_ai_with_memory_search.py
Browser Automation	azure_ai_with_browser_automation.py
Web Search	azure_ai_with_web_search.py
Local MCP	azure_ai_with_local_mcp.py

---

Open Questions

1. Priority Order: Should we prioritize by tool complexity or customer demand?
2. Azure SDK Gaps: Are there any tools without .NET SDK support?
3. Shared Resources: Should samples share common resources (e.g., OpenAPI specs)?

---

Success Criteria

• All high-priority samples created and functional
• Samples follow existing naming conventions (FoundryAgents_StepXX_*)
• Each sample includes necessary Azure resource setup comments
• README updated with new sample descriptions
• All samples build with dotnet build