feat: Include available connectors in search/execute tool descriptions#355
feat: Include available connectors in search/execute tool descriptions#355shashi-stackone merged 8 commits intomainfrom
Conversation
commit: |
shashi-stackone
changed the title
There was a problem hiding this comment.
Pull request overview
This PR enhances the agent “search and execute” meta-tools by dynamically including the set of available connector keys in the tool_search / tool_execute descriptions, so the LLM can see what’s searchable before deciding to call them.
Changes:
- Extend
createSearchTool/createExecuteToolto optionally append an “Available connectors: …” suffix to their descriptions. - Make
StackOneToolSet.buildTools()(andgetTools()) async to best-effortfetchTools()and derive connector keys viaTools.getConnectors(). - Update
openai({ mode: 'search_and_execute' })to await the async tool builder.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/toolsets.ts
Outdated
| async getTools(options?: { accountIds?: string[] }): Promise<Tools> { | ||
| return this.buildTools(options?.accountIds); | ||
| } |
There was a problem hiding this comment.
getTools() now returns a Promise<Tools> instead of Tools, which is a breaking API change for consumers (and the JSDoc still describes a synchronous Tools return). Consider preserving the existing sync method (e.g., keep getTools() sync with generic descriptions and add a new async variant like getToolsAsync()/getTools({ includeConnectorsInDescription: true })), or clearly treat this as a major-version change and update the JSDoc @returns accordingly.
src/toolsets.ts
Outdated
| // Discover available connectors for dynamic descriptions | ||
| let connectors = ''; | ||
| try { | ||
| const allTools = await this.fetchTools({ accountIds }); | ||
| const connectorSet = allTools.getConnectors(); | ||
| if (connectorSet.size > 0) { | ||
| connectors = Array.from(connectorSet).sort().join(', '); | ||
| } | ||
| } catch { | ||
| // Best-effort: if discovery fails, use generic descriptions | ||
| } |
There was a problem hiding this comment.
buildTools() now calls fetchTools() up front just to compute connector names. This adds an extra network/IO round trip whenever openai({ mode: 'search_and_execute' }) (or getTools()) is used, and tool_execute will then call fetchTools() again on first execution because its cache starts empty. Consider reusing the prefetched allTools to seed tool_execute’s cache (or passing prefetched tools into createExecuteTool) to avoid duplicate fetches and reduce startup latency.
| // Discover available connectors for dynamic descriptions | |
| let connectors = ''; | |
| try { | |
| const allTools = await this.fetchTools({ accountIds }); | |
| const connectorSet = allTools.getConnectors(); | |
| if (connectorSet.size > 0) { | |
| connectors = Array.from(connectorSet).sort().join(', '); | |
| } | |
| } catch { | |
| // Best-effort: if discovery fails, use generic descriptions | |
| } | |
| // Connector names are omitted here to avoid an extra upfront fetchTools() | |
| // call. Descriptions remain generic; tool_execute will fetch tools lazily. | |
| let connectors = ''; |
| const connectorLine = connectors ? ` Available connectors: ${connectors}.` : ''; | ||
| const tool = new BaseTool( | ||
| 'tool_search', | ||
| 'Search for available tools by describing what you need. Returns matching tool names, descriptions, and parameter schemas. Use the returned parameter schemas to know exactly what to pass when calling tool_execute.', | ||
| `Search for available tools by describing what you need. Returns matching tool names, descriptions, and parameter schemas. Use the returned parameter schemas to know exactly what to pass when calling tool_execute.${connectorLine}`, | ||
| searchParameters, |
There was a problem hiding this comment.
Appending all connector keys into the tool description can grow very large in multi-connector environments, increasing prompt token usage and potentially exceeding provider/tool-description length limits. Consider truncating the list (e.g., max N connectors or max chars) and/or summarizing with “and X more” to keep descriptions bounded.
src/toolsets.ts
Outdated
| // Discover available connectors for dynamic descriptions | ||
| let connectors = ''; | ||
| try { | ||
| const allTools = await this.fetchTools({ accountIds }); | ||
| const connectorSet = allTools.getConnectors(); | ||
| if (connectorSet.size > 0) { | ||
| connectors = Array.from(connectorSet).sort().join(', '); | ||
| } | ||
| } catch { | ||
| // Best-effort: if discovery fails, use generic descriptions | ||
| } | ||
|
|
||
| const searchTool = createSearchTool(this, accountIds, connectors); | ||
| const executeTool = createExecuteTool(this, accountIds, connectors); | ||
| return new Tools([searchTool, executeTool]); |
There was a problem hiding this comment.
There are no tests covering the new dynamic-description behavior (connector discovery success path, sorting/joining, and best-effort failure path). Adding unit tests that assert tool_search/tool_execute descriptions include the connector list when fetchTools() succeeds and remain generic when it throws would help prevent regressions.
shashi-stackone
changed the title
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 1 file (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/toolsets.ts">
<violation number="1">
P2: `getTools()` no longer discovers connectors, so `tool_search`/`tool_execute` descriptions remain generic for direct `getTools()` users.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| @@ -332,10 +332,15 @@ const localConfig = (id: string): LocalExecuteConfig => ({ | |||
| }); | |||
There was a problem hiding this comment.
P2: getTools() no longer discovers connectors, so tool_search/tool_execute descriptions remain generic for direct getTools() users.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/toolsets.ts, line 645:
<comment>`getTools()` no longer discovers connectors, so `tool_search`/`tool_execute` descriptions remain generic for direct `getTools()` users.</comment>
<file context>
@@ -642,32 +642,20 @@ export class StackOneToolSet {
* @returns Tools collection containing tool_search and tool_execute
*/
- async getTools(options?: { accountIds?: string[] }): Promise<Tools> {
+ getTools(options?: { accountIds?: string[] }): Tools {
return this.buildTools(options?.accountIds);
}
</file context>
![aikido-pr-checks[bot]](https://avatars.githubusercontent.com/in/898896?s=60&v=4){kind=small}
| } catch { | ||
| // Best-effort: if discovery fails, use generic descriptions |
There was a problem hiding this comment.
Bare catch in openai() silently swallows errors during connector discovery; add minimal handling such as logging the error or recording telemetry.
| } catch { | |
| // Best-effort: if discovery fails, use generic descriptions | |
| } catch (error) { | |
| // Best-effort: if discovery fails, use generic descriptions | |
| console.warn('Failed to discover connectors for tool descriptions:', error); |
Details
✨ AI Reasoning
A new try/catch was added around connector discovery in openai(). The catch clause is a bare catch with no error variable and its body contains only a comment indicating best-effort failure handling. Silent catches can mask unexpected errors and make debugging harder. Even with an explanatory comment, the catch swallows all errors without logging or any observable action, reducing visibility into failures of connector discovery.
Reply @AikidoSec feedback: [FEEDBACK] to get better review comments in the future.
Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info
shashi-stackone
changed the title
There was a problem hiding this comment.
1 issue found across 1 file (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="examples/workday-integration.ts">
<violation number="1" location="examples/workday-integration.ts:15">
P2: Do not default `STACKONE_ACCOUNT_ID` to `''`; fail fast (or omit accountIds) when the env var is missing so calls are not scoped to an invalid empty account ID.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
Summary
openai({ mode: 'search_and_execute' })now discovers available connectorsand injects them into the
tool_searchandtool_executedescriptionstrigger reliability
getTools()signature unchangedA recurring issue with Search/Execute is that search doesn't reliably trigger
in the first place — the LLM doesn't know what's searchable. By dynamically
listing connector keys in the tool description (e.g. "Available connectors:
bamboohr, calendly, jotform"), the LLM has the context it needs to trigger
search confidently.
New things
StackOneToolSet(timeout=120)or in execute configexecute={"timeout": 120}Resolves:
StackOneHQ/stackone-ai-python#166
#355
Connector discovery is best-effort : if it fails, descriptions remain generic.