The AI call pipeline
Everything SmartHopper sends to an AI provider — a component run, an assistant reply, an MCP tool call — flows through one normalized request/response pipeline called AICall. It lives in src/SmartHopper.ProviderSdk/AICall/ (the types) and src/SmartHopper.Infrastructure/AICall/ (the orchestration). Read this page before writing a provider, tool, or AI-powered component: it defines the vocabulary all of them share.
The idea
Different AI vendors speak different wire protocols, but every SmartHopper feature needs the same things: messages, images, tool calls and results, JSON output constraints, metrics, and errors. AICall provides that shared format.
A request is an AIBody — a list of interactions — plus per-request metadata (provider, model, capability, filters). A response is an AIReturn — the returned interactions, raw payload, metrics, status, and diagnostics. Providers translate between this neutral format and their vendor's API; consumers never see wire JSON.
Interactions: the message currency
IAIInteraction is the unit of conversation. The concrete types are in ProviderSdk/AICall/Core/Interactions/:
| Type | Carries | Typical use |
|---|---|---|
AIInteractionText | Text plus metrics (tokens, reasoning) | User prompts, assistant replies, system prompts |
AIInteractionImage | Image data/URL | Vision inputs, generated images |
AIInteractionAudio | Audio data | Speech in/out |
AIInteractionToolCall | Id, Name, Arguments (JSON) | A tool the assistant wants executed |
AIInteractionToolResult | Id, Name, result payload | The outcome, fed back to the model |
AIInteractionRuntimeMessage | Severity + message | Warnings/errors surfaced to the user |
Every interaction has an Agent from AIAgent: System, User, Assistant, Context, ToolCall, or ToolResult. Interactions are immutable-style types — helpers like AIBodyBuilder assemble them rather than mutating in place.
AIBody: the request payload
AIBody wraps the interaction list with three filters that change what the provider sees:
| Member | Default | Effect |
|---|---|---|
Interactions | — | The conversation list. Non-mutating injection point: when ContextFilter yields context, a synthesized AIInteractionText with Agent = Context is prepended to the returned copy — the stored list is unchanged. |
ContextFilter | varies | Which context providers inject facts. "*" all, "-*" none, "time, environment" a whitelist, "*, -selection" an exclusion list. |
ToolFilter | -* (none) | Which tools the model may call. "*" enables all; glob patterns like "gh_*" enable subsets. Setting this also makes the request require FunctionCalling capability. |
JsonOutputSchema | none | JSON schema for structured output; requests asserting JsonOutput without a schema fail validation. |
Build bodies with AIBodyBuilder rather than constructing lists by hand:
var body = AIBodyBuilder.Create()
.AddSystem("You are a Grasshopper assistant.")
.AddUser("List the selected components.")
.WithContextFilter("time, environment, file, -viewport")
.WithToolFilter("gh_get, gh_list_categories")
.Build();
AIRequestCall: one turn
AIRequestCall (ProviderSdk/AICall/Core/Requests/) is a single provider round-trip. Set Provider, Model, Body, Origin, then call Exec():
var request = new AIRequestCall
{
Provider = "MistralAI",
Model = "mistral-large-latest",
Body = body,
Origin = "my_feature", // semantic label — who made this call
};
AIReturn result = await request.Exec(cancellationToken);
Two related fields carry deliberately different meanings:
Origin— the semantic provenance label you set: tool name, component tag, chat surface. It survives preparation untouched and drives attribution/logging.Endpoint— the wire route the provider resolves during preparation (e.g./chat/completions). Callers leave it unset; a non-empty value is an explicit wire-path override for backoffice calls (/models, batch status).
Under the hood, Exec runs the full lifecycle — this is the template method every provider plugs into:
PreCallAsync never mutates your request: the public entry creates a detached execution copy (CreateExecutionCopy, which stamps Source lineage back to your canonical request) and invokes PreCallCoreAsync — the provider's extension point — on that copy. AIReturn.Request therefore carries the wire-bound copy; reach your original via result.Request.Source.
Key behaviors to know:
- Validation is declarative.
IsValid()resolves a capable model, requires a non-empty body, requires a schema whenJsonOutputis asserted, and requiresFunctionCallingwhen tools are enabled.Endpointis not required here — providers resolve it during preparation. Problems are reported asSHRuntimeMessages with codes likeNoCapableModel,UnknownModel,CapabilityMismatch— not thrown exceptions. - Model resolution is capability-driven. Leave
Modelempty and the request picks the provider's default model for the effectiveCapability. Request a model that lacks the needed capability and it is swapped (with aCapabilityMismatchwarning) or rejected. - Tools don't loop here. If the model answers with
AIInteractionToolCalls,Execreturns them inAIReturn.Body— executing them and re-querying is the session's job (next section). - Streaming uses
Exec(stream: true, …)when the provider exposes a streaming adapter; incremental chunks are coalesced into stable interactions keyed byIAIKeyedInteraction.
AIReturn: everything that came back
AIReturn (AICall/Core/Returns/) is deliberately rich — it's what powers metrics badges, error badges, and debugging:
Body— the new interactions (assistant text, tool calls, runtime messages).Raw— the provider's verbatim response payload, preserved for debugging.Metrics—AIMetrics/AIMetricsListwith token counts, finish reason, timing, cost.Status—AICallStatus:Idle → Processing → (Streaming | CallingTools) → Finished.Messages— structuredSHRuntimeMessagediagnostics (severity + code).CreateSuccess(body, request)/CreateError(message)— helpers to produce well-formed returns; tools use the same pattern.
ConversationSession: the multi-turn layer
AIRequestCall.Exec() is one turn. The moment tools or multi-step reasoning are involved, use ConversationSession (src/SmartHopper.Infrastructure/AICall/), which owns:
- The tool loop — detect
AIInteractionToolCalls, run each throughAIToolCall.Exec(), appendAIInteractionToolResults, re-query until the model produces a stable answer (or limits hit). - Streaming orchestration — provider streaming when supported, graceful fallback otherwise.
- Observers and cancellation — UI progress reporting,
CancellationTokenplumbing, undo of partial state. RunToStableResult/RunAsync— the entry points consumers use for tool-enabled flows.
Use AIRequestCall.Exec() only when you genuinely want a single stateless exchange (a test, a one-shot transform). If the request can produce tool calls — or a conversation continues across turns — go through ConversationSession. Re-implementing a tool loop in a component or tool is a bug magnet: orchestration belongs in one place.
AIToolCall: one tool execution
AIToolCall is the request type for executing a tool the model asked for. Tool providers receive one in their execute delegate:
private async Task<AIReturn> MyToolAsync(AIToolCall toolCall)
{
var output = new AIReturn { Request = toolCall };
var call = toolCall.GetToolCall(); // AIInteractionToolCall
var args = call.GetArgumentsOrEmpty(); // JObject of arguments
// … do the work, honoring toolCall.CancellationToken …
var body = AIBodyBuilder.Create()
.AddToolResult(result, call.Id, call.Name)
.Build();
output.CreateSuccess(body, toolCall);
return output;
}
Note the shape: tools return an AIReturn whose body contains an AIInteractionToolResult carrying the tool's Id and Name — that's how the session matches results to pending calls. Tools never touch the provider; they just build result interactions.
PolicyPipeline: cross-cutting checks
PolicyPipeline wraps calls with request/response policies — the seam where the host enforces things like trust verification, capability gating, and instrumentation without each provider re-implementing them. AIRequestCall.Exec consults the trust host and can short-circuit with a failed AIReturn before any wire traffic happens (e.g. "Provider trust check failed").
Why this design matters to you
- Writing a provider? You only implement
Encode/Decode/PreCallCoreAsync/CallApiand a streaming adapter — model resolution, tool loops, context injection, and metrics plumbing are already done. See Writing a provider. - Writing a tool? You receive an
AIToolCall, return anAIReturn. Schemas, consent, MCP exposure, and serialization are handled. See AI tools. - Writing a component? You declare which tools you need via
UsingAiToolsand get a managed session for free — you never assemble requests by hand unless you want to. See Components.