Skip to main content

The AI call pipeline

Everything SmartHopper sends to an AI provider — a component run, an assistant reply, an MCP tool call — flows through one normalized request/response pipeline called AICall. It lives in src/SmartHopper.ProviderSdk/AICall/ (the types) and src/SmartHopper.Infrastructure/AICall/ (the orchestration). Read this page before writing a provider, tool, or AI-powered component: it defines the vocabulary all of them share.

The idea​

Different AI vendors speak different wire protocols, but every SmartHopper feature needs the same things: messages, images, tool calls and results, JSON output constraints, metrics, and errors. AICall provides that shared format.

A request is an AIBody — a list of interactions — plus per-request metadata (provider, model, capability, filters). A response is an AIReturn — the returned interactions, raw payload, metrics, status, and diagnostics. Providers translate between this neutral format and their vendor's API; consumers never see wire JSON.

Interactions: the message currency​

IAIInteraction is the unit of conversation. The concrete types are in ProviderSdk/AICall/Core/Interactions/:

TypeCarriesTypical use
AIInteractionTextText plus metrics (tokens, reasoning)User prompts, assistant replies, system prompts
AIInteractionImageImage data/URLVision inputs, generated images
AIInteractionAudioAudio dataSpeech in/out
AIInteractionToolCallId, Name, Arguments (JSON)A tool the assistant wants executed
AIInteractionToolResultId, Name, result payloadThe outcome, fed back to the model
AIInteractionRuntimeMessageSeverity + messageWarnings/errors surfaced to the user

Every interaction has an Agent from AIAgent: System, User, Assistant, Context, ToolCall, or ToolResult. Interactions are immutable-style types — helpers like AIBodyBuilder assemble them rather than mutating in place.

AIBody: the request payload​

AIBody wraps the interaction list with three filters that change what the provider sees:

MemberDefaultEffect
Interactions—The conversation list. Non-mutating injection point: when ContextFilter yields context, a synthesized AIInteractionText with Agent = Context is prepended to the returned copy — the stored list is unchanged.
ContextFiltervariesWhich context providers inject facts. "*" all, "-*" none, "time, environment" a whitelist, "*, -selection" an exclusion list.
ToolFilter-* (none)Which tools the model may call. "*" enables all; glob patterns like "gh_*" enable subsets. Setting this also makes the request require FunctionCalling capability.
JsonOutputSchemanoneJSON schema for structured output; requests asserting JsonOutput without a schema fail validation.

Build bodies with AIBodyBuilder rather than constructing lists by hand:

var body = AIBodyBuilder.Create()
.AddSystem("You are a Grasshopper assistant.")
.AddUser("List the selected components.")
.WithContextFilter("time, environment, file, -viewport")
.WithToolFilter("gh_get, gh_list_categories")
.Build();

AIRequestCall: one turn​

AIRequestCall (ProviderSdk/AICall/Core/Requests/) is a single provider round-trip. Set Provider, Model, Body, Origin, then call Exec():

var request = new AIRequestCall
{
Provider = "MistralAI",
Model = "mistral-large-latest",
Body = body,
Origin = "my_feature", // semantic label — who made this call
};

AIReturn result = await request.Exec(cancellationToken);

Two related fields carry deliberately different meanings:

  • Origin — the semantic provenance label you set: tool name, component tag, chat surface. It survives preparation untouched and drives attribution/logging.
  • Endpoint — the wire route the provider resolves during preparation (e.g. /chat/completions). Callers leave it unset; a non-empty value is an explicit wire-path override for backoffice calls (/models, batch status).

Under the hood, Exec runs the full lifecycle — this is the template method every provider plugs into:

PreCallAsync never mutates your request: the public entry creates a detached execution copy (CreateExecutionCopy, which stamps Source lineage back to your canonical request) and invokes PreCallCoreAsync — the provider's extension point — on that copy. AIReturn.Request therefore carries the wire-bound copy; reach your original via result.Request.Source.

Key behaviors to know:

  • Validation is declarative. IsValid() resolves a capable model, requires a non-empty body, requires a schema when JsonOutput is asserted, and requires FunctionCalling when tools are enabled. Endpoint is not required here — providers resolve it during preparation. Problems are reported as SHRuntimeMessages with codes like NoCapableModel, UnknownModel, CapabilityMismatch — not thrown exceptions.
  • Model resolution is capability-driven. Leave Model empty and the request picks the provider's default model for the effective Capability. Request a model that lacks the needed capability and it is swapped (with a CapabilityMismatch warning) or rejected.
  • Tools don't loop here. If the model answers with AIInteractionToolCalls, Exec returns them in AIReturn.Body — executing them and re-querying is the session's job (next section).
  • Streaming uses Exec(stream: true, …) when the provider exposes a streaming adapter; incremental chunks are coalesced into stable interactions keyed by IAIKeyedInteraction.

AIReturn: everything that came back​

AIReturn (AICall/Core/Returns/) is deliberately rich — it's what powers metrics badges, error badges, and debugging:

  • Body — the new interactions (assistant text, tool calls, runtime messages).
  • Raw — the provider's verbatim response payload, preserved for debugging.
  • Metrics — AIMetrics/AIMetricsList with token counts, finish reason, timing, cost.
  • Status — AICallStatus: Idle → Processing → (Streaming | CallingTools) → Finished.
  • Messages — structured SHRuntimeMessage diagnostics (severity + code).
  • CreateSuccess(body, request) / CreateError(message) — helpers to produce well-formed returns; tools use the same pattern.

ConversationSession: the multi-turn layer​

AIRequestCall.Exec() is one turn. The moment tools or multi-step reasoning are involved, use ConversationSession (src/SmartHopper.Infrastructure/AICall/), which owns:

  • The tool loop — detect AIInteractionToolCalls, run each through AIToolCall.Exec(), append AIInteractionToolResults, re-query until the model produces a stable answer (or limits hit).
  • Streaming orchestration — provider streaming when supported, graceful fallback otherwise.
  • Observers and cancellation — UI progress reporting, CancellationToken plumbing, undo of partial state.
  • RunToStableResult / RunAsync — the entry points consumers use for tool-enabled flows.
Rule of thumb

Use AIRequestCall.Exec() only when you genuinely want a single stateless exchange (a test, a one-shot transform). If the request can produce tool calls — or a conversation continues across turns — go through ConversationSession. Re-implementing a tool loop in a component or tool is a bug magnet: orchestration belongs in one place.

AIToolCall: one tool execution​

AIToolCall is the request type for executing a tool the model asked for. Tool providers receive one in their execute delegate:

private async Task<AIReturn> MyToolAsync(AIToolCall toolCall)
{
var output = new AIReturn { Request = toolCall };
var call = toolCall.GetToolCall(); // AIInteractionToolCall
var args = call.GetArgumentsOrEmpty(); // JObject of arguments
// … do the work, honoring toolCall.CancellationToken …
var body = AIBodyBuilder.Create()
.AddToolResult(result, call.Id, call.Name)
.Build();
output.CreateSuccess(body, toolCall);
return output;
}

Note the shape: tools return an AIReturn whose body contains an AIInteractionToolResult carrying the tool's Id and Name — that's how the session matches results to pending calls. Tools never touch the provider; they just build result interactions.

PolicyPipeline: cross-cutting checks​

PolicyPipeline wraps calls with request/response policies — the seam where the host enforces things like trust verification, capability gating, and instrumentation without each provider re-implementing them. AIRequestCall.Exec consults the trust host and can short-circuit with a failed AIReturn before any wire traffic happens (e.g. "Provider trust check failed").

Why this design matters to you​

  • Writing a provider? You only implement Encode/Decode/PreCallCoreAsync/CallApi and a streaming adapter — model resolution, tool loops, context injection, and metrics plumbing are already done. See Writing a provider.
  • Writing a tool? You receive an AIToolCall, return an AIReturn. Schemas, consent, MCP exposure, and serialization are handled. See AI tools.
  • Writing a component? You declare which tools you need via UsingAiTools and get a managed session for free — you never assemble requests by hand unless you want to. See Components.