Skip to main content

Models and capabilities

XyzProviderModels.cs declares which models your provider supports and what each can do. This metadata drives almost everything the host does with models: the default model per task, capability validation (a request needing FunctionCalling won't silently land on a text-only model), the model pickers in component badges, pricing/cost estimates, and batch eligibility.

The contract​

AIProviderModels has one required method and one optional one:

public class XyzProviderModels : AIProviderModels
{
public XyzProviderModels(XyzProvider provider) : base(provider) { }

// Required — the curated catalog of models + capability records.
public override Task<List<AIModelCapabilities>> RetrieveModels() { … }

// Optional — live model names fetched from your API (e.g. GET /models).
// Used to refresh/discover models beyond the curated list.
public override Task<List<string>> RetrieveApiModels() { … }
}

Wire it into your provider's constructor so the base class can populate the registry:

private XyzProvider()
{
this.Models = new XyzProviderModels(this);
}

The host calls RetrieveModels() during provider initialization and registers the results in AIModelCapabilityRegistry — the central model/capability index every consumer queries. Your provider class never touches the registry directly.

AIModelCapabilities: one record per model​

Each entry is an AIModelCapabilities record. A realistic entry (adapted from DeepSeekProviderModels):

new AIModelCapabilities
{
Provider = this.Provider.Name.ToLowerInvariant(), // must match the provider name, lowercased
Model = "xyz-large",
Capabilities = AICapability.TextInput
| AICapability.TextOutput
| AICapability.FunctionCalling
| AICapability.JsonOutput,

// Composite capability groups this model is the DEFAULT for.
// The host picks this model when a request needs one of these combos.
Default = AICapability.Text2Text
| AICapability.ToolChat
| AICapability.Text2Json,

SupportsStreaming = true,
SupportsBatch = true,
Verified = false,
Rank = 9000, // higher = preferred among equals
ContextLimit = 128_000,
Created = new DateTime(2026, 3, 1),
Aliases = new List<string> { "xyz-large-2026" },
Pricing = new AIModelPricing
{
Prompt = 0.000002m, // cost per input token
Completion = 0.000006m, // cost per output token
InputCacheRead = 0.0000005m,
},
BatchPricing = new AIModelPricing { /* half-price batch rates */ },
}

Key fields:

FieldWhy it matters
CapabilitiesRaw flags — which modalities the model has.
DefaultWhich composite capabilities this model is the default for. Only the default model serves a capability request with no explicit model.
RankTie-breaker when several models qualify — higher wins.
DeprecatedStill listed, but flagged; don't give deprecated models Default.
VerifiedMarks entries confirmed against the live API.
SupportsStreaming / SupportsBatch / SupportsPromptCachingFeature gates used by the pipeline.
ContextLimit, Pricing, BatchPricingCost estimates, context-window warnings.
AliasesAlternate names that resolve to this model.
DiscouragedForToolsTools this model handles poorly — used for warnings/fallback.

AICapability flags​

Capabilities are composable flags (SmartHopper.ProviderSdk.AIModels.AICapability):

PrimitiveMeaning
TextInput, ImageInput, AudioInput, SpeechInput, VideoInputAccepted input modalities
TextOutput, ImageOutput, AudioOutput, SpeechOutput, VideoOutputProduced output modalities
JsonOutputStructured/JSON output
FunctionCallingCan emit tool calls
ReasoningExposes chain-of-thought / thinking
EmbedOutputEmbeddings

Predefined composites for common request shapes:

CompositeExpands toTypical consumer
Text2TextTextInput | TextOutputplain chat/transforms
ToolChatText2Text | FunctionCallingassistant turns with tools
ReasoningChatText2Text | Reasoningreasoning models
ToolReasoningChatText2Text | Reasoning | FunctionCallingreasoning + tools
Text2JsonTextInput | JsonOutputstructured extraction
Text2ImageTextInput | ImageOutputimage generation
Text2Speech, Speech2Text, Audio2Text, Text2Audiospeech/audio pairsaudio pipelines
Image2Text, Image2Imageimage in/out pairsvision

A request's required capability is matched against these flags — e.g. enabling tools on a request makes it require FunctionCalling, and a model without that flag is skipped by selection or flagged with CapabilityMismatch.

How the pieces connect​

  • RetrieveModels = your curated, capability-annotated catalog. This is the source of truth for defaults and validation — even if the live API lists more models, unannotated ones can't be capability-checked.
  • RetrieveApiModels = optional live discovery (e.g. Ollama's local /api/tags). Leave it returning an empty list if your API has no model-listing endpoint.

Checklist​

  • One AIModelCapabilities per supported model, Provider lowercased to match Name.
  • Exactly the models you want auto-selected have Default set for the right composites.
  • Capabilities reflect reality — don't claim FunctionCalling/JsonOutput the model can't do; the pipeline trusts these flags.
  • this.Models = new XyzProviderModels(this); in the provider constructor.
  • Pricing/context kept current — stale numbers mislead cost badges and context warnings.