Provider Integration Guide

Overview

Yabanin abstracts upstream LLM APIs behind the provider.Provider interface (pkg/provider/provider.go). Every provider supports five operations:

Method	Purpose
Name()	Canonical identifier ("openai", "anthropic", "custom")
Path(stream bool)	HTTP endpoint path (/v1/chat/completions or /v1/messages)
PrepareRequest(ctx, baseURL, path, body, apiKey, stream)	Sets URL, headers (Auth, version, beta), content-type
TranslateRequest(openaiBody)	OpenAI JSON → provider-native schema
TranslateResponse(providerBody)	Provider-native JSON → OpenAI ChatCompletion shape
ParseUsage(respBody)	Extracts prompt/completion/cache tokens
StreamHandler	Real-time SSE translation (via pkg/provider/stream.go)

The adapter factory (pkg/provider/factory.go) resolves provider names at runtime via New(name). The upstream pool (pkg/upstream/pool.go) creates one *Client per (provider, URL) pair, each carrying its own adapter.

Z.AI GLM Coding Plan (zai-coding)

OpenAI-compatible Coding Plan adapter pinned to https://api.z.ai/api/coding/paas/v4. Enable with YABANIN_ZAI_CODING_ENABLED=true. Register models such as glm-5.3 with provider: zai-coding; upstream_url is ignored so quota cannot leak to a PAYG or arbitrary host. Feature flag is off by default. OpenCode example: examples/opencode.json.

---

Adding a new provider

1. Create the adapter — pkg/provider/<name>.go implementing Provider with all 6 methods.
2. Register in factory — add a case in New(name) → returns your adapter.
3. Update DB constraint — run a migration adding the new type to chk_provider.
4. Add to admin UI — the models CRUD already accepts a provider field; no code change needed.
5. Write tests — TranslateRequest, TranslateResponse, ParseUsage, StreamHandler for every event type.
6. Integration test — end-to-end via pkg/testutil/mock<name> (see below).
7. Add pricing — extend pkg/pricing/pricing.go table with "provider:model" keys.

---

Translation patterns

System message handling

Provider	System location
OpenAI	Inside messages[] as {role:"system", content:"..."}
Anthropic	Top-level system field (string or array with cache_control)
Gemini	systemInstruction: {parts:[...]} in request body

Tool format

Provider	Schema field	Function declaration
OpenAI	parameters (JSON Schema)	function inside tools[]
Anthropic	input_schema (JSON Schema)	input_schema + name/description in tools[]
Gemini	N/A	functionDeclarations[] in contents

Streaming event mapping

Provider	Native Event	OpenAI chunk equivalent
Anthropic	message_start	{choices:[{delta:{role:"assistant", content:""}}]}
Anthropic	content_block_delta (text)	{choices:[{delta:{content:"..."}}]}
Anthropic	message_stop	data: [DONE]\n\n
Anthropic	error	error event with message

---

Testing checklist for new provider

• TranslateRequest covers: system, multi-turn conversation, tool use, structured content, cache_control blocks
• TranslateResponse: single choice, multi-choice (if supported), usage section
• ParseUsage: prompt_tokens, completion_tokens, total, cache_read, cache_creation — all accounted
• StreamHandler: every native event type translated to OpenAI SSE chunks + [DONE] termination
• Integration test with mock server (pkg/testutil/mock<name>) — end-to-end 200 response through gateway
• Integration error mapping: 4xx errors → proper OpenAI error shape with error.message and error.type
• Health probe: the provider's /health or /v1/models endpoint is a valid health-check target
• Pricing table updated in pkg/pricing/pricing.go with current per-million-token rates

---

Mock server pattern

Use pkg/testutil/mockopenai or pkg/testutil/mockanthropic for integration tests:

srv := mockopenai.New()
ts := httptest.NewServer(srv)
defer ts.Close()

// Configure response
srv.SetResponse(200, `{"id":"test","choices":[...]}`)

// Or simulate rate-limiting then success
srv.FailFirstN(2, 429, "rate limited")
srv.SetResponse(200, `{"id":"ok","choices":[...]}`)

// For streaming endpoints
srv.SetStreamMode(true)
srv.SetResponse(200, `{"delta":{"content":"Hello"}}\n\n`)

// Verify what was sent
recs := srv.Requests() // []Request with Method, URL, Header, Body, Path
assert.Equal(t, "my-key", recs[0].Header.Get("Authorization"))

The same pattern applies to mockanthropic — use MessagesJSON() or MessagesJSONWithCache() for response bodies.

---

Prompt caching

For providers that support prompt caching (Anthropic):

• The adapter detects "cache_control" keys in message content blocks
• When present, the anthropic-beta: prompt-caching-2024-07-31 header is automatically added by PrepareRequest
• Cache tokens are parsed in ParseUsage and billed separately via ComputeCost in pkg/pricing/