Go LLM Interactive Proxy

Go LLM Interactive Proxy (LIP) is a streaming-first control plane for LLM traffic. It sits between AI clients and provider backends so operators can keep client integrations stable while changing routing, provider mix, resilience behavior, observability, and extension policy at the proxy layer.

The standard distribution, cmd/lipstd, serves bundled HTTP frontends, routes through canonical lipapi requests and event streams, and wires the official backends and feature plugins through explicit registration.

What it does

• Multi-protocol frontends - OpenAI Responses, legacy OpenAI-compatible chat, Anthropic Messages, and Gemini generateContent-compatible HTTP surfaces.
• Backend flexibility - hosted provider adapters, OpenAI-compatible/local runtimes, agent-specific backends, custom-compatible backend rows, and a no-key localstub backend for dogfood.
• Canonical translation - frontend and backend adapters translate through one protocol-neutral request model and event stream; no pairwise protocol translators.
• Core-owned routing - ordered failover, weighted routing, parallel races, TTFT budgets, model aliases, route diagnostics, and circuit-breaker eligibility live in the core.
• Continuity and recovery - B2BUA-style A-leg/B-leg lineage records recoverable pre-output attempts, while post-output failures are surfaced instead of silently retried.
• Operator hardening - typed config, auth/access modes, secure sessions, diagnostics secrets, pprof controls, Prometheus metrics, OpenTelemetry tracing, access logs, and resource limits.
• Extension platform - feature bundles use pkg/lipsdk facades for request shaping, tools, completion gates, workspace/state, traffic observation, auxiliary calls, and compatibility hooks.
• Accounting authority SDK - pkg/lipsdk/controlplane and pkg/lipsdk/policydecision expose safe accounting authority DTOs, bounded queries, and policy-compatible accounting evidence for operator-facing control-plane integrations. Usage and authority evidence carry additive dual-plane fields (perspective, boundary, lifecycle, provenance, fact kind, surfaced, version refs) while preserving legacy observed/accounting projections. Metering and CP queries return too-broad/unsupported outcomes instead of scanning. Independent customer/operator/compression/routing-overhead report input contracts (with explicit calculation types) and aggregate protected-traffic readiness are available; readiness is wired live, while DualPlaneReportInputs remain construction/calculator contracts rather than a dedicated live HTTP report assembler. AccountingDecisionRow carries released/overage/adjustment settlement deltas (additive, ,omitzero; zero for reserve decisions) so decision history explains per-mutation outcomes.
• Dual-plane economics contracts - pkg/lipsdk/metering, pkg/lipsdk/economics, and pkg/lipsdk/authority define provider-neutral facts, money/rating/exposure, and request/attempt authority ports (import DAG: authority → economics → metering). economics.Rater rates quantities to money; additive economics.OutputLimitQuoter converts monetary spend caps to enforceable output-token bounds for admission clamps. When a rater is injected, runtime uses that economics contract exclusively for admit spend, usage enrichment, and clamp conversion — never a silent AccountingPriceCatalog fallback. Runtime captures immutable metering checkpoints at frontend-ingress (before submit hooks), backend-ingress (before Open), and egress settlement. Optional durable journal (metering.enabled, memory/SQLite/Postgres via internal/infra/metering/journalstore) wires Executor.MeteringRecorder; default off leaves the recorder nil. Durable authority stores no longer hold a process-wide mutex across DB I/O (lifecycle/readiness only); lip_authority_stage_* Prometheus metrics cover admit evaluation/cleanup budgets. Injected enterprise authority providers isolate panics/malformed decisions through stable fail-closed posture. Feature release gates are listed in docs/release-gates.md (dual-plane section).
• Public production facade - pkg/lipruntime lets a closed Go module build the standard distribution with injected metering, rating, authority, concurrency, snapshot, evidence, metering-query, and observer implementations without importing internal/ packages. Injectable snapshot sources are published at Build and can be republished for new admissions via Runtime.RefreshSnapshots (immutable generations; in-flight pins preserved; source errors expose degraded/unavailable posture without unrelated version fallback). cmd/lipstd remains the OSS CLI composition root.

Standard distribution

Exact registration is code-owned by internal/standardplugins/standard_table.go; mandatory distribution subset is in pkg/lipsdk/standard_bundle.go.

Surface	Bundled support
Frontends	openai-responses, openai-legacy, anthropic, gemini
Hosted/provider backends	openai-responses, openai-legacy, anthropic, gemini, bedrock, acp, openrouter, nvidia, huggingface, openai-codex, opencode-go, opencode-zen
Local / compatible backends	ollama, ollama-cloud, llamacpp, lmstudio, vllm, localstub, custom OpenAI/Anthropic-compatible backend kinds
Local-agent / experimental	cursorcliacp and experimental cursorsdk (Node bridge over @cursor/sdk 1.0.23; see docs/cursor-sdk-backend.md)
Feature plugins	no-op compatibility hooks plus reference/proof plugins for submit, parts, tools, workspace guard, traffic transcript, verifier, pre-request policy, auto-append, and Codex client compatibility; standard distro also default-enables canonical tool-call-repair (ADR 0007; opt out with enabled: false)

Quick start

Start with the no-key local stub path when you want to validate config, routing, inventory, and HTTP serving without hosted provider credentials:

go run ./cmd/lipstd check-config --config ./config/examples/dogfood-local-stub.yaml
go run ./cmd/lipstd routes --config ./config/examples/dogfood-local-stub.yaml
go run ./cmd/lipstd inventory --config ./config/examples/dogfood-local-stub.yaml
go run ./cmd/lipstd serve --config ./config/examples/dogfood-local-stub.yaml

For hosted providers, use config/config.yaml as the sample and provide API keys through YAML or environment variables. standardplugins.ResolveUpstreamAPIKeysFromEnv resolves the supported provider env vars and numbered variants once at startup; see internal/standardplugins/keys.go for the exact names and numbering rules.

go run ./cmd/lipstd --config ./config/config.yaml

lipstd accepts --config before or after the subcommand; if it appears more than once, the later value wins. See docs/dogfood-local.md for the full local dogfood flow. Truncated tool-call repair can be exercised with config/examples/dogfood-tool-call-repair.yaml (see ADR docs/adr/0007-canonical-tool-call-repair.md).

Configuration and operations

• Config - Runtime config is typed and loaded from YAML. config/config.yaml documents access/auth templates, server timeouts, logging, diagnostics, observability, routing, continuity, identity (A-leg Server / B-leg User-Agent and OpenRouter attribution), and provider rows. See docs/proxy-identity.md. config/config.multi-instance.example.yaml shows multiple backend instances of the same adapter.
• Routing - Default selectors come from routing.default_route or the first enabled backend plus registry default model ids. model_aliases rewrite full selector strings before parsing. Route selectors support ordered failover, weights, first-request annotations, parallel ! races, per-leg [handicap=N], global/per-leg TTFT budgets, and per-leaf query generation parameters. Route query parameters such as ?reasoning_effort=xhigh and ?verbosity=high are explicit routing directives: when present, they override matching per-request body/canonical generation options; absent parameters leave request values unchanged.
• OpenAI Codex verbosity bumps - The openai-codex backend defaults to text.verbosity=high for the first 5 turns of each conversation, and then again on every 10th turn by default, when no explicit per-request verbosity is set. Opt out with early_session_verbosity_bump_disabled: true and/or mid_session_verbosity_bump_disabled: true, or tune with early_session_verbosity_bump_turns / mid_session_verbosity_bump_frequency. When the mid-session bump is disabled, the cadence value is ignored. See docs/openai-codex-backend.md and docs/openai-codex-backend.md.
• Experimental Cursor SDK - Optional local-only cursorsdk backend beside cursorcliacp. Install the packaged Node bridge manually (exact @cursor/sdk 1.0.23, Node ≥ 22.13); Go-LIP never runs npm. Use explicit cursorsdk:… routes, separate SDK API-key billing (CURSOR_API_KEY / api_key), and sandbox/settings defaults documented in docs/cursor-sdk-backend.md. Schema example: config/examples/cursor-sdk-experimental.yaml. Offline ACP-vs-SDK matrix: make test-cursor-sdk-comparison-report (methodology).
• Continuity - continuity.store: memory is the default. continuity.store: sqlite with continuity.sqlite_path persists A-leg rows and attempt lineage through internal/core/continuity/sqlitestore. In-memory ttl and max_legs tuning does not apply to SQLite.
• Security - Multi-user or non-loopback deployments need explicit auth/access posture. Local API keys must be at least 16 Unicode code points after trimming. Diagnostics, pprof, metrics, model-catalog diagnostics, and secure-session summaries require a shared secret when exposed beyond loopback. On Unix, OpenAI Codex auth.json and managed-OAuth account files must be 0600 (group/other-readable files are now rejected at load); symlinked managed-OAuth account files are skipped. See docs/openai-codex-backend.md. Optional secrets guard (plugins.features id secrets-guard, disabled by default) scans model-bound ingress for loaded secret values only, including JSON object keys and scalar tokens. It does not scan responses/egress or transformed forms; log leaves ingress unchanged, and JSON redact blocks unsupported key/scalar tokens with a normal block decision so quarantine still applies. Multi-user matching uses only the current request credential and safe attribution identifiers. Only one enabled secrets-guard feature instance is supported per deployment, and rollout is staged as disabled -> log -> redact -> block, one action per deployment. See docs/secrets-guard.md and config/examples/secrets-guard-*.yaml.
• Observability - Optional Prometheus metrics and OpenTelemetry tracing are configured under observability. Access logs use bounded-cardinality route groups by default; raw paths are opt-in.
• HTTP clients - The shared upstream client honors HTTP_PROXY / HTTPS_PROXY by default. Set http_client.trust_environment_proxy: false when process environment is not trusted.
• Resource bounds - lipapi.Call.Validate, lipapi.Collect limits, pending wire event caps (max_pending_wire_events; 0 = unlimited), B2BUA store caps, and shared frontend decode admission (max_concurrent_decodes default 32, max_inflight_decode_bytes default 64 MiB) protect memory and request size boundaries. Absolute decompressed body oversize is 413; temporary decode admission saturation is 429 + Retry-After: 1. Admission runs after body ReadAll (bytes already resident) and covers protocol Decode only. Raise body and inflight budgets together for large multimodal / long-context. See EchoesVault/pages/decode-qos-admission.md.

More detail: docs/proxy-identity.md, docs/secrets-guard.md, docs/database-persistence.md, docs/routing-health-circuit-breaker.md, docs/execerr-classification.md, docs/extension-platform-authoring.md, docs/performance-checks.md, docs/release-gates.md, and docs/cursor-sdk-backend.md.

Developer workflow

make quality-checks        # gofmt drift, go mod tidy drift, build, vet, guard scripts, archtest
make arch-report           # advisory architecture metrics report (non-failing Markdown output)
make test                  # quality-checks + unit tests + parity-checks
make test-unit             # go test -parallel=8 -timeout=10m ./...
make test-precommit-extra  # precommit-tagged hygiene + executor matrices
make test-fast             # quality-checks + staged-package tests, or all when none staged
make parity-checks         # conformance package with -tags=precommit,integration
make test-fuzz             # short fuzz smoke over release-gate fuzz targets
make test-race             # skipped on Windows; strict race runs in nightly CI on Linux
make bench                 # benchmark smoke for hot packages
make pgo-profile           # collect default.pgo from core benches (optional; move under cmd/lipstd)
make pgo-build             # build cmd/lipstd (auto-applies cmd/lipstd/default.pgo when present)
make qa                    # quality-checks + one full tagged test pass + lint + govulncheck
make hooks-install         # install optional pre-commit hooks

PR CI (.github/workflows/qa.yml) runs when the PR changes any *.go file: make quality-checks, PostgreSQL authority proofs, go test -parallel=8 -tags=precommit,integration ./..., golangci-lint v2, and go tool govulncheck ./.... PRs with no *.go changes skip the suite; the required qa gate still reports success. Nightly CI (.github/workflows/race-fuzz-nightly.yml, also workflow_dispatch) runs strict Linux race and Tier-1 fuzz smoke (FUZZTIME=6s). Locally, make lint prefers go tool golangci-lint (pinned in go.mod tool) and falls back to a PATH install. A monthly modernization workflow (.github/workflows/modernize-monthly.yml) re-runs the modernize linter suite and govulncheck. Linter config lives in .golangci.yml.

Recoverability is defined by the specification bundle: tests, testdata/ goldens, stable pkg/lipapi / pkg/lipsdk contracts, steering, and parity/scenario docs. Start at docs/spec-bundle-index.md, docs/conformance-golden-coverage.md, and docs/conformance-matrix-evidence.md.

Repository layout

• cmd/lipstd/ - standard distribution command and wiring tests.
• pkg/lipapi/ - canonical request, event, capability, validation, and error contracts.
• pkg/lipsdk/ - stable plugin SDK contracts and standard distribution requirements.
  • Compatibility note: FrontendMountOptions gained an optional DecodeAdmission field. Use named composite literals; unkeyed literals that previously listed every field in order will not compile.
• internal/core/ - runtime orchestration, routing, continuity, secure sessions, hooks/extensions, stream handling, policy, accounting, config, admin, diagnostics, and safety.
• internal/plugins/ - bundled frontend, backend, feature, compatibility, and protocol-helper packages.
• internal/standardplugins/ - standard bundle registration tables, per-backend factory helpers, and InstallStandardBundleOn.
• internal/featurebundle/ - feature merge surface (MergeFeatureSurface over SDK hook slices).
• internal/pluginreg/ - explicit per-composition-root registry and backend factory helpers.
• internal/infra/runtimebundle/ and internal/stdhttp/ - runtime assembly (executor, hook bus, stores) and HTTP mounting/serving.
• internal/infra/ - logging, HTTP client tuning, metrics, tracing, DB, model catalog/registry, routing health, tokenization/accounting, and auth-event plumbing.
• internal/refbackend/, internal/refclient/, internal/testkit/ - emulators, reference clients, fixtures, stubs, and conformance helpers for tests.
• internal/archtest/, internal/qa/, scripts/, .githooks/, .github/workflows/ - guardrails and quality automation.
• docs/, .kiro/, testdata/, config/ - operator docs, steering/spec artifacts, fixtures, and sample configs.

Relationship to Python LIP

This repository is the Go implementation of LIP with a smaller core and explicit plugin/SDK boundaries. The sibling Python project remains useful historical context and migration reference, but Go documentation should describe only behavior implemented in this repo unless a doc explicitly says a feature is Python-era or future migration work.