From 7658459a5e2496017dcbedd955fd0063bf22d7cb Mon Sep 17 00:00:00 2001 From: Him188 Date: Tue, 18 Aug 2026 14:54:01 +0900 Subject: [PATCH] docs: simplify prose Replace em dashes with plain sentence structure and remove redundant filler sentences across the docs. Co-Authored-By: Claude Fable 5 --- .mintlify/skills/fish-audio-api/SKILL.md | 42 ++++---- .mintlify/skills/fish-audio-sdk/SKILL.md | 22 ++--- .../fish-audio-sdk/references/errors.md | 10 +- .../fish-audio-sdk/references/installation.md | 4 +- .../references/speech-to-text.md | 12 +-- .../references/text-to-speech.md | 14 +-- .../references/voice-cloning.md | 10 +- .../fish-audio-sdk/references/websocket.md | 10 +- agents/build/client-tools.mdx | 20 ++-- agents/build/configuration.mdx | 36 +++---- agents/build/dynamic-variables.mdx | 14 +-- agents/build/knowledge-base.mdx | 32 +++--- agents/build/system-tools.mdx | 8 +- agents/build/time-timezone.mdx | 20 ++-- agents/build/tools.mdx | 26 ++--- agents/build/voice-language.mdx | 26 ++--- agents/build/webhook-tools.mdx | 48 ++++----- agents/concepts.mdx | 22 ++--- agents/deploy/authenticated-sessions.mdx | 52 +++++----- agents/deploy/overview.mdx | 28 +++--- agents/deploy/protocol.mdx | 34 +++---- agents/deploy/public-agents.mdx | 40 ++++---- agents/deploy/react-sdk.mdx | 16 +-- agents/deploy/versions-publishing.mdx | 29 +++--- agents/deploy/web-sdk.mdx | 66 ++++++------- agents/deploy/widget.mdx | 24 ++--- agents/monitor/conversation-history.mdx | 40 ++++---- agents/monitor/post-call-analysis.mdx | 44 ++++----- agents/monitor/webhooks.mdx | 97 ++++++++++++------- agents/overview.mdx | 12 +-- agents/quickstart.mdx | 26 ++--- agents/telephony/inbound-calls.mdx | 16 +-- agents/telephony/phone-numbers.mdx | 31 +++--- agents/telephony/transfers.mdx | 38 ++++---- agents/test/agent-tests.mdx | 22 ++--- agents/test/preview-calls.mdx | 22 ++--- api-reference/agent-errors.mdx | 32 +++--- .../endpoint/openapi-v1/speech-to-text.mdx | 2 +- api-reference/errors.mdx | 12 +-- api-reference/openapi.json | 46 ++++----- api-reference/sdk/python/overview.mdx | 2 +- developer-guide/compat/capabilities.mdx | 58 +++++------ .../compat/migrate-from-elevenlabs.mdx | 97 +++++++++---------- developer-guide/compat/migrate-from-groq.mdx | 16 +-- .../compat/migrate-from-openai.mdx | 41 ++++---- .../compat/migrate-from-openrouter.mdx | 30 +++--- developer-guide/compat/realtime-protocol.mdx | 37 +++---- developer-guide/core-features/emotions.mdx | 4 +- developer-guide/getting-started/api-key.mdx | 6 +- developer-guide/getting-started/migration.mdx | 48 ++++----- developer-guide/integrations/n8n.mdx | 8 +- developer-guide/integrations/telnyx.mdx | 4 +- .../models-pricing/models-overview.mdx | 6 +- .../resources/agent-quickstart.mdx | 24 ++--- developer-guide/resources/coding-agents.mdx | 8 +- .../batch-transcribe-with-language-hint.mdx | 5 +- .../cookbook/clone-and-wait-until-ready.mdx | 2 +- .../cookbook/discover-library-voice.mdx | 4 +- .../cookbook/instant-voice-cloning.mdx | 4 +- .../oneshot-vs-persistent-cloning.mdx | 8 +- .../cookbook/realtime-llm-to-speech.mdx | 4 +- .../sdk-guide/cookbook/streaming-to-file.mdx | 2 +- .../cookbook/telephony-8khz-audio.mdx | 4 +- .../cookbook/transcribe-to-captions.mdx | 6 +- .../sdk-guide/cookbook/voice-agent-loop.mdx | 4 +- developer-guide/sdk-guide/python/errors.mdx | 22 ++--- developer-guide/sdk-guide/quickstart.mdx | 4 +- developer-guide/self-hosting/air-gapped.mdx | 12 +-- developer-guide/self-hosting/all-in-one.mdx | 16 +-- .../self-hosting/enterprise-releases.mdx | 2 +- developer-guide/self-hosting/kubernetes.mdx | 14 +-- developer-guide/self-hosting/operations.mdx | 2 +- .../self-hosting/registry-access.mdx | 10 +- developer-guide/self-hosting/requirements.mdx | 2 +- features/manage-voices.mdx | 4 +- features/realtime-streaming.mdx | 24 ++--- features/speech-to-text.mdx | 18 ++-- features/text-to-speech.mdx | 36 +++---- features/voice-cloning.mdx | 12 +-- overview/capabilities.mdx | 18 ++-- overview/mcp.mdx | 12 +-- overview/platform.mdx | 6 +- snippets/compat-callout.mdx | 2 +- tests/cookbooks/README.md | 22 ++--- 84 files changed, 897 insertions(+), 878 deletions(-) diff --git a/.mintlify/skills/fish-audio-api/SKILL.md b/.mintlify/skills/fish-audio-api/SKILL.md index d023927..50109fe 100644 --- a/.mintlify/skills/fish-audio-api/SKILL.md +++ b/.mintlify/skills/fish-audio-api/SKILL.md @@ -19,7 +19,7 @@ This file condenses those into rules an agent can apply directly. - Auth (all endpoints): `Authorization: Bearer ` - Optional distributed tracing for inference APIs: see `https://docs.fish.audio/api-reference/observability`. - Get API keys: `https://fish.audio/app/api-keys` -- Never hardcode keys — read from an env var like `FISH_API_KEY`. +- Never hardcode keys. Read from an env var like `FISH_API_KEY`. - Errors are JSON `{status, message}` for 401 / 402 / 404, and an array of `{loc, type, msg, ctx, in}` for 422 (validation). ## Endpoint map @@ -38,7 +38,7 @@ This file condenses those into rules an agent can apply directly. | GET | `/wallet/{user_id}/api-credit` | API credit balance (`user_id` defaults to `self`) | | WSS | `/v1/tts/live` | Real-time TTS streaming (MessagePack frames) | -## Text-to-Speech — `POST /v1/tts` +## Text-to-Speech: `POST /v1/tts` Required headers: @@ -47,7 +47,7 @@ Required headers: Optional headers: -- `model` — values: `s1`, `s2-pro`, `s2.1-pro`, `s2.1-pro-free`. If omitted or unrecognized, the server falls back to `s2.1-pro` (paid). Default to `s2.1-pro` for production; use `s2.1-pro-free` for free-tier evaluation and prototyping (same model, no TTFA/DPA guarantees). +- `model`: values `s1`, `s2-pro`, `s2.1-pro`, `s2.1-pro-free`. If omitted or unrecognized, the server falls back to `s2.1-pro` (paid). Default to `s2.1-pro` for production; use `s2.1-pro-free` for free-tier evaluation and prototyping (same model, no TTFA/DPA guarantees). Response: streaming audio bytes (`Transfer-Encoding: chunked`) in the format set by `format`. Write to a file or pipe to a player. There is **no JSON wrapper** on success. @@ -197,7 +197,7 @@ if (!res.ok) throw new Error(`${res.status} ${await res.text()}`); await pipeline(Readable.fromWeb(res.body), createWriteStream("out.mp3")); ``` -## Speech-to-Text — `POST /v1/asr` +## Speech-to-Text: `POST /v1/asr` Required headers: `Authorization`. Content type: `multipart/form-data` or `application/msgpack`. @@ -205,7 +205,7 @@ Form fields: - `audio` (binary, required) - `language` (string, optional; omit to auto-detect) -- `ignore_timestamps` (bool, default `true`; set `false` to get per-segment timestamps — adds latency on clips < 30 s) +- `ignore_timestamps` (bool, default `true`; set `false` to get per-segment timestamps, which adds latency on clips < 30 s) Response (200): @@ -244,7 +244,7 @@ r.raise_for_status() print(r.json()["text"]) ``` -## Voice Design — `POST /v1/voice-design` +## Voice Design: `POST /v1/voice-design` Required headers: @@ -315,11 +315,11 @@ with open("voice.wav", "wb") as f: Billing: one successful generation request is charged once, even when it returns multiple candidates. Authentication, validation, balance, concurrency, and service errors are not billed. -## Voice models — `/model` +## Voice models: `/model` ### List: `GET /model` -Query params: `page_size` (default 10), `page_number` (default 1), `title`, `tag` (string or array), `self` (bool — only your models), `author_id`, `language`, `title_language`, `sort_by` (`score` | `task_count` | `created_at`, default `score`). +Query params: `page_size` (default 10), `page_number` (default 1), `title`, `tag` (string or array), `self` (bool; only your models), `author_id`, `language`, `title_language`, `sort_by` (`score` | `task_count` | `created_at`, default `score`). Returns `{total, items: ModelEntity[]}`. @@ -349,7 +349,7 @@ Returns 201 with the full `ModelEntity` including `_id`, `state` (`created` | `t ### Get / Update / Delete - `GET /model/{id}` → `ModelEntity` -- `PATCH /model/{id}` — JSON, form-urlencoded, multipart, or msgpack. Nullable fields: `title`, `description`, `cover_image` (binary), `visibility`, `tags`. +- `PATCH /model/{id}`: JSON, form-urlencoded, multipart, or msgpack. Nullable fields: `title`, `description`, `cover_image` (binary), `visibility`, `tags`. - `DELETE /model/{id}` → 200 on success. ```bash @@ -362,32 +362,32 @@ curl --request PATCH https://api.fish.audio/model/ \ ## Wallet - `GET /wallet/self/package` → `{user_id, type, total, balance, created_at, updated_at, finished_at}` -- `GET /wallet/self/api-credit` → `{_id, user_id, credit, created_at, updated_at, has_phone_sha256, has_free_credit}`. Pass `?check_free_credit=true` to also populate `has_free_credit` (default `false` — the field is `null` when not checked). +- `GET /wallet/self/api-credit` → `{_id, user_id, credit, created_at, updated_at, has_phone_sha256, has_free_credit}`. Pass `?check_free_credit=true` to also populate `has_free_credit` (default `false`; the field is `null` when not checked). Replace `self` with a specific `user_id` if you have permission; otherwise always use `self`. -## WebSocket TTS — `wss://api.fish.audio/v1/tts/live` +## WebSocket TTS: `wss://api.fish.audio/v1/tts/live` For low-latency / streaming TTS (e.g. LLM token stream → speech). All frames are **MessagePack-encoded** binary messages. ### Connection headers - `Authorization: Bearer ` -- `model` — optional; same values and fallback behavior as `POST /v1/tts` (falls back to `s2.1-pro` when omitted or unrecognized) +- `model`: optional; same values and fallback behavior as `POST /v1/tts` (falls back to `s2.1-pro` when omitted or unrecognized) ### Event sequence Client → server: -1. `StartEvent` — once, first message: `{event: "start", request: }`. The `request` object is the same schema as `POST /v1/tts` above. Usually `request.text = ""` and the real text streams in `TextEvent`s. -2. `TextEvent` — one per text chunk: `{event: "text", text: "..."}`. Send as many as needed. -3. `FlushEvent` — optional: `{event: "flush"}`. Forces the server to synthesize buffered text immediately (use for turn-taking / low-latency flushes). -4. `CloseEvent` — final: `{event: "stop"}`. **Note the literal is `stop`, not `close`.** +1. `StartEvent` (once, first message): `{event: "start", request: }`. The `request` object is the same schema as `POST /v1/tts` above. Usually `request.text = ""` and the real text streams in `TextEvent`s. +2. `TextEvent` (one per text chunk): `{event: "text", text: "..."}`. Send as many as needed. +3. `FlushEvent` (optional): `{event: "flush"}`. Forces the server to synthesize buffered text immediately (use for turn-taking / low-latency flushes). +4. `CloseEvent` (final): `{event: "stop"}`. **Note the literal is `stop`, not `close`.** Server → client: -- `AudioEvent`: `{event: "audio", audio: }` — many of these, concatenate in order to reconstruct the audio stream in the format set by `request.format`. -- `FinishEvent`: `{event: "finish", reason: "stop" | "error"}` — exactly one, then the server closes the socket. Ignore unknown events for forward compatibility. +- `AudioEvent`: `{event: "audio", audio: }`. Many of these; concatenate in order to reconstruct the audio stream in the format set by `request.format`. +- `FinishEvent`: `{event: "finish", reason: "stop" | "error"}`. Exactly one, then the server closes the socket. Ignore unknown events for forward compatibility. ### Python example (`websockets>=14` + `msgpack`) @@ -489,11 +489,11 @@ ws.on("message", (buf) => { ## Emotion / expression control -The S1 model uses `(parenthesis)` tags inside `text`, e.g. `(happy) What a day!`. S2-Pro uses free-form `[bracket]` natural-language tags, e.g. `[slightly sarcastic, rising tone]`. Either works through `text` — no separate parameter. Full list: `https://docs.fish.audio/api-reference/emotion-reference.md`. +The S1 model uses `(parenthesis)` tags inside `text`, e.g. `(happy) What a day!`. S2-Pro uses free-form `[bracket]` natural-language tags, e.g. `[slightly sarcastic, rising tone]`. Either works through `text`; there is no separate parameter. Full list: `https://docs.fish.audio/api-reference/emotion-reference.md`. ## Encoding and content-type rules -- Use `application/json` for normal TTS requests — it's the simplest and works for `reference_id` flows. +- Use `application/json` for normal TTS requests. It's the simplest and works for `reference_id` flows. - Use `application/msgpack` when you need to send raw audio bytes inline (inline `references`, or the WebSocket protocol). - Use `multipart/form-data` for `/v1/asr` and `POST /model` because they upload files. - All WebSocket frames are MessagePack binary, regardless of inner payload. @@ -508,7 +508,7 @@ The S1 model uses `(parenthesis)` tags inside `text`, e.g. `(happy) What a day!` - `references` sent with `Content-Type: application/json` (must be msgpack). - Numeric param out of range (`temperature`, `top_p`, `chunk_length`, `min_chunk_length`, `prosody.speed`, `early_stop_threshold`). - `mp3_bitrate` / `opus_bitrate` set without matching `format`. -- WebSocket: a `finish` event with `reason: "error"` means the server failed mid-stream — surface the message and reconnect rather than retrying on the same socket. +- WebSocket: a `finish` event with `reason: "error"` means the server failed mid-stream. Surface the message and reconnect rather than retrying on the same socket. ## Decision shortcuts diff --git a/.mintlify/skills/fish-audio-sdk/SKILL.md b/.mintlify/skills/fish-audio-sdk/SKILL.md index d0123b8..493a61d 100644 --- a/.mintlify/skills/fish-audio-sdk/SKILL.md +++ b/.mintlify/skills/fish-audio-sdk/SKILL.md @@ -1,14 +1,14 @@ --- name: fish-audio-sdk -description: Write code with the official Fish Audio SDKs — Python (`fishaudio`, PyPI `fish-audio-sdk`) and JavaScript/TypeScript (`fish-audio`). Use when the user wants text-to-speech, speech-to-text, voice cloning / voice-model management, or realtime WebSocket TTS through the installed SDK rather than raw HTTP. Covers install and auth, sync + async Python, the TypeScript client, exact method signatures and defaults, model selection (including the S2.1 typing caveat), the real exception types, and the Python↔JavaScript naming differences. For raw REST/WebSocket calls without an SDK (curl, unsupported languages, edge runtimes), use the `fish-audio-api` skill instead. +description: Write code with the official Fish Audio SDKs, Python (`fishaudio`, PyPI `fish-audio-sdk`) and JavaScript/TypeScript (`fish-audio`). Use when the user wants text-to-speech, speech-to-text, voice cloning / voice-model management, or realtime WebSocket TTS through the installed SDK rather than raw HTTP. Covers install and auth, sync + async Python, the TypeScript client, exact method signatures and defaults, model selection (including the S2.1 typing caveat), the real exception types, and the Python↔JavaScript naming differences. For raw REST/WebSocket calls without an SDK (curl, unsupported languages, edge runtimes), use the `fish-audio-api` skill instead. --- # Fish Audio SDK Skill Use this skill to generate correct, runnable code with the **official Fish Audio SDKs**: -- **Python** — package `fish-audio-sdk` on PyPI, imported as `fishaudio`. (The same wheel still ships a separate legacy `fish_audio_sdk` package — do **not** mix them; everything here is the modern `fishaudio` package.) -- **JavaScript / TypeScript** — package `fish-audio` on npm, imported as `FishAudioClient`. +- **Python**: package `fish-audio-sdk` on PyPI, imported as `fishaudio`. (The same wheel still ships a separate legacy `fish_audio_sdk` package. Do **not** mix them; everything here is the modern `fishaudio` package.) +- **JavaScript / TypeScript**: package `fish-audio` on npm, imported as `FishAudioClient`. If the user wants raw `curl` / HTTP / WebSocket without installing an SDK, use the **`fish-audio-api`** skill instead. @@ -16,13 +16,13 @@ If the user wants raw `curl` / HTTP / WebSocket without installing an SDK, use t ## Global facts -- **Auth:** both SDKs read the API key from the `FISH_API_KEY` environment variable automatically. Get keys at `https://fish.audio/app/api-keys`. Never hardcode a key — read it from the environment. +- **Auth:** both SDKs read the API key from the `FISH_API_KEY` environment variable automatically. Get keys at `https://fish.audio/app/api-keys`. Never hardcode a key. - **Base URL:** `https://api.fish.audio` (override with `base_url=` in Python / `baseUrl:` in JS). -- **Models:** the API supports `s1`, `s2-pro`, `s2.1-pro` (recommended for production), and `s2.1-pro-free` (free tier), but the SDK type definitions currently list only `s1` and `s2-pro` (`s2-pro` = SDK default). Both SDKs forward the model value without runtime validation, so `"s2.1-pro"` works over the wire — static type checkers will flag it, so add `# type: ignore` (Python) / an `as` cast (TS), or use the `fish-audio-api` skill for raw calls. `speech-1.5` / `speech-1.6` are **deprecated**. In Python pass `model="s2-pro"` (keyword); in JS pass the **positional** `backend` argument. +- **Models:** the API supports `s1`, `s2-pro`, `s2.1-pro` (recommended for production), and `s2.1-pro-free` (free tier), but the SDK type definitions currently list only `s1` and `s2-pro` (`s2-pro` = SDK default). Both SDKs forward the model value without runtime validation, so `"s2.1-pro"` works over the wire. Static type checkers will flag it, so add `# type: ignore` (Python) / an `as` cast (TS), or use the `fish-audio-api` skill for raw calls. `speech-1.5` / `speech-1.6` are **deprecated**. In Python pass `model="s2-pro"` (keyword); in JS pass the **positional** `backend` argument. - **Audio formats:** `mp3` (default), `wav`, `pcm`, `opus`. -- **Playback in examples:** `play()` shells out to a system audio tool — Python uses **ffmpeg/ffplay** (or `mpv`), JS uses **ffplay**. It is for local/desktop use; in a server, `save()` to a file or stream the bytes instead. See [references/installation.md](references/installation.md). +- **Playback in examples:** `play()` shells out to a system audio tool: Python uses **ffmpeg/ffplay** (or `mpv`), JS uses **ffplay**. It is for local/desktop use; in a server, `save()` to a file or stream the bytes instead. See [references/installation.md](references/installation.md). -## Quick start — Python +## Quick start: Python ```python from fishaudio import FishAudio @@ -37,7 +37,7 @@ save(audio, "output.mp3") # write to a file # play(audio) # or play locally (needs ffmpeg) ``` -Async — identical resource tree on `AsyncFishAudio`, used as a context manager: +Async: identical resource tree on `AsyncFishAudio`, used as a context manager: ```python import asyncio @@ -52,7 +52,7 @@ async def main(): asyncio.run(main()) ``` -## Quick start — JavaScript / TypeScript +## Quick start: JavaScript / TypeScript ```ts import { FishAudioClient, play } from "fish-audio"; @@ -113,7 +113,7 @@ The two SDKs do **not** use the same names. Use this map when porting code betwe ## Gotchas (verified against the SDK source) -- Python `latency` accepts only **`"normal"` or `"balanced"`** (default `"balanced"`) — there is no `"low"`. +- Python `latency` accepts only **`"normal"` or `"balanced"`** (default `"balanced"`); there is no `"low"`. - The Python client has **no `max_retries`** and does **not** auto-retry; the JS client **does** auto-retry (configurable via per-call `requestOptions.maxRetries`). See [errors](references/errors.md). -- Python defines a `ValidationError` class but **never raises it** — don't catch it expecting validation failures; a 422 surfaces as `APIError`. The JS SDK throws `UnprocessableEntityError` on 422. +- Python defines a `ValidationError` class but **never raises it**, so don't catch it expecting validation failures; a 422 surfaces as `APIError`. The JS SDK throws `UnprocessableEntityError` on 422. - ASR segment `start` / `end` are in **seconds**, but `duration` is in **milliseconds**. See [speech-to-text](references/speech-to-text.md). diff --git a/.mintlify/skills/fish-audio-sdk/references/errors.md b/.mintlify/skills/fish-audio-sdk/references/errors.md index b5b15bd..c788ff0 100644 --- a/.mintlify/skills/fish-audio-sdk/references/errors.md +++ b/.mintlify/skills/fish-audio-sdk/references/errors.md @@ -1,6 +1,6 @@ # Errors, Retries & Timeouts -The two SDKs have **different** exception models. The tables below reflect what the SDK source actually raises — not every exported class is thrown. +The two SDKs have **different** exception models. The tables below reflect what the SDK source actually raises; not every exported class is thrown. ## Python exceptions @@ -9,9 +9,9 @@ Hierarchy (all subclasses of `FishAudioError`): | Exception | When | Attributes | | --------------------- | ---------------------------------------------- | --------------------------------- | | `APIError` | base for HTTP errors | `.status`, `.message`, `.body` | -| `AuthenticationError` | 401 — bad/missing key | (APIError) | +| `AuthenticationError` | 401 (bad/missing key) | (APIError) | | `PermissionError` | 403 | (APIError) | -| `NotFoundError` | 404 — voice id not found | (APIError) | +| `NotFoundError` | 404 (voice id not found) | (APIError) | | `RateLimitError` | 429 | (APIError) | | `ServerError` | 5xx | (APIError) | | `WebSocketError` | realtime stream failed | — | @@ -48,7 +48,7 @@ except FishAudioError as e: - **No automatic retries.** The Python client makes a single request and raises on failure. Implement your own retry loop if you need one (e.g. back off on `RateLimitError`). - **Timeout** is set on the client: `FishAudio(timeout=240.0)` (seconds, default 240). -- `RequestOptions(max_retries=...)` exists but is currently a **no-op** — don't rely on it. `RequestOptions(timeout=..., additional_headers=...)` does work per request: +- `RequestOptions(max_retries=...)` exists but is currently a **no-op**, so don't rely on it. `RequestOptions(timeout=..., additional_headers=...)` does work per request: ```python from fishaudio.core.request_options import RequestOptions @@ -92,7 +92,7 @@ What the JS client actually throws: | Error | When | | ----------------------------------------------------- | -------------------------------------------------------------------------------------------- | -| `UnprocessableEntityError` (extends `FishAudioError`) | 422 — the **only** typed HTTP subclass thrown; `.body` is `{ detail: [{ loc, msg, type }] }` | +| `UnprocessableEntityError` (extends `FishAudioError`) | 422, the **only** typed HTTP subclass thrown; `.body` is `{ detail: [{ loc, msg, type }] }` | | `FishAudioError` | every other non-2xx response; read `.statusCode`, `.body`, `.rawResponse` | | `FishAudioTimeoutError` | request exceeded the timeout | diff --git a/.mintlify/skills/fish-audio-sdk/references/installation.md b/.mintlify/skills/fish-audio-sdk/references/installation.md index 1c1459d..ff05bb4 100644 --- a/.mintlify/skills/fish-audio-sdk/references/installation.md +++ b/.mintlify/skills/fish-audio-sdk/references/installation.md @@ -52,7 +52,7 @@ Never hardcode a key in source. If neither the argument nor `FISH_API_KEY` is se | Request timeout | `timeout=240.0` (seconds) | per-call `requestOptions.timeoutInSeconds` | | Custom HTTP client | `httpx_client=` | (not exposed) | -> Python caveat: if you pass your own `httpx_client`, the SDK uses it **as-is** — your `base_url`, `timeout`, and the `Authorization` header are **not** applied to it. Pre-configure those on the client you inject. +> Python caveat: if you pass your own `httpx_client`, the SDK uses it **as-is**. Your `base_url`, `timeout`, and the `Authorization` header are **not** applied to it. Pre-configure those on the client you inject. There is no client-level `max_retries` or `default_headers` option in Python. Per-request headers go through `request_options`. See [errors.md](errors.md) for retry/timeout behavior. @@ -72,7 +72,7 @@ brew install ffmpeg sudo apt-get install ffmpeg ``` -In a server or browser context, don't use `play()` — use `save()` (Python) or write/stream the bytes yourself. +In a server or browser context, don't use `play()`. Use `save()` (Python) or write/stream the bytes yourself. ## Verify a key works diff --git a/.mintlify/skills/fish-audio-sdk/references/speech-to-text.md b/.mintlify/skills/fish-audio-sdk/references/speech-to-text.md index d7769f3..c62e401 100644 --- a/.mintlify/skills/fish-audio-sdk/references/speech-to-text.md +++ b/.mintlify/skills/fish-audio-sdk/references/speech-to-text.md @@ -1,6 +1,6 @@ # Speech-to-Text (ASR) -## Python — `client.asr.transcribe` +## Python: `client.asr.transcribe` ```python from fishaudio import FishAudio @@ -28,18 +28,18 @@ Keyword params: ### Response shape (`ASRResponse`) ```python -result.text # str — full transcript -result.duration # float — total audio duration in MILLISECONDS +result.text # str: full transcript +result.duration # float: total audio duration in MILLISECONDS result.segments # list[ASRSegment] # each segment: seg.text # str -seg.start # float — seconds -seg.end # float — seconds +seg.start # float: seconds +seg.end # float: seconds ``` > **Unit gotcha (verified in source):** segment `start` / `end` are in **seconds**, but `duration` is in **milliseconds**. Don't assume they share a unit. -## JavaScript — `client.speechToText.convert` +## JavaScript: `client.speechToText.convert` ```ts import { FishAudioClient } from "fish-audio"; diff --git a/.mintlify/skills/fish-audio-sdk/references/text-to-speech.md b/.mintlify/skills/fish-audio-sdk/references/text-to-speech.md index 58cf2e1..a3e9910 100644 --- a/.mintlify/skills/fish-audio-sdk/references/text-to-speech.md +++ b/.mintlify/skills/fish-audio-sdk/references/text-to-speech.md @@ -1,6 +1,6 @@ # Text-to-Speech -## Python — `client.tts` +## Python: `client.tts` `convert()` returns the **complete audio as `bytes`**. `stream()` returns an iterable of byte chunks. @@ -32,13 +32,13 @@ All keyword-only: | ----------------- | ----------------------------------- | ------------- | ---------------------------------------------------------------- | | `text` | `str` | — (required) | Text to synthesize. | | `reference_id` | `str \| None` | `None` | Voice model id to speak with. | -| `references` | `list[ReferenceAudio] \| None` | `None` | Inline clone samples — see [voice-cloning.md](voice-cloning.md). | +| `references` | `list[ReferenceAudio] \| None` | `None` | Inline clone samples; see [voice-cloning.md](voice-cloning.md). | | `format` | `"mp3" \| "wav" \| "pcm" \| "opus"` | `"mp3"` | Output format. | | `latency` | `"normal" \| "balanced"` | `"balanced"` | `normal` = higher quality, `balanced` = faster. (No `"low"`.) | | `speed` | `float` | — | Shortcut for prosody speed (0.5–2.0). | | `config` | `TTSConfig` | `TTSConfig()` | Reusable bundle of the settings below. | -| `model` | `"s2-pro" \| "s1"` | `"s2-pro"` | Synthesis model per current SDK typing. The API also accepts `"s2.1-pro"` / `"s2.1-pro-free"` — forwarded fine at runtime, but add `# type: ignore` for static checkers. `speech-1.5` / `speech-1.6` are deprecated. | -| `request_options` | `RequestOptions \| None` | `None` | Per-request timeout / headers — see [errors.md](errors.md). | +| `model` | `"s2-pro" \| "s1"` | `"s2-pro"` | Synthesis model per current SDK typing. The API also accepts `"s2.1-pro"` / `"s2.1-pro-free"`, forwarded fine at runtime, but add `# type: ignore` for static checkers. `speech-1.5` / `speech-1.6` are deprecated. | +| `request_options` | `RequestOptions \| None` | `None` | Per-request timeout / headers; see [errors.md](errors.md). | Direct params (`reference_id`, `format`, `latency`, `speed`) override the matching field on `config` when set. @@ -76,9 +76,9 @@ audio = client.tts.stream(text="Hello!").collect() Async: every method mirrors onto `AsyncFishAudio`; `await client.tts.convert(...)`, and `client.tts.stream(...)` must be awaited before iterating with `async for`. -## JavaScript — `client.textToSpeech` +## JavaScript: `client.textToSpeech` -`convert(request, backend?, requestOptions?)` resolves to a `ReadableStream` you can `play()` or pipe to a file. `backend` is the **second positional** argument (default `"s2-pro"`) — **not** a named option. +`convert(request, backend?, requestOptions?)` resolves to a `ReadableStream` you can `play()` or pipe to a file. `backend` is the **second positional** argument (default `"s2-pro"`), **not** a named option. ```ts import { FishAudioClient, play } from "fish-audio"; @@ -116,4 +116,4 @@ await new Promise((resolve, reject) => ## Model & expression notes - `s2-pro` is the default and highest quality; `s1` is the previous generation. -- Emotion/expression is controlled inline in `text` (S1 uses `(parenthesis)` tags, S2-Pro uses free-form `[bracket]` tags) — there is no separate SDK parameter. Full tag list: `https://docs.fish.audio/api-reference/emotion-reference`. +- Emotion/expression is controlled inline in `text` (S1 uses `(parenthesis)` tags, S2-Pro uses free-form `[bracket]` tags); there is no separate SDK parameter. Full tag list: `https://docs.fish.audio/api-reference/emotion-reference`. diff --git a/.mintlify/skills/fish-audio-sdk/references/voice-cloning.md b/.mintlify/skills/fish-audio-sdk/references/voice-cloning.md index a56903b..c21c195 100644 --- a/.mintlify/skills/fish-audio-sdk/references/voice-cloning.md +++ b/.mintlify/skills/fish-audio-sdk/references/voice-cloning.md @@ -2,8 +2,8 @@ Two ways to use a custom voice: -1. **Instant (zero-shot)** — pass reference audio inline on each `convert` call. Nothing is saved. -2. **Persistent voice model** — create a model once, then reuse its `id` as `reference_id`. +1. **Instant (zero-shot)**: pass reference audio inline on each `convert` call. Nothing is saved. +2. **Persistent voice model**: create a model once, then reuse its `id` as `reference_id`. ## 1. Instant cloning (inline references) @@ -51,7 +51,7 @@ In JS, `ReferenceAudio.audio` is a `File`. ## 2. Persistent voice models -### Create — Python `voices.create` +### Create: Python `voices.create` ```python with open("sample1.wav", "rb") as f1, open("sample2.wav", "rb") as f2: @@ -69,7 +69,7 @@ print(voice.id, voice.state) # state: created | training | trained | failed `voices.create` keyword params: `title` (required), `voices: list[bytes]` (required), `description`, `texts`, `tags`, `cover_image: bytes`, `visibility="private"`, `train_mode="fast"`, `enhance_audio_quality=True`. -### Create — JavaScript `voices.ivc.create` +### Create: JavaScript `voices.ivc.create` ```ts import { readFile } from "node:fs/promises"; @@ -108,6 +108,6 @@ const audio = await client.textToSpeech.convert({ | Update | `client.voices.update(voice_id, title=..., visibility=...)` | `client.voices.update(voiceId, { title, visibility })` | | Delete | `client.voices.delete(voice_id)` | `client.voices.delete(voiceId)` | -Python `voices.list` is manually paged: `page_size` (default 10), `page_number` (default 1), plus filters `title`, `tags`, `self_only`, `author_id`, `language`, `title_language`, and `sort_by` (`"task_count"` default, or `"created_at"`). There is no auto-pager — loop `page_number` yourself. +Python `voices.list` is manually paged: `page_size` (default 10), `page_number` (default 1), plus filters `title`, `tags`, `self_only`, `author_id`, `language`, `title_language`, and `sort_by` (`"task_count"` default, or `"created_at"`). There is no auto-pager; loop `page_number` yourself. A model is usable as a `reference_id` once its `state` is `"trained"`. States: `created → training → trained` (or `failed`). diff --git a/.mintlify/skills/fish-audio-sdk/references/websocket.md b/.mintlify/skills/fish-audio-sdk/references/websocket.md index 470af14..e7e4ff5 100644 --- a/.mintlify/skills/fish-audio-sdk/references/websocket.md +++ b/.mintlify/skills/fish-audio-sdk/references/websocket.md @@ -1,8 +1,8 @@ # Realtime WebSocket TTS -Stream text in and get audio out as it's generated — ideal for piping an LLM's token stream to speech. +Stream text in and get audio out as it's generated, ideal for piping an LLM's token stream to speech. -## Python — `client.tts.stream_websocket` +## Python: `client.tts.stream_websocket` The first argument is an **iterable of text chunks** (plain `str`, or `TextEvent` / `FlushEvent` for fine control). The sync method returns an `Iterator[bytes]`; the async method must be awaited and returns an `AsyncIterator[bytes]`. @@ -60,9 +60,9 @@ def events(): yield TextEvent(text="Second sentence.") ``` -The SDK sends the start/stop frames for you — you only supply text/flush. +The SDK sends the start/stop frames for you; you only supply text/flush. -## JavaScript — `client.textToSpeech.convertRealtime` +## JavaScript: `client.textToSpeech.convertRealtime` Returns a `RealtimeConnection`; subscribe to events with `RealtimeEvents`. Set `request.text` to `""` and stream the real text via the second argument. @@ -103,5 +103,5 @@ connection.on(RealtimeEvents.CLOSE, async () => { ## Protocol notes -- The close frame's event literal is **`"stop"`**, not `"close"` (handled for you by both SDKs; relevant only if you drop to raw frames — use the `fish-audio-api` skill for that). +- The close frame's event literal is **`"stop"`**, not `"close"` (handled for you by both SDKs; relevant only if you drop to raw frames; use the `fish-audio-api` skill for that). - A realtime run that fails mid-stream surfaces as `WebSocketError` (Python) / an `ERROR` event (JS). Reconnect rather than retrying on the same socket. diff --git a/agents/build/client-tools.mdx b/agents/build/client-tools.mdx index 938aa85..15b7b7f 100644 --- a/agents/build/client-tools.mdx +++ b/agents/build/client-tools.mdx @@ -1,10 +1,10 @@ --- title: "Client Tools" -description: "Let the agent trigger actions in your own app — declared on the agent, executed by your code through the SDK" +description: "Let the agent trigger actions in your own app: declared on the agent, executed by your code through the SDK" icon: "code" --- -Client tools run inside your application, not on Fish Audio's servers. You declare the tool on the agent — name, description, parameters — and register a handler in your app with the [Web SDK](/agents/deploy/web-sdk). When the agent decides to call the tool mid-conversation, the SDK invokes your handler and returns its result to the agent. +Client tools run inside your application, not on Fish Audio's servers. You declare the tool on the agent (name, description, parameters) and register a handler in your app with the [Web SDK](/agents/deploy/web-sdk). When the agent decides to call the tool mid-conversation, the SDK invokes your handler and returns its result to the agent. Use client tools for anything only your frontend can do: navigate to a page, highlight a product, open a modal, read app state, or hand off to a human. @@ -25,7 +25,7 @@ Client tools need code on your side. For tools that call an HTTP endpoint from F During the conversation, the agent invokes the tool with parameter values. The SDK dispatches the call to your handler. - The handler's return value is JSON-serialized and sent back, and the agent continues with the result — unless the tool is fire-and-forget. + The handler's return value is JSON-serialized and sent back, and the agent continues with the result, unless the tool is fire-and-forget. @@ -47,17 +47,17 @@ Client tools sit alongside webhook tools in the agent's tool list. A declaration | Field | Description | |---|---| -| `name` | How the model refers to the tool — must match the name you register in the SDK exactly | +| `name` | How the model refers to the tool. Must match the name you register in the SDK exactly | | `description` | Tells the model what the tool does and when to use it | | `arguments` | Named inputs the model fills in; delivered to your handler as one JSON object | | `expects_response` | Whether the agent waits for your handler's return value before continuing | ### Naming rules -Tool names must match `^[a-zA-Z][a-zA-Z0-9_-]{0,63}$` — start with a letter, then letters, digits, underscores, or hyphens, up to 64 characters total. +Tool names must match `^[a-zA-Z][a-zA-Z0-9_-]{0,63}$`: start with a letter, then letters, digits, underscores, or hyphens, up to 64 characters total. -- Built-in tools live under a leading underscore (like `_transfer_call`), which the pattern makes unreachable — your names never collide with them. -- Names must be unique per agent — declaring a duplicate fails with a `400`. +- Built-in tools live under a leading underscore (like `_transfer_call`), which the pattern makes unreachable. +- Names must be unique per agent. Declaring a duplicate fails with a `400`. ## Handle the call in your app @@ -109,17 +109,17 @@ Parameter values are produced by the model. Validate them in your handler before | Exceeds the timeout (default 15 s) | An error result is sent back automatically; code `tool_timeout` | | Tool called but no handler registered | An error result is sent back automatically | -The agent is never left hanging: while `expects_response` is `true` the agent suspends that tool call until your result arrives or the timeout fires, then continues either way. Errors and timeouts return to the model as tool errors, so the agent can recover in conversation ("I couldn't open that page — let me try something else"). To change the handler timeout, pass `clientToolTimeoutMs` (in milliseconds) when starting the session. Note there is a second, server-side deadline: the agent stops waiting after the tool's `timeout_seconds` (default 30 s, settable 1–120 on the tool), and a handler result arriving after that is ignored — so `clientToolTimeoutMs` can only tighten the client-side deadline, not extend the agent's wait. +The agent is never left hanging: while `expects_response` is `true` the agent suspends that tool call until your result arrives or the timeout fires, then continues either way. Errors and timeouts return to the model as tool errors, so the agent can recover in conversation ("I couldn't open that page. Let me try something else"). To change the handler timeout, pass `clientToolTimeoutMs` (in milliseconds) when starting the session. Note there is a second, server-side deadline: the agent stops waiting after the tool's `timeout_seconds` (default 30 s, settable 1–120 on the tool), and a handler result arriving after that is ignored, so `clientToolTimeoutMs` can only tighten the client-side deadline, not extend the agent's wait. ## Fire-and-forget tools -Set `expects_response` to `false` for tools that are pure side effects — the agent triggers your handler and keeps talking without waiting. Any return value is ignored. +Set `expects_response` to `false` for tools that are pure side effects: the agent triggers your handler and keeps talking without waiting. Any return value is ignored. This suits UI actions where confirmation adds nothing: scrolling, opening a panel, firing an analytics event. If the agent should react to the outcome ("the page is open, now walk the user through it"), keep `expects_response: true`. ## Observe tool activity -Every tool call — client and webhook alike — emits `toolCallStarted`, `toolCallCompleted`, and `toolCallFailed` events on the session, with the call ID, tool name, and payloads. The events are on by default; sessions created with `tool_events: false` don't emit them. Use them to render tool activity in your UI. See the [Web SDK](/agents/deploy/web-sdk) for the event reference. +Every tool call (client and webhook alike) emits `toolCallStarted`, `toolCallCompleted`, and `toolCallFailed` events on the session, with the call ID, tool name, and payloads. The events are on by default; sessions created with `tool_events: false` don't emit them. Use them to render tool activity in your UI. See the [Web SDK](/agents/deploy/web-sdk) for the event reference. ## Going further diff --git a/agents/build/configuration.mdx b/agents/build/configuration.mdx index 900d28e..6bf8eb9 100644 --- a/agents/build/configuration.mdx +++ b/agents/build/configuration.mdx @@ -1,21 +1,21 @@ --- title: "Configuration" -description: "Set your agent's system prompt, first message, and conversation behavior — in the Builder or through the API" +description: "Set your agent's system prompt, first message, and conversation behavior, in the Builder or through the API" icon: "sliders" --- -Configuration is the Builder's home tab: define how your agent behaves, how it opens each conversation, and how it paces its turns. Every edit is saved to the agent's **draft** automatically — callers only hear your changes after you [publish](/agents/deploy/versions-publishing). New to Agents? Start with the [Quickstart](/agents/quickstart). +Configuration is the Builder's home tab: define how your agent behaves, how it opens each conversation, and how it paces its turns. Every edit is saved to the agent's **draft** automatically. Callers only hear your changes after you [publish](/agents/deploy/versions-publishing). New to Agents? Start with the [Quickstart](/agents/quickstart). ## System prompt -The system prompt sets the personality, goals, and guardrails that steer every reply. It is capped at **4,000 tokens** — the API rejects longer prompts with `422 Unprocessable Entity`. Keeping it under **2,000 tokens** is recommended: shorter prompts cut latency and cost, and hold the model's attention better. It can also be replaced for a single session by sending `overrides.system_prompt` when [creating the session](/agents/deploy/authenticated-sessions#overrides). +The system prompt sets the personality, goals, and guardrails that steer every reply. It is capped at **4,000 tokens**. The API rejects longer prompts with `422 Unprocessable Entity`. Keeping it under **2,000 tokens** is recommended: shorter prompts cut latency and cost, and hold the model's attention better. It can also be replaced for a single session by sending `overrides.system_prompt` when [creating the session](/agents/deploy/authenticated-sessions#overrides). Keep the prompt focused: - State who the agent is and what it should accomplish, in a few sentences each. - Spell out guardrails explicitly ("If asked about pricing, direct the caller to sales"). -- Write for the ear — replies are spoken aloud, so ask for short, conversational answers. +- Write for the ear: replies are spoken aloud, so ask for short, conversational answers. ## First message @@ -40,17 +40,17 @@ The Voice panel selects the voice your agent speaks with (`voice_id`), its speak `conversation.eagerness` controls how quickly the agent starts talking after the caller stops: -- `relaxed` — waits longer, never talks over the caller. -- `balanced` (default) — waits for a natural pause. -- `eager` — jumps in quickly. +- `relaxed`: Waits longer, never talks over the caller. +- `balanced` (default): Waits for a natural pause. +- `eager`: Jumps in quickly. ### Interruptions -- `conversation.interruptible` (default `true`) — whether the caller can barge in while the agent is speaking. -- `conversation.interruption_sensitivity` — how much caller speech counts as an interruption: - - `low` — the agent stops less readily, talking through background noise and short acknowledgements. - - `balanced` (default) — interrupts on normal speech. - - `high` — the agent stops more readily when the caller speaks, even on brief utterances. +- `conversation.interruptible` (default `true`): Whether the caller can barge in while the agent is speaking. +- `conversation.interruption_sensitivity` sets how much caller speech counts as an interruption: + - `low`: The agent stops less readily, talking through background noise and short acknowledgements. + - `balanced` (default): Interrupts on normal speech. + - `high`: The agent stops more readily when the caller speaks, even on brief utterances. ### Call duration @@ -62,11 +62,11 @@ The Voice panel selects the voice your agent speaks with (`voice_id`), its speak ### Recording -`conversation.record_audio` (default `true`) — store per-speaker audio for [playback and download](/agents/monitor/conversation-history#what-gets-stored), with a per-session override on the [session request](/agents/deploy/authenticated-sessions). In the console this lives under your agent's **Settings**. Recording defaults to on — make sure callers are informed and consent where your jurisdiction requires it. +`conversation.record_audio` (default `true`) stores per-speaker audio for [playback and download](/agents/monitor/conversation-history#what-gets-stored), with a per-session override on the [session request](/agents/deploy/authenticated-sessions). In the console this lives under your agent's **Settings**. Make sure callers are informed and consent where your jurisdiction requires it. ### Timezone -`conversation.timezone` is the default IANA timezone (like `Asia/Shanghai`) the agent uses for dates and times in conversation. Leave it empty for **automatic** — each session follows the caller's device or phone number, falling back to UTC. Set one when your agent serves a single region regardless of who calls. A per-session `timezone` on the [session request](/agents/build/time-timezone) overrides this. See [Time & timezone](/agents/build/time-timezone) for the full resolution order. +`conversation.timezone` is the default IANA timezone (like `Asia/Shanghai`) the agent uses for dates and times in conversation. Leave it empty for **automatic**: each session follows the caller's device or phone number, falling back to UTC. Set one when your agent serves a single region regardless of who calls. A per-session `timezone` on the [session request](/agents/build/time-timezone) overrides this. See [Time & timezone](/agents/build/time-timezone) for the full resolution order. ## Autosave and publishing @@ -76,7 +76,7 @@ Drafts never affect live traffic: active integrations keep using the last publis ## Configure through the API -The same draft is readable and writable over REST — useful for provisioning agents from your own systems. +The same draft is readable and writable over REST, which is useful for provisioning agents from your own systems. ### Read the draft @@ -107,11 +107,11 @@ curl --request GET "https://api.fish.audio/v1/agent/agents/$AGENT_ID/config" \ } ``` -The response also includes the `tools`, `knowledge_base`, `analysis`, and `webhooks` sections, each covered on its own page. `config_hash` identifies this draft revision — it changes whenever an edit changes the draft's content, and a mismatch with the published version is what marks an agent as having unpublished changes. +The response also includes the `tools`, `knowledge_base`, `analysis`, and `webhooks` sections, each covered on its own page. `config_hash` identifies this draft revision; a mismatch with the published version is what marks an agent as having unpublished changes. ### Update sections -`PATCH` takes a partial body and deep-merges it into the draft: send only the sections and fields you want to change, and everything else keeps its value. Fields that hold a list — `webhooks.post_call` among them — are replaced as a whole, so send the complete list whenever you change one. +`PATCH` takes a partial body and deep-merges it into the draft: send only the sections and fields you want to change, and everything else keeps its value. Fields that hold a list (`webhooks.post_call` among them) are replaced as a whole, so send the complete list whenever you change one. ```bash curl --request PATCH "https://api.fish.audio/v1/agent/agents/$AGENT_ID/config" \ @@ -123,7 +123,7 @@ curl --request PATCH "https://api.fish.audio/v1/agent/agents/$AGENT_ID/config" \ }' ``` -The response includes the draft's new `config_hash`. Values outside the documented limits — a system prompt over 4,000 tokens, `max_duration_seconds` outside 60–3600 — are rejected with `422 Unprocessable Entity` and the draft is left unchanged. +The response includes the draft's new `config_hash`. Values outside the documented limits (a system prompt over 4,000 tokens, `max_duration_seconds` outside 60–3600) are rejected with `422 Unprocessable Entity` and the draft is left unchanged. ### Config sections diff --git a/agents/build/dynamic-variables.mdx b/agents/build/dynamic-variables.mdx index 9ed03a4..1fabc34 100644 --- a/agents/build/dynamic-variables.mdx +++ b/agents/build/dynamic-variables.mdx @@ -4,14 +4,14 @@ description: "Personalize each session with template variables supplied at creat icon: "brackets-curly" --- -An agent's published configuration is shared by every caller — dynamic variables personalize it per session. Write `{{variable_name}}` placeholders in your agent's system prompt or first message and supply values when you create the session. Substitution happens once, when the published configuration is assembled for the session. +An agent's published configuration is shared by every caller. Dynamic variables personalize it per session. Write `{{variable_name}}` placeholders in your agent's system prompt or first message and supply values when you create the session. Substitution happens once, when the published configuration is assembled for the session. ```text System prompt You are a support agent for {{company}}. The caller's name is {{customer_name}} and they are on the {{plan}} plan. Greet them by name. ``` -Pass values as a flat object of strings, numbers, or booleans. Variable names must match `[A-Za-z][A-Za-z0-9_]*` (no hyphens or dots), string values are capped at 1,000 characters, and a request can carry at most 50 variables — violations reject session creation with `422`: +Pass values as a flat object of strings, numbers, or booleans. Variable names must match `[A-Za-z][A-Za-z0-9_]*` (no hyphens or dots), string values are capped at 1,000 characters, and a request can carry at most 50 variables. Violations reject session creation with `422`: ```bash API (curl) @@ -40,7 +40,7 @@ const session = await AgentSession.start({ -Placeholders with no matching variable are **not** removed — the literal `{{variable_name}}` text stays in the prompt, visible to the model. Make sure every placeholder in your configuration has a value at session creation. +Placeholders with no matching variable are **not** removed: the literal `{{variable_name}}` text stays in the prompt, visible to the model. Make sure every placeholder in your configuration has a value at session creation. @@ -48,10 +48,10 @@ Session records never store dynamic variable values. -You don't need a variable for the current date or time — the agent already knows both, in the session's timezone. See [Time & timezone](/agents/build/time-timezone). +You don't need a variable for the current date or time. The agent already knows both, in the session's timezone. See [Time & timezone](/agents/build/time-timezone). -Variables fill placeholders in the configured text. To replace whole configuration fields for a session — the prompt itself, the opener, voice, language — use [overrides](/agents/deploy/authenticated-sessions#overrides) on the same request; `{{placeholders}}` render inside overridden text too. +Variables fill placeholders in the configured text. To replace whole configuration fields for a session (the prompt itself, the opener, voice, language), use [overrides](/agents/deploy/authenticated-sessions#overrides) on the same request; `{{placeholders}}` render inside overridden text too. ## Who supplies the values @@ -62,7 +62,7 @@ Where variable values come from depends on the session's [access mode](/agents/d | `agentId` ([public agents](/agents/deploy/public-agents)) | The SDK, directly from the browser | The browser, via `AgentSession.start()` options | | `sessionToken` (private agents) | Your backend, with your API key | Your backend, in its `POST /v1/agent/sessions` call | -In `sessionToken` mode the SDK's `dynamicVariables` option has no effect — the session already exists by the time the token reaches the browser. Attach personalization server-side instead: +In `sessionToken` mode the SDK's `dynamicVariables` option has no effect. The session already exists by the time the token reaches the browser. Attach personalization server-side instead: ```javascript Backend (sessionToken mode) // Your backend endpoint, called by your own frontend @@ -81,7 +81,7 @@ const sessionToken = await response.json(); // pass to AgentSession.start({ sess ``` -In `agentId` mode, values arrive from the end user's browser — treat them as untrusted input, and use `sessionToken` mode when personalization must come from data only your backend knows. +In `agentId` mode, values arrive from the end user's browser. Treat them as untrusted input, and use `sessionToken` mode when personalization must come from data only your backend knows. ## Going further diff --git a/agents/build/knowledge-base.mdx b/agents/build/knowledge-base.mdx index c4b7d3a..71a0678 100644 --- a/agents/build/knowledge-base.mdx +++ b/agents/build/knowledge-base.mdx @@ -1,21 +1,21 @@ --- title: "Knowledge Base" -description: "Ground your agent in your own documents — add a source once, attach it to any agent in your workspace" +description: "Ground your agent in your own documents: add a source once, attach it to any agent in your workspace" icon: "book" --- -The knowledge base is a workspace-wide document library. Add a source once — an uploaded Markdown or plain-text file — and attach it to any number of agents. In conversation, the agent draws on attached sources to answer questions its [instructions](/agents/build/configuration) alone can't cover: product details, policies, FAQs, procedures. +The knowledge base is a workspace-wide document library. Add a source once (an uploaded Markdown or plain-text file) and attach it to any number of agents. In conversation, the agent draws on attached sources to answer questions its [instructions](/agents/build/configuration) alone can't cover: product details, policies, FAQs, procedures. ## Add sources Sources come in two scopes: -- **Workspace sources** live in the shared library. Add them from the workspace-level **Knowledge base** page with **Add file** — choose a `.md` or `.txt` file (UTF-8 text, up to 1 MB) and give the source a name and an optional description. Workspace sources can be attached to any number of agents. +- **Workspace sources** live in the shared library. Add them from the workspace-level **Knowledge base** page with **Add file**: choose a `.md` or `.txt` file (UTF-8 text, up to 1 MB) and give the source a name and an optional description. Workspace sources can be attached to any number of agents. - **Agent-private sources** belong to a single agent. On your agent's **Knowledge base** tab, **New private file** creates a source that is attached to that agent immediately. It doesn't appear in the shared library, can't be attached to other agents, and is not listed by the [knowledge-sources API](#manage-sources-over-the-api). To attach a shared source to an agent, open the agent's **Knowledge base** tab and click **Import from global knowledge base**. -Every source is plain text and stays editable after upload: open a source to view its content and edit the text in place — you never re-upload a file to make a change. +Every source is plain text and stays editable after upload: open a source to view its content and edit the text in place. PDF, Office, and scanned documents are not supported. Convert them to Markdown or plain text first. @@ -23,20 +23,20 @@ PDF, Office, and scanned documents are not supported. Convert them to Markdown o ### Upload validation -Files are validated, parsed, and indexed as part of the upload itself — there is no background processing to wait for. When the upload succeeds, the source is immediately available to attached agents. If the file is empty, over 1 MB, or not UTF-8 text, the upload is rejected with an error and nothing is created — fix the file and try again. +Files are validated, parsed, and indexed as part of the upload itself. When the upload succeeds, the source is immediately available to attached agents. If the file is empty, over 1 MB, or not UTF-8 text, the upload is rejected with an error and nothing is created. ### Folders -Group sources into folders with **New folder**, and rename sources from the row menu. Deleting a folder never deletes its sources — anything inside moves back to **All sources**. +Group sources into folders with **New folder**, and rename sources from the row menu. Deleting a folder never deletes its sources. Anything inside moves back to **All sources**. ## Share sources across agents Workspace sources belong to the workspace, not to a single agent (agent-private files offer only **Edit** and **Delete**). In the library, the **Used by** column shows how many agents attach each source, and **Manage access** lists every agent with a toggle: -- **Attach** — turn an agent on to give it access to the source. -- **Detach** — turn an agent off. The source stays in the library and remains attached to other agents. +- **Attach**: Turn an agent on to give it access to the source. +- **Detach**: Turn an agent off. The source stays in the library and remains attached to other agents. -Editing a source propagates to every agent that attaches it — you never update agents one by one. Each attached agent shows the edit as an unpublished draft change. +Editing a source propagates to every agent that attaches it. Each attached agent shows the edit as an unpublished draft change. **Delete** removes a source from *all* agents that reference it, not just the one you're editing. To remove a source from a single agent, detach it with **Manage access** instead. @@ -49,7 +49,7 @@ Editing a source propagates to every agent that attaches it — you never update If the total size of an agent's attached sources is **8 KB or less**, the full content is inlined into the agent's instructions. This is the fastest path: the agent has everything up front, with zero per-turn lookup cost. -Inlined knowledge produces no retrieval events. If your corpus is small and you don't see retrieval activity, nothing is wrong — the content is already in the prompt. +Inlined knowledge produces no retrieval events. If your corpus is small and you don't see retrieval activity, nothing is wrong. ### Larger corpora are searched per turn @@ -73,14 +73,14 @@ Attaching more than 100 sources to one agent is rejected with `422`. ## Manage sources over the API -Knowledge sources are exposed at `/v1/agent/knowledge-sources`. Only workspace sources appear on this surface — agent-private files are a console-only convenience: +Knowledge sources are exposed at `/v1/agent/knowledge-sources`. Only workspace sources appear on this surface. Agent-private files are a console-only convenience: | Endpoint | Description | |---|---| | `GET /v1/agent/knowledge-sources` | List sources in your workspace | -| `POST /v1/agent/knowledge-sources` | Create a source — multipart upload with the `.md`/`.txt` file in the `source` field | +| `POST /v1/agent/knowledge-sources` | Create a source: multipart upload with the `.md`/`.txt` file in the `source` field | | `GET /v1/agent/knowledge-sources/{source_id}` | Retrieve a source | -| `PATCH /v1/agent/knowledge-sources/{source_id}` | Rename a source or replace its content — multipart like create; omitted fields keep their values, and a new `source` file bumps `revision_number` | +| `PATCH /v1/agent/knowledge-sources/{source_id}` | Rename a source or replace its content. Multipart like create; omitted fields keep their values, and a new `source` file bumps `revision_number` | | `DELETE /v1/agent/knowledge-sources/{source_id}` | Delete a source | | `GET /v1/agent/knowledge-sources/{source_id}/agents` | List agents that attach the source | @@ -103,10 +103,10 @@ curl https://api.fish.audio/v1/agent/knowledge-sources/{source_id}/agents \ --header "Authorization: Bearer $FISH_API_KEY" ``` -Only the `source` file part is required on create — `name` falls back to the uploaded file's name. +Only the `source` file part is required on create: `name` falls back to the uploaded file's name. -`DELETE` returns `409` while the source is still attached to any agent — in its draft or its published version. Call the dependents endpoint to see which agents use it, then detach it from each — remove its id from `knowledge_base.knowledge_source_ids` in an agent update — and republish any agent whose published version still references it. Deleting from the console instead removes the source from all agents at once, after a confirmation. +`DELETE` returns `409` while the source is still attached to any agent, in its draft or its published version. Call the dependents endpoint to see which agents use it, then detach it from each (remove its id from `knowledge_base.knowledge_source_ids` in an agent update) and republish any agent whose published version still references it. Deleting from the console instead removes the source from all agents at once, after a confirmation. ## Going further @@ -116,7 +116,7 @@ Only the `source` file part is required on create — `name` falls back to the u Instructions, voice, and everything else your agent is made of. - How publishing snapshots your agent — knowledge revisions included. + How publishing snapshots your agent, knowledge revisions included. Talk to your draft and check knowledge answers before publishing. diff --git a/agents/build/system-tools.mdx b/agents/build/system-tools.mdx index 23f3677..3671497 100644 --- a/agents/build/system-tools.mdx +++ b/agents/build/system-tools.mdx @@ -4,19 +4,19 @@ description: "Built-in agent capabilities like hanging up the call, enabled per icon: "toggle-on" --- -System tools are capabilities built into the platform. There is nothing to create or host — each agent simply enables the ones it needs. +System tools are capabilities built into the platform. There is nothing to create or host. Each agent simply enables the ones it needs. | Tool | Key | What it does | |---|---|---| | Hang up call | `hang_up_call` | Lets the agent end the call itself. | -Unlike [webhook tools](/agents/build/webhook-tools), system tools are not workspace resources shared across agents. The on/off state is stored per agent — toggling a system tool on one agent never affects another. +Unlike [webhook tools](/agents/build/webhook-tools), system tools are not workspace resources shared across agents. The on/off state is stored per agent. ## Toggle in the Builder -Open your agent's **Tools** page in the Builder. The **System tools** card holds the **Hang up call** switch — flip it to enable or disable that capability for this agent. +Open your agent's **Tools** page in the Builder. The **System tools** card holds the **Hang up call** switch. ## Toggle via the API @@ -24,7 +24,7 @@ Over the REST API, system tool switches are part of the agent's configuration, i ## Transfer call -`transfer_call` has no switch of its own: it is enabled automatically once the agent has a transfer destination configured, and only on phone sessions — web sessions have no call to transfer. See [Transfers](/agents/telephony/transfers) for configuring destinations and everything else about transfers. +`transfer_call` has no switch of its own: it is enabled automatically once the agent has a transfer destination configured, and only on phone sessions (web sessions have no call to transfer). See [Transfers](/agents/telephony/transfers) for configuring destinations and everything else about transfers. ## Going further diff --git a/agents/build/time-timezone.mdx b/agents/build/time-timezone.mdx index 3c6391d..d37d9f3 100644 --- a/agents/build/time-timezone.mdx +++ b/agents/build/time-timezone.mdx @@ -1,10 +1,10 @@ --- title: "Time & Timezone" -description: "Your agent knows the current date and time by default — how the timezone is chosen and how to opt out" +description: "Your agent knows the current date and time by default: how the timezone is chosen and how to opt out" icon: "clock" --- -Agents are time-aware out of the box: every session knows today's date and the current time in the session's timezone. You don't reference a variable or add anything to your prompt — "tomorrow morning" and "next Tuesday" ground correctly from the first turn, on every channel (voice, text, and phone). +Agents are time-aware out of the box: every session knows today's date and the current time in the session's timezone. You don't reference a variable or add anything to your prompt: "tomorrow morning" and "next Tuesday" ground correctly from the first turn, on every channel (voice, text, and phone). ## What the agent knows @@ -15,7 +15,7 @@ Fish Audio injects two pieces of world context server-side: | At session start | Today's date and the session timezone | `Today is Thursday, July 23, 2026. Session timezone: Asia/Shanghai (UTC+8).` | | Before every reply | The current time, minute precision | `Current date and time: Thursday, July 23, 2026 at 13:00 (Asia/Shanghai).` | -The time is refreshed on every turn, so it stays accurate through long conversations and past midnight — it never freezes at the session's start time. +The time is refreshed on every turn, so it stays accurate through long conversations and past midnight. Injection happens outside your configuration, so it never counts toward the @@ -26,15 +26,15 @@ The time is refreshed on every turn, so it stays accurate through long conversat The timezone is resolved once, when the session is created, taking the first that applies: -1. **`timezone`** in the creation request — your explicit per-session choice, as an IANA name like `Asia/Shanghai`. An invalid name rejects the request with `422`. -2. **The agent's configured timezone** — set on the agent in the Builder (**Configuration → Timezone**) or via the [agent config API](/agents/build/configuration). Pin one when your agent serves a single region regardless of who calls. -3. **`client_timezone`** — a hint with the end user's browser timezone, sent automatically by the [Web SDK](/agents/deploy/web-sdk) in public-agent mode. Used only when neither of the above is set; an invalid hint is ignored rather than failing the session. -4. **The caller's phone number** — inbound calls infer the timezone from the caller's country when that country has a single timezone. +1. **`timezone`** in the creation request: your explicit per-session choice, as an IANA name like `Asia/Shanghai`. An invalid name rejects the request with `422`. +2. **The agent's configured timezone**: set on the agent in the Builder (**Configuration → Timezone**) or via the [agent config API](/agents/build/configuration). Pin one when your agent serves a single region regardless of who calls. +3. **`client_timezone`**: a hint with the end user's browser timezone, sent automatically by the [Web SDK](/agents/deploy/web-sdk) in public-agent mode. Used only when neither of the above is set; an invalid hint is ignored rather than failing the session. +4. **The caller's phone number**: inbound calls infer the timezone from the caller's country when that country has a single timezone. 5. **UTC** otherwise. In the browser, this means zero configuration: the SDK detects the visitor's real timezone and the agent talks about "today" in the user's local terms, not yours. -With [authenticated sessions](/agents/deploy/authenticated-sessions), the SDK's automatic hint doesn't apply — your backend creates the session, so set `timezone` (or forward the browser's value as `client_timezone`) in the creation request: +With [authenticated sessions](/agents/deploy/authenticated-sessions), the SDK's automatic hint doesn't apply. Your backend creates the session, so set `timezone` (or forward the browser's value as `client_timezone`) in the creation request: ```bash API (curl) curl --request POST https://api.fish.audio/v1/agent/sessions \ @@ -48,13 +48,13 @@ curl --request POST https://api.fish.audio/v1/agent/sessions \ Phone-number inference is country-level only. Countries that span several - timezones (the US, Canada, Australia, Russia, Brazil) are skipped — set the + timezones (the US, Canada, Australia, Russia, Brazil) are skipped. Set the agent's timezone for those. Calls that resolve nothing run in UTC. ## Turn it off -Time awareness is on by default. To withhold both the date and the per-turn time from a session — for example a role-play agent set on a fictional date, or a test that must be reproducible — set `world_context: false` when creating the session: +Time awareness is on by default. To withhold both the date and the per-turn time from a session (for example a role-play agent set on a fictional date, or a test that must be reproducible), set `world_context: false` when creating the session: ```bash API (curl) diff --git a/agents/build/tools.mdx b/agents/build/tools.mdx index 1a036ec..2e64943 100644 --- a/agents/build/tools.mdx +++ b/agents/build/tools.mdx @@ -1,6 +1,6 @@ --- title: "Tools" -description: "Give your agent the ability to act during a conversation — call your backend, trigger actions in your app, or use built-in capabilities" +description: "Give your agent the ability to act during a conversation: call your backend, trigger actions in your app, or use built-in capabilities" icon: "wrench" --- @@ -22,41 +22,41 @@ Out of the box, an agent can only talk. Tools let it act: fetch an order status | Type | Executed by | Use it for | |---|---|---| -| [Webhook](/agents/build/webhook-tools) | Fish Audio — we call your HTTP endpoint during the conversation | Order lookups, CRM reads, bookings — anything your backend can answer | -| [Client](/agents/build/client-tools) | Your app — the SDK hands the call to code you register | Navigation, UI updates, device actions — anything only the client can do | -| [System](/agents/build/system-tools) | The platform — no code involved | Hanging up the call | +| [Webhook](/agents/build/webhook-tools) | Fish Audio: we call your HTTP endpoint during the conversation | Order lookups, CRM reads, bookings, and anything else your backend can answer | +| [Client](/agents/build/client-tools) | Your app: the SDK hands the call to code you register | Navigation, UI updates, device actions, and anything else only the client can do | +| [System](/agents/build/system-tools) | The platform: no code involved | Hanging up the call | -Webhook and client tools are **custom tools**: you define a name, a description, and arguments, and the agent fills in the argument values when it calls. System tools are ready-made — enable them with a switch on each agent. +Webhook and client tools are **custom tools**: you define a name, a description, and arguments, and the agent fills in the argument values when it calls. System tools are ready-made. Enable them with a switch on each agent. ## Where tools live Custom tools are **workspace resources**, not part of a single agent. Define a `lookup_order` tool once and attach it to your support, sales, and after-hours agents. You manage them in two places: - **Builder → Tools** shows the tools attached to the agent you are editing, plus that agent's system-tool switches. Tools you create here are attached to the agent immediately. -- **The workspace tool library** lists every custom tool with how many agents use it. Tools you create here start unattached — grant agents access afterwards. +- **The workspace tool library** lists every custom tool with how many agents use it. Tools you create here start unattached. Grant agents access afterwards. -Editing a shared tool updates the draft of **every** agent that uses it. Those agents show unpublished changes until they are published again — see [Versions & publishing](/agents/deploy/versions-publishing). +Editing a shared tool updates the draft of **every** agent that uses it. Those agents show unpublished changes until they are published again. See [Versions & publishing](/agents/deploy/versions-publishing). ## Detach vs. delete -- **Detach** is agent-scoped. Removing a tool from an agent in the Builder leaves the tool — and its other attachments — untouched. +- **Detach** is agent-scoped. Removing a tool from an agent in the Builder leaves the tool (and its other attachments) untouched. - **Delete** is workspace-wide. Deleting a tool from the library in the console detaches it from every agent that uses it, and those agents show unpublished changes. Over the API, `DELETE` is stricter: it returns `409` while the tool is still attached to any agent. List the tool's dependent agents first and detach it from each one. ## Manage tools over the API -Every endpoint requires your API key — see the [API introduction](/api-reference/introduction). +Every endpoint requires your API key. See the [API introduction](/api-reference/introduction). | Endpoint | Description | |---|---| | `GET /v1/agent/tools` | List the custom tools in your workspace | -| `POST /v1/agent/tools` | Create a tool — starts unattached | +| `POST /v1/agent/tools` | Create a tool: starts unattached | | `GET /v1/agent/tools/{tool_id}` | Retrieve a tool's configuration | -| `PATCH /v1/agent/tools/{tool_id}` | Update a tool — every field is optional | -| `DELETE /v1/agent/tools/{tool_id}` | Delete a tool — returns `409` while any agent still uses it | +| `PATCH /v1/agent/tools/{tool_id}` | Update a tool: every field is optional | +| `DELETE /v1/agent/tools/{tool_id}` | Delete a tool: returns `409` while any agent still uses it | | `GET /v1/agent/tools/{tool_id}/agents` | List the agents that use this tool | Create a webhook tool: @@ -88,7 +88,7 @@ curl https://api.fish.audio/v1/agent/tools/{tool_id}/agents \ The response lists each dependent agent's `agent_id` and `name`, plus a `total` count. -Field-level configuration — custom headers, request body templates, timeouts, error handling, and mock responses — is covered in [Webhook tools](/agents/build/webhook-tools). +Field-level configuration (custom headers, request body templates, timeouts, error handling, and mock responses) is covered in [Webhook tools](/agents/build/webhook-tools). ## Going further diff --git a/agents/build/voice-language.mdx b/agents/build/voice-language.mdx index bc6bdab..37313f0 100644 --- a/agents/build/voice-language.mdx +++ b/agents/build/voice-language.mdx @@ -4,7 +4,7 @@ description: "Choose the voice your agent speaks with and the language it conver icon: "microphone" --- -Your agent speaks with a voice model from the Fish Audio Voice Library — the same voices you use for text to speech. Pick one in the Builder, or set it through the API, and choose the language the agent holds conversations in. +Your agent speaks with a voice model from the Fish Audio Voice Library: the same voices you use for text to speech. Pick one in the Builder, or set it through the API, and choose the language the agent holds conversations in. @@ -27,10 +27,10 @@ The voice picker on the **Configuration** page has two levels: a curated list fo In your agent's **Configuration** page, open the voice card. The **Choose a voice** view shows a curated selection of voices. - Not seeing the right fit? Select **More voices** to open the **Select Voice** browser — the full Voice Library, with your own cloned voices under **My Voices**. + Not seeing the right fit? Select **More voices** to open the **Select Voice** browser: the full Voice Library, with your own cloned voices under **My Voices**. - Your selection is written to the agent's draft configuration as soon as you pick it — there is no Save button. Start a [preview call](/agents/test/preview-calls) to hear the voice in a real conversation. + Your selection is written to the agent's draft configuration as soon as you pick it. Start a [preview call](/agents/test/preview-calls) to hear the voice in a real conversation. Draft changes don't affect live sessions until you **Publish**. See [Versions & publishing](/agents/deploy/versions-publishing). @@ -39,14 +39,14 @@ The voice picker on the **Configuration** page has two levels: a curated list fo ## Use any voice model -The agent's voice is a **voice model id** (`voice_id`) — the same ids used as `reference_id` in [Text to Speech](/features/text-to-speech). Any public voice model from the Voice Library works, including: +The agent's voice is a **voice model id** (`voice_id`): the same ids used as `reference_id` in [Text to Speech](/features/text-to-speech). Any public voice model from the Voice Library works, including: -- **Library voices** — ready-made public voices. Find ids in the [Voice Library](/features/manage-voices). -- **Your cloned voices** — [clone a voice](/features/voice-cloning) once, then use its model id as your agent's voice. +- **Library voices**: Ready-made public voices. Find ids in the [Voice Library](/features/manage-voices). +- **Your cloned voices**: [Clone a voice](/features/voice-cloning) once, then use its model id as your agent's voice. ## Set the voice via API -Voice settings live in the `voice` section of the agent's configuration. Patches are partial — only the fields you send change, and the result is saved to the draft: +Voice settings live in the `voice` section of the agent's configuration. Patches are partial: only the fields you send change, and the result is saved to the draft: ```bash API (curl) @@ -61,21 +61,21 @@ curl --request PATCH "https://api.fish.audio/v1/agent/agents/$AGENT_ID/config" \ ``` -`voice.speaking_language` lives in the same section and is patched the same way. As in the Builder, API edits land in the draft — publish to roll them out. +`voice.speaking_language` lives in the same section and is patched the same way. As in the Builder, API edits land in the draft. Publish to roll them out. ## Speaking language -**Speaking language** sets the language for the agent's conversations — one of `en`, `ja`, `zh`, `ko`, `es`, `fr`, or `de` (`voice.speaking_language` on the wire). Every session converses in this language. +**Speaking language** sets the language for the agent's conversations: one of `en`, `ja`, `zh`, `ko`, `es`, `fr`, or `de` (`voice.speaking_language` on the wire). Every session converses in this language. The voice model and the speaking language are independent settings: picking a voice does not change the language, and vice versa. Choose a voice that sounds natural in the language you configure. -Both settings can also be replaced for a single session — send `overrides.voice_id` or `overrides.language` on the session request. See [Overrides](/agents/deploy/authenticated-sessions#overrides). +Both settings can also be replaced for a single session: send `overrides.voice_id` or `overrides.language` on the session request. See [Overrides](/agents/deploy/authenticated-sessions#overrides). ## Expressive mode -**Expressive mode** makes the agent steer its own delivery: it opens sentences with emotion cues, adds natural pauses and emphasis, laughs where it genuinely fits, and speaks the way people talk — contractions and the occasional "um". You get lively, emotionally aware speech without writing any delivery rules into your prompt. +**Expressive mode** makes the agent steer its own delivery: it opens sentences with emotion cues, adds natural pauses and emphasis, laughs where it genuinely fits, and speaks the way people talk, with contractions and the occasional "um". You get lively, emotionally aware speech without writing any delivery rules into your prompt. Turn it on with the **Expressive mode** switch in the Builder's voice section, or via the API (`voice.expressive`, default `false`): @@ -92,10 +92,10 @@ curl --request PATCH "https://api.fish.audio/v1/agent/agents/$AGENT_ID/config" \ ``` -The delivery cues are rendered by the voice model — they are never spoken and never appear in transcripts or message history. Expressive mode applies to spoken sessions (voice calls and phone); text chat is unaffected. +The delivery cues are rendered by the voice model. They are never spoken and never appear in transcripts or message history. Expressive mode applies to spoken sessions (voice calls and phone); text chat is unaffected. - Keep your system prompt about **what** the agent says — who it is, what it knows, what it should do. With expressive mode on, **how** it sounds is handled for you. + Keep your system prompt about **what** the agent says: who it is, what it knows, what it should do. With expressive mode on, **how** it sounds is handled for you. ## Going further diff --git a/agents/build/webhook-tools.mdx b/agents/build/webhook-tools.mdx index 5efbe32..3e93019 100644 --- a/agents/build/webhook-tools.mdx +++ b/agents/build/webhook-tools.mdx @@ -4,10 +4,10 @@ description: "Let your agent call your HTTP endpoints in the middle of a convers icon: "webhook" --- -A webhook tool lets your agent call an HTTP endpoint mid-conversation — look up an order, create a ticket, check availability — and use the response in its next reply. You declare the arguments; the agent fills them in from the conversation and the platform makes the request. +A webhook tool lets your agent call an HTTP endpoint mid-conversation (look up an order, create a ticket, check availability) and use the response in its next reply. You declare the arguments; the agent fills them in from the conversation and the platform makes the request. -Tools are workspace-level resources shared across agents. Removing a tool from an agent **detaches** it — the tool stays in your workspace and remains attached to other agents. Editing a tool updates every agent that uses it, and those agents show unpublished changes until you [publish](/agents/deploy/versions-publishing). +Tools are workspace-level resources shared across agents. Removing a tool from an agent **detaches** it: the tool stays in your workspace and remains attached to other agents. Editing a tool updates every agent that uses it, and those agents show unpublished changes until you [publish](/agents/deploy/versions-publishing). ## Create a webhook tool @@ -16,16 +16,16 @@ In the Builder, open **Tools** and choose **Add tool → Webhook**. A tool creat - The model decides when to call a tool based on its name and description. Write the description for the model: say what the tool does and when to use it — "Look up the status of an order. Use when the caller asks where their order is." + The model decides when to call a tool based on its name and description. Write the description for the model: say what the tool does and when to use it, e.g. "Look up the status of an order. Use when the caller asks where their order is." - Each argument is a name plus a description. Arguments are exposed to the agent as tool parameters — the description tells the model what value to extract from the conversation. + Each argument is a name plus a description. Arguments are exposed to the agent as tool parameters. The description tells the model what value to extract from the conversation. - Pick a method (`GET`, `POST`, `PUT`, `PATCH`, or `DELETE` — default `POST`), the endpoint URL, a content type (default `application/json`), and an optional body template. + Pick a method (`GET`, `POST`, `PUT`, `PATCH`, or `DELETE`; default `POST`), the endpoint URL, a content type (default `application/json`), and an optional body template. - Attach authentication or any custom headers — they are sent with every webhook request. + Attach authentication or any custom headers. They are sent with every webhook request. Decide whether the agent waits for the response, fires the request and moves on, or keeps talking while the request runs in the background. See [Execution modes](#execution-modes). @@ -37,7 +37,7 @@ In the Builder, open **Tools** and choose **Add tool → Webhook**. A tool creat ## Argument templating -Reference any declared argument with `{{name}}` — in the URL, in the body template, or both. The platform substitutes the values the agent supplies before sending the request. +Reference any declared argument with `{{name}}`: in the URL, in the body template, or both. The platform substitutes the values the agent supplies before sending the request. ```text Endpoint URL https://api.example.com/orders/{{order_number}} @@ -58,10 +58,10 @@ Each header has a kind that identifies what it carries: | Kind | Wire value | Sent as | |---|---|---| | Custom | `custom` | The header name and value you provide, e.g. `X-API-Key` | -| Bearer | `authorization_bearer` | An `Authorization` header — provide the full value, e.g. `Bearer ` | -| Basic | `authorization_basic` | An `Authorization` header — provide the full value, e.g. `Basic ` | +| Bearer | `authorization_bearer` | An `Authorization` header. Provide the full value, e.g. `Bearer ` | +| Basic | `authorization_basic` | An `Authorization` header. Provide the full value, e.g. `Basic ` | -Header values are sent exactly as you store them. The Builder's Bearer and Basic presets create the `Authorization` header with the `Bearer ` or `Basic ` prefix pre-filled — complete the value with your credential. +Header values are sent exactly as you store them. The Builder's Bearer and Basic presets create the `Authorization` header with the `Bearer ` or `Basic ` prefix pre-filled. Complete the value with your credential. Bearer and Basic authorization values are write-only. The API accepts them on create and update but never echoes them when you read the tool back: credential headers return `value: null`, with `has_secret: true` confirming a secret is stored. To rotate a secret, submit a new value. @@ -74,12 +74,12 @@ Bearer and Basic authorization values are write-only. The API accepts them on cr | Mode | The agent | Your endpoint's response | |---|---|---| | `blocking` | Waits for the response before replying | Used in the agent's next reply | -| `fire_and_forget` | Continues immediately | Discarded — recorded in the tool-call history, never spoken | +| `fire_and_forget` | Continues immediately | Discarded: recorded in the tool-call history, never spoken | | `background` | Keeps talking while the request runs | Announced at the next natural pause once it arrives | -**Blocking** is right when the answer drives the conversation — an order lookup the caller is waiting on. Keep those endpoints fast: the caller hears silence while your backend works. +**Blocking** is right when the answer drives the conversation, like an order lookup the caller is waiting on. Keep those endpoints fast: the caller hears silence while your backend works. -**Fire and forget** fits actions that need no spoken confirmation from your backend: recording a consent, logging an outcome to your CRM. The agent tells the model the request was dispatched and moves on. Failures appear only in the conversation's tool-call history — the agent is never told, so `error_handling` has no spoken effect in this mode. +**Fire and forget** fits actions that need no spoken confirmation from your backend: recording a consent, logging an outcome to your CRM. The agent tells the model the request was dispatched and moves on. Failures appear only in the conversation's tool-call history. The agent is never told, so `error_handling` has no spoken effect in this mode. **Background** fits slow work that would otherwise stall the call: aggregating a shipment status across carriers, generating a report, confirming a payment. The agent acknowledges that the request is running and continues the conversation; when the response arrives, it works the result into the conversation at the next natural pause. The model is explicitly told not to guess the result before it arrives. Failures are delivered the same way, shaped by `error_handling`. @@ -91,36 +91,36 @@ A few things to know about background calls: - Up to **10 background calls** can be in flight per conversation. Further calls fail immediately with an error the agent can react to ("let me finish checking the first thing you asked about"). - When the conversation ends, in-flight background calls are cancelled; the attempt stays visible in the conversation's tool-call history. -- In text-only runs — [agent tests](/agents/test/agent-tests), for example — every webhook tool executes as `blocking`, so results always appear in the transcript in order. +- In text-only runs ([agent tests](/agents/test/agent-tests), for example), every webhook tool executes as `blocking`, so results always appear in the transcript in order. Client tools have their own lighter-weight equivalent: the [`expects_response` switch](/agents/build/client-tools), which makes a client tool fire-and-forget. ## Timeouts and error handling -`timeout_seconds` caps how long the platform waits for your endpoint: 1–120 seconds, default 30. Background tools may go up to 300 seconds — they don't hold up the conversation while they run. Voice conversations happen live — for blocking tools especially, keep endpoints fast and lower the timeout so a slow backend can't stall the call. +`timeout_seconds` caps how long the platform waits for your endpoint: 1–120 seconds, default 30. Background tools may go up to 300 seconds, since they don't hold up the conversation while they run. For blocking tools especially, keep endpoints fast and lower the timeout so a slow backend can't stall the call. `error_handling` controls what the agent learns when a call fails (default `passthrough`): | Option | What the agent sees | |---|---| | `passthrough` | The error response, so the agent can react to it in conversation ("I couldn't find that order number"). | -| `hide` | Only that the call failed — no error details reach the agent. | +| `hide` | Only that the call failed: no error details reach the agent. | Use `hide` when error responses might leak internal details you don't want spoken aloud. Under `fire_and_forget` the agent never learns about failures at all, whichever option is set. ## Mock responses -A tool can store mock responses — canned payloads, each with a `name`, a `status_code` (100–599), a `content_type`, and a `body`. Mocks are saved with the tool's configuration for test scenarios, but they don't intercept anything yet: preview and live calls always hit the real endpoint. +A tool can store mock responses: canned payloads, each with a `name`, a `status_code` (100–599), a `content_type`, and a `body`. Mocks are saved with the tool's configuration for test scenarios, but they don't intercept anything yet: preview and live calls always hit the real endpoint. ## Test your tool -The tool editor's **Test** tab fires a real request at your endpoint. Fill in the arguments as JSON (pre-filled with a sample based on your declared arguments) and send — you get back the status code, latency, response headers, and response body. A failing test never blocks saving the tool. +The tool editor's **Test** tab fires a real request at your endpoint. Fill in the arguments as JSON (pre-filled with a sample based on your declared arguments) and send. You get back the status code, latency, response headers, and response body. A failing test never blocks saving the tool. Response bodies are captured up to 64 KB; larger bodies are cut off and flagged with `response_truncated`. ## Create tools via the API -`POST /v1/agent/tools` creates a tool in your workspace. Tools created this way are not attached to any agent — enable them per agent afterwards. +`POST /v1/agent/tools` creates a tool in your workspace. Tools created this way are not attached to any agent. Enable them per agent afterwards. ```bash Create a webhook tool curl --request POST https://api.fish.audio/v1/agent/tools \ @@ -148,7 +148,7 @@ curl --request POST https://api.fish.audio/v1/agent/tools \ ## Escalate to a ticket mid-call -When the agent can't resolve an issue, the strongest close is a ticket opened while the caller is still on the line, with the ticket number read back aloud. The `create_ticket` tool above is the entire integration — what makes it work is the prompt around it and the response your endpoint returns. +When the agent can't resolve an issue, the strongest close is a ticket opened while the caller is still on the line, with the ticket number read back aloud. The `create_ticket` tool above is the entire integration. What makes it work is the prompt around it and the response your endpoint returns. **Set the escalation policy in the prompt.** The tool description says what the tool does; the [system prompt](/agents/build/configuration) says when escalating is the right move: @@ -160,7 +160,7 @@ After the tool returns, read the ticket number back and say when to expect a reply. ``` -**Return something worth saying.** With `error_handling: passthrough` (the default), the agent sees your response body and uses it in its next reply — so respond with what the caller should hear: +**Return something worth saying.** With `error_handling: passthrough` (the default), the agent sees your response body and uses it in its next reply, so respond with what the caller should hear: ```json Endpoint response { "ticket_id": "T-1042", "expected_reply": "within 24 hours" } @@ -170,8 +170,8 @@ Ticket numbers get spoken aloud: short, pronounceable IDs survive text-to-speech An in-call ticket depends on the model choosing to escalate. For a safety net - that catches every unresolved call — including those where the agent never - called the tool — pair this with [auto-ticketing from post-call + that catches every unresolved call (including those where the agent never + called the tool), pair this with [auto-ticketing from post-call analysis](/agents/monitor/webhooks#auto-ticket-unresolved-calls). @@ -179,7 +179,7 @@ Ticket numbers get spoken aloud: short, pronounceable IDs survive text-to-speech | Field | Limit | |---|---| -| `name` | 1–120 characters. Display text — the model-facing function name is derived from it. [Client tool names](/agents/build/client-tools#naming-rules) are stricter: a 64-character pattern, since the SDK registers handlers by exact name | +| `name` | 1–120 characters. Display text; the model-facing function name is derived from it. [Client tool names](/agents/build/client-tools#naming-rules) are stricter: a 64-character pattern, since the SDK registers handlers by exact name | | `description` | up to 2,000 characters | | Argument `name` | 1–64 characters | | Argument `description` | up to 500 characters | diff --git a/agents/concepts.mdx b/agents/concepts.mdx index fc55588..e9765bf 100644 --- a/agents/concepts.mdx +++ b/agents/concepts.mdx @@ -1,11 +1,11 @@ --- title: "Core Concepts" sidebarTitle: "Concepts" -description: "Workspaces, agents, versions, and sessions — the mental model behind Fish Agents" +description: "Workspaces, agents, versions, and sessions: the mental model behind Fish Agents" icon: "sitemap" --- -Four nouns cover the whole platform. A **workspace** holds everything your team builds. An **agent** is a voice assistant you configure. Publishing an agent's draft creates an immutable **version**. And every conversation with an agent — from your app, a phone call, or the console — is a **session**. +Four nouns cover the whole platform. A **workspace** holds everything your team builds. An **agent** is a voice assistant you configure. Publishing an agent's draft creates an immutable **version**. And every conversation with an agent (from your app, a phone call, or the console) is a **session**. ## Workspaces @@ -24,7 +24,7 @@ Create keys under [API keys](https://fish.audio/app/api-keys/) in the console. ## Agents -An agent is a bundle of configuration: a system prompt that steers behavior, a first-message policy, a [voice and speaking language](/agents/build/voice-language), attached [tools](/agents/build/tools), and [knowledge sources](/agents/build/knowledge-base). You assemble it in the Builder — the console page you land on after creating an agent — or through the API at `/v1/agent/agents`. +An agent is a bundle of configuration: a system prompt that steers behavior, a first-message policy, a [voice and speaking language](/agents/build/voice-language), attached [tools](/agents/build/tools), and [knowledge sources](/agents/build/knowledge-base). You assemble it in the Builder (the console page you land on after creating an agent) or through the API at `/v1/agent/agents`. Creating an agent takes only a name. Everything else is configured afterward, so start minimal and iterate. @@ -34,8 +34,8 @@ Every agent has exactly one **draft** and any number of immutable **published ve - Everything you change in the Builder autosaves to the draft — there is no - Save button. Watch for the `Saving… / Saved` indicator. + Everything you change in the Builder autosaves to the draft. Watch for the + `Saving… / Saved` indicator. **Publish** snapshots the draft into a numbered, immutable version and makes @@ -44,7 +44,7 @@ Every agent has exactly one **draft** and any number of immutable **published ve Continue editing the draft without affecting live conversations. When you're - ready, publish again — the version number increments. + ready, publish again. The version number increments. @@ -57,7 +57,7 @@ The agent list shows one of two states: | **Live** | The agent has been published at least once | | **Draft** | The agent has never been published | -Editing the draft of a Live agent does not change its badge — the Builder shows an unpublished-changes indicator next to **Publish** instead. +Editing the draft of a Live agent does not change its badge. The Builder shows an unpublished-changes indicator next to **Publish** instead. ### Which configuration runs @@ -71,7 +71,7 @@ Editing the draft of a Live agent does not change its badge — the Builder show ### Restore and clone -Past versions stay in the agent's history — review them or restore one back into the draft from [Versions & publishing](/agents/deploy/versions-publishing). +Past versions stay in the agent's history. Review them or restore one back into the draft from [Versions & publishing](/agents/deploy/versions-publishing). **Clone** creates a new agent from an existing one's current draft: prompt, voice, and tool and knowledge attachments carry over (shared resources are referenced, not copied). The clone is named `{name} copy`, starts as Draft, and does not inherit the source's publish history. @@ -95,15 +95,15 @@ A session is one conversation between one user and one agent. Sessions start fro -Except [preview calls](/agents/test/preview-calls), which are ephemeral and never enter history, every session appears in [Conversation history](/agents/monitor/conversation-history) and `/v1/agent/sessions` — with its transcript, and its recording when the agent [records audio](/agents/monitor/conversation-history#what-gets-stored) — and can trigger [post-call analysis](/agents/monitor/post-call-analysis) when configured. +Except [preview calls](/agents/test/preview-calls), which are ephemeral and never enter history, every session appears in [Conversation history](/agents/monitor/conversation-history) and `/v1/agent/sessions` (with its transcript, and its recording when the agent [records audio](/agents/monitor/conversation-history#what-gets-stored)) and can trigger [post-call analysis](/agents/monitor/post-call-analysis) when configured. ## Deleting resources Deletion is immediate and, from the console, cannot be undone: a deleted agent disappears from lists, the Builder, and all API responses. The same applies to tools, knowledge sources, and tests. - Names are freed on deletion — you can immediately create a new agent (or tool) - with the same name as one you deleted. + Names are freed on deletion: you can immediately create a new agent (or tool) + with the same name. ## Where everything lives diff --git a/agents/deploy/authenticated-sessions.mdx b/agents/deploy/authenticated-sessions.mdx index 9172ca9..9c4d3a1 100644 --- a/agents/deploy/authenticated-sessions.mdx +++ b/agents/deploy/authenticated-sessions.mdx @@ -4,7 +4,7 @@ description: "Create short-lived session tokens on your backend and run private icon: "server" --- -An authenticated session starts on your server: your backend calls the session endpoint with your API key and receives a short-lived **session token**, which your client uses to connect. The API key never leaves your server, and every session parameter — user identity, overrides, dynamic variables — is set by code you trust. This is how private agents run in production. +An authenticated session starts on your server: your backend calls the session endpoint with your API key and receives a short-lived **session token**, which your client uses to connect. The API key never leaves your server, and every session parameter (user identity, overrides, dynamic variables) is set by code you trust. This is how private agents run in production. The token is client-agnostic. Create it the same way regardless of which surface renders the conversation: @@ -18,7 +18,7 @@ The token is client-agnostic. Create it the same way regardless of which surface No backend, and anyone may talk to the agent? A [public agent](/agents/deploy/public-agents) lets the SDK create sessions with just an - `agentId` — no token involved, gated by an origin allowlist and rate limits. + `agentId`: no token involved, gated by an origin allowlist and rate limits. ## Create a token on your backend @@ -28,7 +28,7 @@ Your backend exchanges your API key for a single-conversation token. Call `POST /v1/agent/sessions` with your API key. This is where you set - per-session parameters — user identity, overrides, dynamic variables. + per-session parameters: user identity, overrides, dynamic variables. The response is the session token. Forward it verbatim. @@ -109,7 +109,7 @@ More on what the SDK can do once connected is in the [Web SDK](/agents/deploy/we `overrides`, `dynamic_variables`, `tool_events`, `timezone`, and - `world_context` belong in your backend's creation request — the SDK forwards + `world_context` belong in your backend's creation request. The SDK forwards these options (and its `language` shorthand for `overrides.language`) only in [public agent](/agents/deploy/public-agents) mode. @@ -119,45 +119,45 @@ More on what the SDK can do once connected is in the [Web SDK](/agents/deploy/we | Field | Type | Description | | ------------------- | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `agent_id` | string, required | The agent to talk to. It must have a [published version](/agents/deploy/versions-publishing). | -| `name` | string, optional | Display name for this session in the console's Conversations list, up to 128 characters. Omit it to show the session's start time instead. API-key requests only — keyless (public) creation rejects it with `400`. | -| `overrides` | object, optional | Replace parts of the published configuration for this session — see [Overrides](#overrides). | +| `name` | string, optional | Display name for this session in the console's Conversations list, up to 128 characters. Omit it to show the session's start time instead. API-key requests only: keyless (public) creation rejects it with `400`. | +| `overrides` | object, optional | Replace parts of the published configuration for this session. See [Overrides](#overrides). | | `dynamic_variables` | object, optional | Up to 50 entries of string, number, or boolean values, substituted into `{{placeholders}}`. See [Dynamic variables](/agents/build/dynamic-variables). | | `tool_events` | boolean, optional | Stream tool lifecycle events (`toolCallStarted` / `toolCallCompleted` / `toolCallFailed`) to the client. Default `true`; set `false` to keep tool inputs and outputs off the client. | | `end_user_id` | string, optional | Your identifier for the end user, up to 256 characters. Stored on the session and echoed in [webhook](/agents/monitor/webhooks) payloads and [custom LLM](/agents/build/custom-llm) requests. | -| `metadata` | object, optional | Your own key-value namespace. Stored and returned verbatim on session queries and webhooks — never read or interpreted by the platform. | -| `record_audio` | boolean, optional | Whether to record this session's audio. Overrides the agent's [recording setting](/agents/monitor/conversation-history#what-gets-stored) for this session only — it never changes the agent; omit it to use the agent's configuration. | +| `metadata` | object, optional | Your own key-value namespace. Stored and returned verbatim on session queries and webhooks, never read or interpreted by the platform. | +| `record_audio` | boolean, optional | Whether to record this session's audio. Overrides the agent's [recording setting](/agents/monitor/conversation-history#what-gets-stored) for this session only; omit it to use the agent's configuration. | | `timezone` | string, optional | IANA timezone (like `Asia/Shanghai`) for the agent's sense of local time. Invalid names are rejected with `422`. See [Time & timezone](/agents/build/time-timezone). | | `client_timezone` | string, optional | The end user's browser timezone, filled automatically by the SDK in public-agent mode. A hint, not a demand: it applies only when neither `timezone` nor the agent's configured timezone is set, and invalid values are ignored. See the [resolution order](/agents/build/time-timezone). | | `world_context` | boolean, optional | Whether the agent knows the current date and time. Default `true`; set `false` to withhold both from this session. | -Unknown fields — top-level or inside `overrides` — are rejected with `422`. +Unknown fields (top-level or inside `overrides`) are rejected with `422`. ## Overrides -`overrides` replaces parts of the agent's published configuration for one session — the agent itself never changes: +`overrides` replaces parts of the agent's published configuration for one session: | Field | Type | Effect for this session | | ---------------------- | ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `first_message` | string, up to 10,000 characters | The agent opens with exactly this text, spoken verbatim — whatever first-message mode the configuration sets. `{{placeholders}}` render inside it. | -| `first_message_prompt` | string, up to 10,000 characters | Instructions the agent generates its opener from, replacing the configured first-message behavior. `{{placeholders}}` render inside it. Mutually exclusive with `first_message` — sending both is `422`. | +| `first_message` | string, up to 10,000 characters | The agent opens with exactly this text, spoken verbatim, whatever first-message mode the configuration sets. `{{placeholders}}` render inside it. | +| `first_message_prompt` | string, up to 10,000 characters | Instructions the agent generates its opener from, replacing the configured first-message behavior. `{{placeholders}}` render inside it. Mutually exclusive with `first_message`: sending both is `422`. | | `system_prompt` | string, up to 4,000 tokens | Replaces the configured system prompt entirely, under the same token budget. `{{placeholders}}` render inside it. | -| `voice_id` | string | The voice the agent speaks with — any [voice model id](/agents/build/voice-language#use-any-voice-model) from the Voice Library. Voices bias pronunciation toward their own language, so pair it with `language`. | +| `voice_id` | string | The voice the agent speaks with: any [voice model id](/agents/build/voice-language#use-any-voice-model) from the Voice Library. Voices bias pronunciation toward their own language, so pair it with `language`. | | `language` | `en`, `ja`, `zh`, `ko`, `es`, `fr`, `de` | Pins the conversation language, taking precedence over the configured [speaking language](/agents/build/voice-language#speaking-language). | ```json { "agent_id": "YOUR_AGENT_ID", "overrides": { - "first_message": "Welcome back, {{name}} — picking up where we left off.", + "first_message": "Welcome back, {{name}}, picking up where we left off.", "voice_id": "802e3bc2b27e49c2995d23ef70e6ac89", "language": "ja" } } ``` -On [public agents](/agents/deploy/public-agents), keyless (browser-created) sessions may override only `language` and `voice_id` — the prompt-shaping fields are rejected with `400` regardless of the agent. Create the session from your backend to use them. +On [public agents](/agents/deploy/public-agents), keyless (browser-created) sessions may override only `language` and `voice_id`. The prompt-shaping fields are rejected with `400` regardless of the agent. Create the session from your backend to use them. -Everything else on the request is not an override: `dynamic_variables`, `timezone`, `record_audio`, and `tool_events` are separate top-level fields. Session length is not settable per session — `max_duration_seconds` comes from the agent's [conversation configuration](/agents/build/configuration#call-duration). +Everything else on the request is not an override: `dynamic_variables`, `timezone`, `record_audio`, and `tool_events` are separate top-level fields. Session length is not settable per session: `max_duration_seconds` comes from the agent's [conversation configuration](/agents/build/configuration#call-duration). ## Response @@ -174,35 +174,35 @@ Everything else on the request is not an override: `dynamic_variables`, `timezon | Field | Description | | ---------------------- | ----------------------------------------------------------------------------------------------------------- | -| `session_id` | The session's id — use it later to look up the [conversation record](/agents/monitor/conversation-history). | +| `session_id` | The session's id. Use it later to look up the [conversation record](/agents/monitor/conversation-history). | | `expires_at` | Deadline for the client to connect. Create the token right before starting, not ahead of time. | | `max_duration_seconds` | Hard cap on session length. | | `transport` | Which transport the SDK uses for this session; `livekit` today. | | `livekit_url`, `token` | Connection details for that transport, consumed by the SDK. | -Treat the response as opaque and pass it to `start()` unmodified. If the SDK does not recognize the `transport` value, it fails fast with an `unsupported_transport` error asking you to upgrade the SDK — it never silently degrades. +Treat the response as opaque and pass it to `start()` unmodified. If the SDK does not recognize the `transport` value, it fails fast with an `unsupported_transport` error asking you to upgrade the SDK. ## Token lifetime - A session token is **single-use**: `start()` consumes it once to establish the conversation. -- On network drops the SDK reconnects at the transport level using the same connection state — it never re-creates the session, so you never need a fresh token mid-call. +- On network drops the SDK reconnects at the transport level; it never re-creates the session, so you never need a fresh token mid-call. - Once a session ends, the token is spent. Create a new token for each conversation. ## Ending sessions from your backend -A session normally ends from the client side — the user disconnects, or the agent [hangs up](/agents/build/system-tools). To force-end a live session server-side, call the end endpoint with your API key and the `session_id` from the creation response: +A session normally ends from the client side: the user disconnects, or the agent [hangs up](/agents/build/system-tools). To force-end a live session server-side, call the end endpoint with your API key and the `session_id` from the creation response: ```bash Request curl --request POST https://api.fish.audio/v1/agent/sessions/$SESSION_ID/end \ --header "Authorization: Bearer $FISH_API_KEY" ``` -The agent disconnects and the call terminates — the response is `204`. The session record stays readable in [conversation history](/agents/monitor/conversation-history), along with the transcript and recording when the agent [stores them](/agents/monitor/conversation-history#what-gets-stored). The same endpoint ends in-progress [phone calls](/agents/telephony/inbound-calls) too. +The call terminates and the response is `204`. The session record stays readable in [conversation history](/agents/monitor/conversation-history), along with the transcript and recording when the agent [stores them](/agents/monitor/conversation-history#what-gets-stored). The same endpoint ends in-progress [phone calls](/agents/telephony/inbound-calls) too. ## Keep API keys on the server - Your API key grants full access to every resource in your team — agents, + Your API key grants full access to every resource in your team: agents, sessions, tools, phone numbers. Never embed it in a browser, mobile app, or any code you ship to users. The only credential that belongs in a client is the session token, and the only credential-free path is a public agent ID. @@ -218,11 +218,11 @@ Session creation fails with standard HTTP statuses; the SDK surfaces them as a ` | `401` | Invalid API key. A request with no `Authorization` header at all is treated as a public-agent request instead. | | `402` | Quota exceeded. | | `403` | Public-agent request rejected: the agent is not public, or the page's `Origin` is not on the allow-list. | -| `409` | Conflict — most commonly the agent has no published version yet. [Publish](/agents/deploy/versions-publishing) to resolve. | -| `422` | The request body failed validation — an unknown field, an unsupported `language` code, an invalid `timezone`, or an invalid dynamic variable. | -| `429` | Rate limited — a public-agent request exceeded the per-agent or per-IP limit. | +| `409` | Conflict: most commonly the agent has no published version yet. [Publish](/agents/deploy/versions-publishing) to resolve. | +| `422` | The request body failed validation: an unknown field, an unsupported `language` code, an invalid `timezone`, or an invalid dynamic variable. | +| `429` | Rate limited: a public-agent request exceeded the per-agent or per-IP limit. | -The complete status-code reference for every `/v1/agent` endpoint — error shapes, `400` vs `422`, conflict semantics — is on [Agents API errors](/api-reference/agent-errors). +The complete status-code reference for every `/v1/agent` endpoint (error shapes, `400` vs `422`, conflict semantics) is on [Agents API errors](/api-reference/agent-errors). ## Going further diff --git a/agents/deploy/overview.mdx b/agents/deploy/overview.mdx index 1c2a774..1bb8343 100644 --- a/agents/deploy/overview.mdx +++ b/agents/deploy/overview.mdx @@ -1,14 +1,14 @@ --- title: "Deploy Your Agent" sidebarTitle: "Overview" -description: "Choose where users meet your agent and who may start sessions — widget, SDKs, or phone; public or authenticated" +description: "Choose where users meet your agent and who may start sessions: widget, SDKs, or phone; public or authenticated" icon: "plug" --- Your agent is one artifact with many front doors. Deploying it is two independent decisions: -1. **Where users meet your agent** — a prebuilt widget, your own UI built on the SDKs, a phone number, or a custom client. -2. **Who may start sessions** — anyone visiting your pages, or only clients your backend has vouched for. +1. **Where users meet your agent**: a prebuilt widget, your own UI built on the SDKs, a phone number, or a custom client. +2. **Who may start sessions**: anyone visiting your pages, or only clients your backend has vouched for. Every surface runs the same agent with the same published configuration, and the browser surfaces work with either access mode. @@ -25,15 +25,15 @@ In increasing order of integration effort: - Drop a prebuilt call widget into your site — two lines of HTML, zero build + Drop a prebuilt call widget into your site: two lines of HTML, zero build step. Build a custom experience with the [Web SDK](/agents/deploy/web-sdk) or - [React SDK](/agents/deploy/react-sdk) — events, transcripts, audio controls. + [React SDK](/agents/deploy/react-sdk): events, transcripts, audio controls. - Bind a number and callers dial straight in — no client code at all. + Bind a number and callers dial straight in. - The client connects with just an `agentId` — no backend, gated by an origin + The client connects with just an `agentId`: no backend, gated by an origin allowlist and rate limits. - Your backend creates a short-lived session token with your API key — trusted + Your backend creates a short-lived session token with your API key: trusted attribution and full control. @@ -85,14 +85,14 @@ The widget and the SDKs each work with **either** access mode — pick per agent ## How a browser session starts -For private agents, your backend creates the session — your API key never ships to the browser. The browser SDK takes the resulting session token and handles audio and transport from there. +For private agents, your backend creates the session. Your API key never ships to the browser. The browser SDK takes the resulting session token and handles audio and transport from there. ```bash Your backend curl --request POST https://api.fish.audio/v1/agent/sessions \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "agent_id": "YOUR_AGENT_ID" }' -# → 201 session token — return it to your frontend as-is +# → 201 session token. Return it to your frontend as-is ``` ```javascript Your frontend @@ -104,11 +104,11 @@ const session = await AgentSession.start({ sessionToken }); Agents marked [public](/agents/deploy/public-agents) skip the backend step - entirely — the SDK connects with just an `agentId`, gated by an origin + entirely: the SDK connects with just an `agentId`, gated by an origin allowlist and rate limits. -For phone calls there is no client integration: assign a number, and inbound calls connect directly to your agent — which can [transfer callers](/agents/telephony/transfers) to another number mid-call. See [Phone numbers](/agents/telephony/phone-numbers). +For phone calls there is no client integration: assign a number, and inbound calls connect directly to your agent, which can [transfer callers](/agents/telephony/transfers) to another number mid-call. See [Phone numbers](/agents/telephony/phone-numbers). ## Going further @@ -125,7 +125,7 @@ For phone calls there is no client integration: assign a number, and inbound cal icon="server" href="/agents/deploy/authenticated-sessions" > - The full session-creation reference — request fields, token lifetime, + The full session-creation reference: request fields, token lifetime, errors. This is an escape hatch. For web and React apps, use the [Web - SDK](/agents/deploy/web-sdk) or [React SDK](/agents/deploy/react-sdk) instead - — they implement everything below (connection, reconnection, message parsing, - tool dispatch) and stay current as the protocol evolves. + SDK](/agents/deploy/web-sdk) or [React SDK](/agents/deploy/react-sdk) + instead. They implement everything below (connection, reconnection, message + parsing, tool dispatch) and stay current as the protocol evolves. ## The protocol package -Every message shape on this page is published as TypeScript definitions in `@fishaudio/agent-protocol` — zero runtime dependencies. This page documents the protocol as of package version **0.1.0**. The package is the source of truth for shapes; this page fixes the semantics. +Every message shape on this page is published as TypeScript definitions in `@fishaudio/agent-protocol`, with zero runtime dependencies. This page documents the protocol as of package version **0.1.0**. The package is the source of truth for shapes; this page fixes the semantics. ```bash npm npm install @fishaudio/agent-protocol @@ -47,13 +47,13 @@ The response is a discriminated union on `transport`. The `livekit` arm carries If you receive a `transport` value you don't recognize, fail with an explicit - "unsupported transport" error. Never guess or silently degrade — new transport - arms may be introduced, each with its own payload. + "unsupported transport" error. New transport arms may be introduced, each + with its own payload. ## Channels -Once connected, a session uses these channels. The two `*-event` topics carry reliable data packets — one complete JSON object per packet. +Once connected, a session uses these channels. The two `*-event` topics carry reliable data packets: one complete JSON object per packet. | Channel | Direction | Carries | | -------------------------------------- | -------------- | --------------------------------------------------------- | @@ -65,7 +65,7 @@ Once connected, a session uses these channels. The two `*-event` topics carry re Data-channel message fields are camelCase; REST bodies are snake_case. -## Agent events — `agent-event` +## Agent events: `agent-event` ### `client_tool.call` @@ -83,7 +83,7 @@ The agent wants to run a [client tool](/agents/build/client-tools) in your app. Run the tool, then publish a `client_tool.result` with the same `callId`. While `expectsResponse` is `true`, the agent suspends the model's tool call until your result arrives or the tool's configured timeout elapses (default 30 seconds, configurable from 1 to 120). When `expectsResponse` is `false`, the call is fire-and-forget: the agent continues immediately and any result you send is ignored. -### Tool lifecycle — `tool.started`, `tool.completed`, `tool.failed` +### Tool lifecycle: `tool.started`, `tool.completed`, `tool.failed` Each tool the agent invokes emits one `tool.started`, resolved by exactly one terminal message (`tool.completed` or `tool.failed`) with the same `callId`. Terminal messages repeat `toolName` and `toolSource`, so a client that missed the start can still render a complete entry. @@ -96,7 +96,7 @@ Each tool the agent invokes emits one `tool.started`, resolved by exactly one te | `inputTruncated` / `outputTruncated` | boolean | Present when the payload was truncated | | `error` | string | `tool.failed` only | -These events are on by default and their payloads travel to the end user's client. Pass `tool_events: false` when creating the session to keep tool data off the wire — the session then receives none of the three. +These events are on by default and their payloads travel to the end user's client. Pass `tool_events: false` when creating the session to keep tool data off the wire: the session then receives none of the three. ### `error` @@ -106,7 +106,7 @@ These events are on by default and their payloads travel to the end user's clien `code` is a coarse category only: `provider_error` (an upstream model or voice provider failed) or `internal_error` (the runtime failed). The message deliberately carries no raw error detail. -## Client events — `client-event` +## Client events: `client-event` Publish these on the `client-event` topic. The agent ignores malformed JSON and unknown types. @@ -130,7 +130,7 @@ Publish these on the `client-event` topic. The agent ignores malformed JSON and } ``` -- `user.message` gets **no server echo** — you already hold the text, so render the bubble locally. Add `"audio": false` to have the agent answer that turn in text only: no speech is synthesized and the reply arrives over transcription. When the field is absent the agent speaks as usual; note the web SDK's `sendUserMessage` sends `"audio": false` unless called with `audio: true`. +- `user.message` gets **no server echo**: you already hold the text, so render the bubble locally. Add `"audio": false` to have the agent answer that turn in text only: no speech is synthesized and the reply arrives over transcription. When the field is absent the agent speaks as usual; note the web SDK's `sendUserMessage` sends `"audio": false` unless called with `audio: true`. - `client_tool.result` may carry `result` (any JSON value) or `"isError": true` to report the tool as failed to the model. Results for unknown or already-settled `callId`s are ignored. ## Transcription and agent state @@ -139,7 +139,7 @@ These ride the transport's built-in mechanisms rather than custom messages. **Transcription** arrives as text streams on the `lk.transcription` topic. Segments are identified by the `lk.segment_id` stream attribute; `lk.transcription_final: "true"` marks a segment as final. Roles are distinguished by sender identity: the user's segments are sent under the user's own participant identity, the agent's under the agent participant. -- Agent segments stream incrementally, paced to audio playback. An interrupted segment closes containing only the words actually spoken — there is no residual text. +- Agent segments stream incrementally, paced to audio playback. An interrupted segment closes containing only the words actually spoken; there is no residual text. - User segments are interim until final; each interim update **replaces the entire segment text** under the same segment id. **Agent state** is published as the sticky `lk.agent.state` participant attribute with values `initializing`, `idle`, `listening`, `thinking`, and `speaking`. Sticky means a client that connects late or reconnects reads the current value immediately. The SDKs derive their three public modes from this attribute plus transcript segment open/close. @@ -148,13 +148,13 @@ These ride the transport's built-in mechanisms rather than custom messages. 1. **Ignore unknown `type` values and unknown fields.** This is required consumer behavior and the foundation of forward compatibility. 2. **Evolution is additive-only.** Published fields never change name or meaning and are never removed; new fields are always optional. A semantic change ships as a new `type`. -3. **No replay.** Data-channel delivery is reliable and ordered within a connection, but after a reconnect or late join, missed messages are gone — never wait for history. Tool terminal messages repeat their identifying fields, and the state attribute is sticky, precisely to soften this. +3. **No replay.** Data-channel delivery is reliable and ordered within a connection, but after a reconnect or late join, missed messages are gone. Never wait for history. Tool terminal messages repeat their identifying fields, and the state attribute is sticky, precisely to soften this. ## Version history | Version | Changes | | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | -| 0.0.1 | Initial public release — the complete message set on this page, including tool lifecycle events and the optional `audio` flag on `user.message` | +| 0.0.1 | Initial public release: the complete message set on this page, including tool lifecycle events and the optional `audio` flag on `user.message` | | 0.1.0 (current) | Revised the session-creation `SessionOverrides` shape; realtime messages unchanged | ## Going further diff --git a/agents/deploy/public-agents.mdx b/agents/deploy/public-agents.mdx index 626aaea..8ec952e 100644 --- a/agents/deploy/public-agents.mdx +++ b/agents/deploy/public-agents.mdx @@ -1,12 +1,12 @@ --- title: "Public Agents" -description: "Let visitors talk to your agent directly from the browser — no backend required" +description: "Let visitors talk to your agent directly from the browser, no backend required" icon: "globe" --- -A public agent accepts sessions straight from the browser: the SDK calls the session endpoint with just an `agent_id` — no API key, no server of your own. The platform gates access with three controls: an agent-level public switch (off by default), a required origin allowlist, and rate limiting. +A public agent accepts sessions straight from the browser: the SDK calls the session endpoint with just an `agent_id`, no API key and no server of your own. The platform gates access with three controls: an agent-level public switch (off by default), a required origin allowlist, and rate limiting. -Use this mode for demos, marketing pages, and support bubbles where standing up a backend isn't worth it. For production apps with their own users, [authenticated sessions](/agents/deploy/authenticated-sessions) are usually the better fit — see [when to use session tokens instead](#when-to-use-session-tokens-instead). +Use this mode for demos, marketing pages, and support bubbles where standing up a backend isn't worth it. For production apps with their own users, [authenticated sessions](/agents/deploy/authenticated-sessions) are usually the better fit. See [when to use session tokens instead](#when-to-use-session-tokens-instead). ## Enable public access @@ -18,7 +18,7 @@ Use this mode for demos, marketing pages, and support bubbles where standing up On the agent's **Deploy** page in the console, turn on **Public access**. It - is **off by default** — until you enable it, sessions require + is **off by default**. Until you enable it, sessions require authentication. @@ -31,7 +31,7 @@ Use this mode for demos, marketing pages, and support bubbles where standing up ## Start a session from the browser -Pass `agentId` to the SDK and it creates the session directly with the Fish Audio API, then connects the audio — your page never handles credentials. +Pass `agentId` to the SDK and it creates the session directly with the Fish Audio API, then connects the audio. @@ -46,7 +46,7 @@ session.on("agentResponse", ({ text }) => console.log("Agent:", text)); ``` ```bash API (curl) -# No Authorization header — the Origin header is checked instead. +# No Authorization header. The Origin header is checked instead. # Browsers send Origin automatically; curl must set it explicitly. curl --request POST https://api.fish.audio/v1/agent/sessions \ --header "Content-Type: application/json" \ @@ -56,19 +56,19 @@ curl --request POST https://api.fish.audio/v1/agent/sessions \ -The response is the same session token an authenticated request returns — the SDK consumes it internally. Everything else (events, methods, client tools) works exactly as in the [Web SDK](/agents/deploy/web-sdk). +The response is the same session token an authenticated request returns. The SDK consumes it internally. Everything else (events, methods, client tools) works exactly as in the [Web SDK](/agents/deploy/web-sdk). ## Origin matching -An entry matches only on **exact scheme + host + port**. The host is case-insensitive, and a trailing slash on an entry is tolerated — everything else must match exactly. +An entry matches only on **exact scheme + host + port**. The host is case-insensitive, and a trailing slash on an entry is tolerated. | Allowlist entry | Browser origin | Allowed | | ------------------------- | ------------------------------ | ------------------------------ | | `https://app.example.com` | `https://app.example.com` | Yes | -| `https://app.example.com` | `https://APP.example.com` | Yes — host is case-insensitive | -| `https://app.example.com` | `http://app.example.com` | No — scheme differs | -| `https://app.example.com` | `https://app.example.com:8443` | No — port differs | -| `http://localhost:5173` | `http://127.0.0.1:5173` | No — different hosts | +| `https://app.example.com` | `https://APP.example.com` | Yes (host is case-insensitive) | +| `https://app.example.com` | `http://app.example.com` | No (scheme differs) | +| `https://app.example.com` | `https://app.example.com:8443` | No (port differs) | +| `http://localhost:5173` | `http://127.0.0.1:5173` | No (different hosts) | `localhost` and `127.0.0.1` are **different origins**. If you develop against both, list both: @@ -86,15 +86,15 @@ Public session creation is rate limited on two dimensions at once: per **client If legitimate traffic outgrows these limits, or you need per-user quotas, move - to [authenticated sessions](/agents/deploy/authenticated-sessions) — your + to [authenticated sessions](/agents/deploy/authenticated-sessions): your backend becomes the gate, and the platform-side public limits no longer apply. ## Treat public input as untrusted -On the public path, the entire session request originates in the visitor's browser. The platform enforces its own guardrails — public sessions accept only the `language` and `voice_id` [overrides](/agents/deploy/authenticated-sessions#overrides), rejecting everything else with `400` — but anything that passes through verbatim is attacker-controllable: +On the public path, the entire session request originates in the visitor's browser. The platform enforces its own guardrails (public sessions accept only the `language` and `voice_id` [overrides](/agents/deploy/authenticated-sessions#overrides), rejecting everything else with `400`), but anything that passes through verbatim is attacker-controllable: -- **`metadata`** is stored and returned exactly as sent, never interpreted by the platform. When you read it back in [conversation history](/agents/monitor/conversation-history) or [webhooks](/agents/monitor/webhooks), treat it as untrusted data — never as proof of who the visitor is. +- **`metadata`** is stored and returned exactly as sent, never interpreted by the platform. When you read it back in [conversation history](/agents/monitor/conversation-history) or [webhooks](/agents/monitor/webhooks), treat it as untrusted data, never as proof of who the visitor is. - **`dynamic_variables`** are chosen by the page that starts the session. Don't inject anything through them that the visitor shouldn't control. See [Dynamic variables](/agents/build/dynamic-variables). For trusted attribution (a verified `end_user_id`, server-set metadata), create sessions from your backend instead. @@ -107,18 +107,18 @@ Public-mode failures surface as `FishAgentError` codes on `AgentSession.start()` | ------------------------ | -------------------------------------------------------------------------- | | `agent_not_public` | The agent's public switch is off. | | `origin_forbidden` | The page's origin is not in `allowed_origins` (HTTP `403`). | -| `session_request_failed` | Other creation failures — for example, the agent has no published version. | +| `session_request_failed` | Other creation failures, for example the agent has no published version. | ## When to use session tokens instead -Public mode trades control for zero setup. Prefer the `sessionToken` flow — your backend calls the session endpoint with an API key and hands the token to the browser — when any of these apply: +Public mode trades control for zero setup. Prefer the `sessionToken` flow (your backend calls the session endpoint with an API key and hands the token to the browser) when any of these apply: - The agent should stay **private** rather than callable by anyone who loads your page. - You need **trusted attribution**: an `end_user_id` and `metadata` set by your server, not the browser. - You want server-side control of `overrides`, `dynamic_variables`, or `tool_events` instead of accepting the browser's values. - You already authenticate users and want **your own quotas** per account, not IP-based platform limits. -The SDK call changes by one field — pass `{ sessionToken }` instead of `{ agentId }`. See [Authenticated sessions](/agents/deploy/authenticated-sessions) for the full flow. +The SDK call changes by one field: pass `{ sessionToken }` instead of `{ agentId }`. See [Authenticated sessions](/agents/deploy/authenticated-sessions) for the full flow. ## Going further @@ -128,10 +128,10 @@ The SDK call changes by one field — pass `{ sessionToken }` instead of `{ agen icon="server" href="/agents/deploy/authenticated-sessions" > - The session token flow in full — request fields, lifetime, errors. + The session token flow in full: request fields, lifetime, errors. - Events, methods, and client tools — identical in both modes. + Events, methods, and client tools, identical in both modes. Hooks and components for React apps. diff --git a/agents/deploy/react-sdk.mdx b/agents/deploy/react-sdk.mdx index ecdc807..bce7261 100644 --- a/agents/deploy/react-sdk.mdx +++ b/agents/deploy/react-sdk.mdx @@ -4,7 +4,7 @@ description: "Add voice conversations to your React app with hooks and drop-in c icon: "react" --- -`@fishaudio/agent-react` wraps the [Web SDK](/agents/deploy/web-sdk) in idiomatic React: a `useConversation` hook for session control, an optional provider that shares one session across your component tree, and a ready-made audio visualizer. The SDK handles microphone capture, audio playback, and transport internally — you write UI. +`@fishaudio/agent-react` wraps the [Web SDK](/agents/deploy/web-sdk) in idiomatic React: a `useConversation` hook for session control, an optional provider that shares one session across your component tree, and a ready-made audio visualizer. The SDK handles microphone capture, audio playback, and transport internally. @@ -86,7 +86,7 @@ export function CallButton() { } ``` -`status`, `mode`, and `isSpeaking` are React state — your component re-renders as the conversation progresses. When the component unmounts, the session ends automatically. +`status`, `mode`, and `isSpeaking` are React state, so your component re-renders as the conversation progresses. When the component unmounts, the session ends automatically. Call `startSession` from a user gesture (such as a click handler) so the @@ -121,14 +121,14 @@ See [Authenticated sessions](/agents/deploy/authenticated-sessions) for the back | `session` | The live `AgentSession` object (`null` before the first start) for direct event access | -`sendUserMessage(text)` injects a typed turn; the agent replies in text only by default — the reply streams back as transcript with no speech. Pass `{ audio: true }` to have the agent speak its reply for that turn. +`sendUserMessage(text)` injects a typed turn; the agent replies in text only by default; the reply streams back as transcript. Pass `{ audio: true }` to have the agent speak its reply for that turn. -For transcripts, tool-call events, and error handling, subscribe to events on `session` — see the [Web SDK event reference](/agents/deploy/web-sdk). +For transcripts, tool-call events, and error handling, subscribe to events on `session`. See the [Web SDK event reference](/agents/deploy/web-sdk). ## Share one session across components -Wrap your tree in `AgentSessionProvider` when several components need the same conversation — call controls in the header, a transcript panel elsewhere. +Wrap your tree in `AgentSessionProvider` when several components need the same conversation: call controls in the header, a transcript panel elsewhere. ```tsx App.tsx import { AgentSessionProvider, useAgentMessages } from "@fishaudio/agent-react"; @@ -157,17 +157,17 @@ function Transcript() { } ``` -The provider hosts the conversation, so components inside it read the shared state with `useAgentSessionContext()` instead of calling `useConversation` themselves — it returns the same fields, so the quick-start `CallButton` only needs its hook call swapped. +The provider hosts the conversation, so components inside it read the shared state with `useAgentSessionContext()` instead of calling `useConversation` themselves. It returns the same fields, so the quick-start `CallButton` only needs its hook call swapped. | Hook | Purpose | | -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | | `useAgentSessionContext()` | Access the active session from anywhere inside the provider | -| `useAgentMessages()` | Live conversation as a list of `{ key, role, text, final }` messages — segments update in place as they stream, no manual aggregation | +| `useAgentMessages()` | Live conversation as a list of `{ key, role, text, final }` messages: segments update in place as they stream, no manual aggregation | | `useAudioLevels(session?, fps?)` | Live input and output volume as `{ input, output }` (0–1), polled `fps` times per second (default 20); lower `fps` to reduce re-renders | ## Audio visualizer -`` renders animated canvas bars driven by the agent's output audio — a drop-in "the agent is talking" indicator. Inside an `AgentSessionProvider` it picks up the active session automatically; elsewhere, pass a `session` prop. Optional `bars`, `width`, `height`, and `className` props control the rendering, and the bars follow the element's CSS `color`. +`` renders animated canvas bars driven by the agent's output audio, a drop-in "the agent is talking" indicator. Inside an `AgentSessionProvider` it picks up the active session automatically; elsewhere, pass a `session` prop. Optional `bars`, `width`, `height`, and `className` props control the rendering, and the bars follow the element's CSS `color`. ```tsx import { AgentAudioVisualizer } from "@fishaudio/agent-react"; diff --git a/agents/deploy/versions-publishing.mdx b/agents/deploy/versions-publishing.mdx index 8c73010..924ef3f 100644 --- a/agents/deploy/versions-publishing.mdx +++ b/agents/deploy/versions-publishing.mdx @@ -4,7 +4,7 @@ description: "Edit safely in a draft, then publish an immutable version that liv icon: "code-branch" --- -Every agent has one **draft** and a linear history of **published versions**. Edits in the Builder or via the API always land in the draft — production sessions never see them until you publish. Publishing snapshots the draft into an immutable, numbered version, so you can iterate freely without touching live traffic. +Every agent has one **draft** and a linear history of **published versions**. Edits in the Builder or via the API always land in the draft. Production sessions never see them until you publish. Publishing snapshots the draft into an immutable, numbered version. - Prompt, voice, and conversation settings — all autosaved. + Prompt, voice, and conversation settings, all autosaved. - An agent that has never been published cannot take production sessions — - session creation returns `409` until the first publish. Publishing is the - single gate between editing and live traffic. + An agent that has never been published cannot take production sessions: + session creation returns `409` until the first publish. ## Publish from the Builder - - There is no Save button. Every change is written to the draft automatically; + + Every change is written to the draft automatically; the **Saving… / Draft saved** indicator in the top bar shows the current state. @@ -65,7 +64,7 @@ Every agent has one **draft** and a linear history of **published versions**. Ed -New sessions use the newly published version from that point on. To hear a change before it goes live, run a [preview call](/agents/test/preview-calls) — previews always use the draft. +New sessions use the newly published version from that point on. To hear a change before it goes live, run a [preview call](/agents/test/preview-calls). Previews always use the draft. ## Publish via the API @@ -91,7 +90,7 @@ The publish response includes the new `version_number`. ## Version history -List published versions, or fetch the complete configuration snapshot of any one of them — including the version currently serving production: +List published versions, or fetch the complete configuration snapshot of any one of them, including the version currently serving production: ```bash List versions curl "https://api.fish.audio/v1/agent/agents/$AGENT_ID/versions" \ @@ -116,8 +115,8 @@ Each version record includes: Fetching a single version returns the full configuration as it was published. Because `GET .../config` always shows the draft, the snapshot endpoint is how you read back what is actually running in production. - Version snapshots never echo credential secrets. Write-only fields — such as - webhook secrets and tool authorization headers — return a `has_secret: true` + Version snapshots never echo credential secrets. Write-only fields (such as + webhook secrets and tool authorization headers) return a `has_secret: true` marker instead of the value, and the custom LLM [`api_key`](/agents/build/custom-llm) reads back as `null`, in every version including historical ones. @@ -139,13 +138,13 @@ curl --request PATCH "https://api.fish.audio/v1/agent/agents/$AGENT_ID/config" \ --data '{ "prompt": { "system_prompt": "" } }' ``` -Patching updates the draft — it does **not** publish. Review or [preview](/agents/test/preview-calls) the restored draft, then publish it as a new version; the rollback becomes part of the linear history, so the audit trail stays intact. Because snapshots never echo secrets (see above), credential fields keep their current draft values unless you set them again explicitly. +Patching updates the draft. It does **not** publish. Review or [preview](/agents/test/preview-calls) the restored draft, then publish it as a new version; the rollback becomes part of the linear history, so the audit trail stays intact. Because snapshots never echo secrets (see above), credential fields keep their current draft values unless you set them again explicitly. ## Clone an agent Cloning creates a new agent from an existing one's **draft** configuration. In the console, open the agents list and choose **Clone** from the agent's row menu. -The clone starts with a copy of the source's current draft. Published versions do not carry over — the new agent is unpublished until you publish it yourself. Use clones to template a base configuration across many agents, or to experiment without risking an agent that is already live. +The clone starts with a copy of the source's current draft. Published versions do not carry over: the new agent is unpublished until you publish it yourself. Use clones to template a base configuration across many agents, or to experiment without risking an agent that is already live. ## Going further diff --git a/agents/deploy/web-sdk.mdx b/agents/deploy/web-sdk.mdx index cf21d0e..3da0883 100644 --- a/agents/deploy/web-sdk.mdx +++ b/agents/deploy/web-sdk.mdx @@ -1,10 +1,10 @@ --- title: "Web SDK" -description: "Voice sessions in the browser with @fishaudio/agent-client — events, transcripts, text input, audio controls, and client tools" +description: "Voice sessions in the browser with @fishaudio/agent-client: events, transcripts, text input, audio controls, and client tools" icon: "js" --- -`@fishaudio/agent-client` runs a live voice conversation with your agent from any web page: open the microphone, stream audio both ways, and react to typed events for transcripts, agent state, and tool calls. The SDK handles the realtime transport (WebRTC) internally — your code never touches connection plumbing. +`@fishaudio/agent-client` runs a live voice conversation with your agent from any web page: open the microphone, stream audio both ways, and react to typed events for transcripts, agent state, and tool calls. The SDK handles the realtime transport (WebRTC) internally. Using React? [`@fishaudio/agent-react`](/agents/deploy/react-sdk) wraps this SDK in hooks and a provider. @@ -29,8 +29,8 @@ yarn add @fishaudio/agent-client `AgentSession.start()` creates the session, connects, and opens the microphone in one call. Authenticate one of two ways: -- **`agentId`** — for [public agents](/agents/deploy/public-agents). The SDK creates the session directly from the browser; no backend needed. -- **`sessionToken`** — for private agents. Your backend calls `POST /v1/agent/sessions` with your API key and hands the JSON response to the browser; pass it through unchanged. See [Authenticated sessions](/agents/deploy/authenticated-sessions). +- **`agentId`**: for [public agents](/agents/deploy/public-agents). The SDK creates the session directly from the browser; no backend needed. +- **`sessionToken`**: for private agents. Your backend calls `POST /v1/agent/sessions` with your API key and hands the JSON response to the browser; pass it through unchanged. See [Authenticated sessions](/agents/deploy/authenticated-sessions). @@ -46,7 +46,7 @@ const session = await AgentSession.start({ import { AgentSession } from "@fishaudio/agent-client"; // Your backend calls POST /v1/agent/sessions and returns the -// response body. Pass it to the SDK as-is — no reshaping. +// response body. Pass it to the SDK as-is. const resp = await fetch("/api/voice-session", { method: "POST" }); const sessionToken = await resp.json(); @@ -61,14 +61,14 @@ const session = await AgentSession.start({ sessionToken }); | -------------------------- | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `agentId` / `sessionToken` | — | One of the two, required | | `clientTools` | `Record` | Handlers for [client tools](/agents/build/client-tools) declared on the agent | -| `overrides` | `SessionOverrides` | Per-session config [overrides](/agents/deploy/authenticated-sessions#overrides) (`agentId` mode only — with `sessionToken`, your backend sends them when creating the session). Keyless sessions accept only `voice_id` and `language`; the prompt-shaping fields are rejected with `400` | -| `dynamicVariables` | `Record` | Values for `{{placeholders}}` in the agent config — see [Dynamic variables](/agents/build/dynamic-variables) | +| `overrides` | `SessionOverrides` | Per-session config [overrides](/agents/deploy/authenticated-sessions#overrides) (`agentId` mode only; with `sessionToken`, your backend sends them when creating the session). Keyless sessions accept only `voice_id` and `language`; the prompt-shaping fields are rejected with `400` | +| `dynamicVariables` | `Record` | Values for `{{placeholders}}` in the agent config; see [Dynamic variables](/agents/build/dynamic-variables) | | `language` | `string` | Shorthand for overriding the agent's language | -| `toolEvents` | `boolean` | Whether tool lifecycle events reach this client (default `true`; `agentId` mode only — with `sessionToken`, your backend sets `tool_events`) | -| `timezone` | `string` | IANA timezone for the agent's sense of local time — the top of the [resolution order](/agents/build/time-timezone), overriding the agent's configured timezone. When omitted, the SDK still sends the browser timezone as a lower-priority hint (`client_timezone`), which applies only if the agent has no timezone configured | -| `worldContext` | `boolean` | Whether the agent knows the current date and time (default `true`; `agentId` mode only — with `sessionToken`, your backend sets `world_context`) | +| `toolEvents` | `boolean` | Whether tool lifecycle events reach this client (default `true`; `agentId` mode only, with `sessionToken` your backend sets `tool_events`) | +| `timezone` | `string` | IANA timezone for the agent's sense of local time: the top of the [resolution order](/agents/build/time-timezone), overriding the agent's configured timezone. When omitted, the SDK still sends the browser timezone as a lower-priority hint (`client_timezone`), which applies only if the agent has no timezone configured | +| `worldContext` | `boolean` | Whether the agent knows the current date and time (default `true`; `agentId` mode only, with `sessionToken` your backend sets `world_context`) | | `audio` | `{ inputDeviceId?, outputDeviceId? }` | Pick specific microphone and output devices | -| `callbacks` | `Partial` | Shorthand — each key is auto-subscribed via `.on()` | +| `callbacks` | `Partial` | Shorthand: each key is auto-subscribed via `.on()` | ## Session lifecycle @@ -84,14 +84,14 @@ When the session ends, `disconnect` fires with a reason (also available as `sess | `EndReason` | Meaning | | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `user_hangup` | You called `session.end()` | -| `agent_hangup` | The server ended the session — the agent hung up (for example via the hang-up [system tool](/agents/build/system-tools)), the session hit its maximum duration, or it was force-ended; the protocol does not distinguish these | +| `agent_hangup` | The server ended the session: the agent hung up (for example via the hang-up [system tool](/agents/build/system-tools)), the session hit its maximum duration, or it was force-ended; the protocol does not distinguish these | | `connection_lost` | The connection dropped and could not be recovered | Brief network drops don't end the session: the SDK moves to `reconnecting` and back to `connected` automatically, reusing the same session. Events are not replayed after a reconnect. Anything emitted while you were in - `reconnecting` — transcript updates, tool events, errors — is dropped, not + `reconnecting` (transcript updates, tool events, errors) is dropped, not resent. Design your UI to tolerate gaps: tool `toolCallCompleted` / `toolCallFailed` events repeat the tool name and source, so a terminal event still renders even if you missed `toolCallStarted`. @@ -99,7 +99,7 @@ Brief network drops don't end the session: the SDK moves to `reconnecting` and b ## Events -The session is a typed event emitter — subscribe with `session.on(event, handler)`, remove with `off`, or use `once`. +The session is a typed event emitter: subscribe with `session.on(event, handler)`, remove with `off`, or use `once`. | Event | Payload | Fires when | | -------------------- | ------------------------------------------------------- | --------------------------------------------------------------------------- | @@ -108,13 +108,13 @@ The session is a typed event emitter — subscribe with `session.on(event, handl | `statusChange` | `SessionStatus` | Status transitions (`connecting` / `connected` / `reconnecting` / `ended`) | | `modeChange` | `AgentMode` | The agent switches between `listening`, `thinking`, and `speaking` | | `userTranscript` | `{ segmentId, text, final }` | The user's speech is transcribed; interim updates replace the whole segment | -| `agentResponseDelta` | `{ segmentId, delta, text }` | The agent speaks — `delta` is the new text, `text` the segment so far | +| `agentResponseDelta` | `{ segmentId, delta, text }` | The agent speaks: `delta` is the new text, `text` the segment so far | | `agentResponse` | `{ segmentId, text }` | An agent segment is finalized | -| `message` | `{ role: "user" \| "agent", text }` | A finalized message from either side — a ready-made chat feed | +| `message` | `{ role: "user" \| "agent", text }` | A finalized message from either side, a ready-made chat feed | | `toolCallStarted` | `{ callId, toolName, source, input, inputTruncated }` | A tool call begins; `input` is a JSON string (truncated at 4 KB) | | `toolCallCompleted` | `{ callId, toolName, source, output, outputTruncated }` | A tool call succeeds | | `toolCallFailed` | `{ callId, toolName, source, error }` | A tool call fails | -| `error` | `FishAgentError` | A session or tool error occurs — see [Errors](#errors) | +| `error` | `FishAgentError` | A session or tool error occurs; see [Errors](#errors) | ```javascript Subscribe to events session.on("userTranscript", ({ segmentId, text, final }) => { @@ -131,11 +131,11 @@ session.on("disconnect", ({ reason }) => showCallEnded(reason)); ### Transcript semantics -Transcripts on both sides arrive as **segments** — one segment per utterance or response, identified by `segmentId`: +Transcripts on both sides arrive as **segments**, one segment per utterance or response, identified by `segmentId`: - **User segments**: interim results **replace the entire segment text** (they never append). Render by upserting on `segmentId`; `final: true` marks the segment as finalized. -- **Agent segments**: text streams in sync with audio playback — what you display matches what the user has actually heard. If the agent is interrupted, the segment finalizes containing only the words that were spoken. -- The `message` event delivers only finalized messages from both sides, in order — use it when you want a simple transcript list without handling interim updates. +- **Agent segments**: text streams in sync with audio playback: what you display matches what the user has actually heard. If the agent is interrupted, the segment finalizes containing only the words that were spoken. +- The `message` event delivers only finalized messages from both sides, in order. Use it when you want a simple transcript list without handling interim updates. ## Agent modes @@ -143,18 +143,18 @@ Transcripts on both sides arrive as **segments** — one segment per utterance o | Mode | Meaning | | ----------- | ------------------------------------------------------------ | -| `listening` | Default state — the agent is waiting for or hearing the user | +| `listening` | Default state: the agent is waiting for or hearing the user | | `thinking` | The agent is preparing a response | | `speaking` | Agent audio is playing; ends when playback finishes | -`session.isSpeaking` is a convenience boolean for the `speaking` mode. Mode does not react to the user's own speech — for instant "the mic hears you" feedback, poll `getInputVolume()` locally. +`session.isSpeaking` is a convenience boolean for the `speaking` mode. Mode does not react to the user's own speech. For instant "the mic hears you" feedback, poll `getInputVolume()` locally. ## Send text Users can type instead of talking, in the same session: ```javascript Text input -// Send a typed user turn. There is no server echo — the SDK emits +// Send a typed user turn. There is no server echo; the SDK emits // the finalized `message` event locally from the text you passed. // By default the agent answers a typed turn in text only // (agentResponseDelta / agentResponse) without speaking out loud. @@ -170,7 +170,7 @@ input.addEventListener("input", () => session.sendUserActivity()); session.interrupt(); ``` -Text-only replies are not paced to audio playback — the transcript streams as fast as it generates. +Text-only replies are not paced to audio playback: the transcript streams as fast as it generates. ## Audio controls @@ -178,9 +178,9 @@ Text-only replies are not paced to audio playback — the transcript streams as | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------- | | `setMicMuted(muted)` / `micMuted` | Mute or unmute the microphone | | `setOutputVolume(v)` | Set playback volume, `0`–`1` | -| `getInputVolume()` / `getOutputVolume()` | Current mic / agent volume, `0`–`1` — poll per frame | +| `getInputVolume()` / `getOutputVolume()` | Current mic / agent volume, `0`–`1`; poll per frame | | `getInputFrequencyData()` / `getOutputFrequencyData()` | FFT data as `Uint8Array`, for visualizers | -| `startAudio()` | Unlock playback under browser autoplay policies — call inside a user gesture (for example the click that starts the call) | +| `startAudio()` | Unlock playback under browser autoplay policies; call inside a user gesture (for example the click that starts the call) | Select specific devices at start with the `audio` option (`inputDeviceId` / `outputDeviceId`). @@ -198,7 +198,7 @@ session.on("disconnect", () => cancelAnimationFrame(rafId)); ## Client tools -Register handlers for tools of type `client` declared on the agent — the agent calls them mid-conversation and your return value goes back to the model: +Register handlers for tools of type `client` declared on the agent. The agent calls them mid-conversation and your return value goes back to the model: ```javascript Register a client tool const session = await AgentSession.start({ @@ -219,23 +219,23 @@ Handlers can be sync or async; a thrown error or a timeout (default 15 s) is ret ## Errors -Failures surface as `FishAgentError` — thrown from `AgentSession.start()` when the session can't be created, emitted on the `error` event otherwise. Each carries a `code`, an optional `statusCode` (set on session-creation HTTP errors), and `cause`. +Failures surface as `FishAgentError`: thrown from `AgentSession.start()` when the session can't be created, emitted on the `error` event otherwise. Each carries a `code`, an optional `statusCode` (set on session-creation HTTP errors), and `cause`. | Code | Meaning | | ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `session_request_failed` | Session creation failed — check `statusCode` | +| `session_request_failed` | Session creation failed; check `statusCode` | | `agent_not_public` | `agentId` mode, but the agent is not enabled for public access | | `origin_forbidden` | The page's origin is not in the agent's allowed origins | -| `unsupported_transport` | The session token uses a transport this SDK version doesn't know — upgrade the SDK | +| `unsupported_transport` | The session token uses a transport this SDK version doesn't know; upgrade the SDK | | `mic_permission_denied` | The user denied microphone access; the SDK ends the session | -| `device_change_failed` | A requested audio device could not be activated, or the browser doesn't support selecting it (output selection is unsupported on some mobile browsers) — thrown from `start()` with `audio.outputDeviceId` set, or from device-switching calls | +| `device_change_failed` | A requested audio device could not be activated, or the browser doesn't support selecting it (output selection is unsupported on some mobile browsers). Thrown from `start()` with `audio.outputDeviceId` set, or from device-switching calls | | `connection_failed` | Could not establish the realtime connection | | `session_expired` | The session token's join deadline passed before connecting | | `tool_failed` / `tool_timeout` | A client tool handler threw or timed out | -| `provider_error` / `internal_error` | The session failed server-side — upstream model/voice provider vs. platform runtime | +| `provider_error` / `internal_error` | The session failed server-side: upstream model/voice provider vs. platform runtime | - `provider_error` and `internal_error` carry only the category code — raw + `provider_error` and `internal_error` carry only the category code. Raw provider or infrastructure details are never sent to the browser. diff --git a/agents/deploy/widget.mdx b/agents/deploy/widget.mdx index 2219bf0..003a00a 100644 --- a/agents/deploy/widget.mdx +++ b/agents/deploy/widget.mdx @@ -1,15 +1,15 @@ --- title: "Widget" -description: "Embed a complete voice and chat UI on any website with two lines of HTML — no build step" +description: "Embed a complete voice and chat UI on any website with two lines of HTML, no build step" icon: "puzzle-piece" --- -The widget is the zero-code way to put your agent on a website. A `` custom element renders the complete experience — a floating launcher that expands into a voice-first chat card with live transcript, typing during the call, and inline tool activity — and a single script tag registers it. Under the hood it runs the same sessions as the [Web SDK](/agents/deploy/web-sdk), so everything downstream (history, analysis, webhooks) works unchanged. +The widget is the zero-code way to put your agent on a website. A `` custom element renders the complete experience (a floating launcher that expands into a voice-first chat card with live transcript, typing during the call, and inline tool activity), and a single script tag registers it. Under the hood it runs the same sessions as the [Web SDK](/agents/deploy/web-sdk), so everything downstream (history, analysis, webhooks) works unchanged. ## Prerequisites - An agent with a [published version](/agents/deploy/versions-publishing). -- **Public access** enabled on the agent, with your site's origin on the allowed-origins list — see [Public agents](/agents/deploy/public-agents). `localhost` and `127.0.0.1` count as different origins. +- **Public access** enabled on the agent, with your site's origin on the allowed-origins list. See [Public agents](/agents/deploy/public-agents). `localhost` and `127.0.0.1` count as different origins. - To keep the agent private instead, skip public access and supply session tokens from your backend with [`sessionTokenProvider`](#private-agents). ## Two-line embed @@ -29,7 +29,7 @@ Add the element and the script anywhere on the page: ## Install from npm -Bundlers can install the element instead: `npm install @fishaudio/agent-widget`, then call `registerWidget()` once. Importing the package has no side effects — registration happens only when you call it. +Bundlers can install the element instead: `npm install @fishaudio/agent-widget`, then call `registerWidget()` once. Importing the package has no side effects. ```javascript import { registerWidget } from "@fishaudio/agent-widget"; @@ -38,7 +38,7 @@ registerWidget(); // defines ### React -React apps get a real component: `` registers and renders the element with camelCase props, object props serialized for you, and the [page events](#page-events) as callback props — `clientTools` is just a prop: +React apps get a real component: `` registers and renders the element with camelCase props, object props serialized for you, and the [page events](#page-events) as callback props. `clientTools` is just a prop: ```tsx import { FishAgentWidget } from "@fishaudio/agent-widget/react"; @@ -58,7 +58,7 @@ Every attribute below has a camelCase prop; `dynamicVariables` and `textContents | Attribute | Description | | --------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | -| `agent-id` | Public agent ID. Required unless the `sessionTokenProvider` property is set — see [Private agents](#private-agents). | +| `agent-id` | Public agent ID. Required unless the `sessionTokenProvider` property is set. See [Private agents](#private-agents). | | `agent-name` | Display name in the header. | | `greeting` | Home-screen headline. | | `proactive-message` | Enables the attention bubble next to the launcher. | @@ -67,7 +67,7 @@ Every attribute below has a camelCase prop; `dynamicVariables` and `textContents | `consent` | `"true"` shows a first-run terms card (default off). Acceptance is remembered in `localStorage`. | | `consent-text`, `terms-url`, `privacy-url`, `consent-key` | Consent copy, linked policies, and the `localStorage` key (default `fish-agent-consent`). | | `position` | `bottom-right` (default), `bottom-left`, `top-right`, `top-left`. | -| `language` | Pin the session language — see [Overrides](/agents/deploy/authenticated-sessions#overrides). | +| `language` | Pin the session language. See [Overrides](/agents/deploy/authenticated-sessions#overrides). | | `dynamic-variables` | JSON object of `{{name}}` template values. See [Dynamic variables](/agents/build/dynamic-variables). | | `user-id` | Your end-user identifier, stored on the session. | | `server-url` | Fish API base override. Default `https://api.fish.audio`. | @@ -75,7 +75,7 @@ Every attribute below has a camelCase prop; `dynamicVariables` and `textContents ## Private agents -Keep the agent non-public and set `sessionTokenProvider` instead of an `agent-id`. It's a JS property on the element (functions can't be attributes), called before every session start: fetch the session token from your backend — with whatever auth headers, payload, or credentials the request needs — and return the JSON; it's used verbatim. +Keep the agent non-public and set `sessionTokenProvider` instead of an `agent-id`. It's a JS property on the element (functions can't be attributes), called before every session start: fetch the session token from your backend (with whatever auth headers, payload, or credentials the request needs) and return the JSON; it's used verbatim. ```html @@ -100,7 +100,7 @@ Your backend holds the API key and creates the session with `POST /v1/agent/sess ## Theming -Set CSS custom properties on the element. The widget's internals live in a shadow root — page CSS can't leak in, but every `--fish-*` token is public: +Set CSS custom properties on the element. The widget's internals live in a shadow root: page CSS can't leak in, but every `--fish-*` token is public: ```css fish-agent { @@ -130,14 +130,14 @@ The element dispatches `CustomEvent`s (bubbling, composed): | Event | `detail` | When | | ----------------------- | -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | -| `fish-agent:call` | `{ options }` — **mutable** `AgentSession.start` options | Right before a session starts. Mutate `detail.options` to inject `clientTools`, `overrides`, anything. | +| `fish-agent:call` | `{ options }`: **mutable** `AgentSession.start` options | Right before a session starts. Mutate `detail.options` to inject `clientTools`, `overrides`, anything. | | `fish-agent:connect` | `{ sessionId }` | Session established. | | `fish-agent:disconnect` | `{ reason }` | Session ended. | | `fish-agent:error` | `{ code, message }` | Start failure or in-call error. | Inbound: dispatch `fish-agent:expand` on the element or `document` to open the panel programmatically. -Registering [client tools](/agents/build/client-tools) is just the `:call` event (React apps pass the `clientTools` prop instead — same injection, wrapped): +Registering [client tools](/agents/build/client-tools) is just the `:call` event (React apps pass the `clientTools` prop instead; same injection, wrapped): ```javascript document @@ -151,7 +151,7 @@ document ## Console settings and precedence -On load, the widget anonymously fetches the agent's console widget settings from `GET /v1/agent/agents/{agent_id}/widget` — the same public-plus-origin gate as session creation, so it only answers for public agents to allowed origins. Settings resolve with a fixed precedence: **HTML attribute > console widget config > built-in default**. The endpoint being unreachable never breaks the widget — it renders from attributes and defaults. +On load, the widget anonymously fetches the agent's console widget settings from `GET /v1/agent/agents/{agent_id}/widget`, the same public-plus-origin gate as session creation, so it only answers for public agents to allowed origins. Settings resolve with a fixed precedence: **HTML attribute > console widget config > built-in default**. The endpoint being unreachable never breaks the widget: it renders from attributes and defaults. ## Going further diff --git a/agents/monitor/conversation-history.mdx b/agents/monitor/conversation-history.mdx index b7de400..693c649 100644 --- a/agents/monitor/conversation-history.mdx +++ b/agents/monitor/conversation-history.mdx @@ -4,24 +4,24 @@ description: "List past sessions, replay the full timeline of messages and tool icon: "clock-rotate-left" --- -Every production session your agents handle — from the API, the console, [public agents](/agents/deploy/public-agents), or [phone calls](/agents/telephony/inbound-calls) — is queryable over REST: a lightweight list for browsing, a merged timeline of messages and tool activity per session, and per-speaker recordings when the agent [records audio](#what-gets-stored). You can query a session while the call is still in progress. +Every production session your agents handle (from the API, the console, [public agents](/agents/deploy/public-agents), or [phone calls](/agents/telephony/inbound-calls)) is queryable over REST: a lightweight list for browsing, a merged timeline of messages and tool activity per session, and per-speaker recordings when the agent [records audio](#what-gets-stored). You can query a session while the call is still in progress. -This is a server-side API — authenticate with your API key. The client SDKs deliberately expose no history interface; fetch history from your backend and pass it to your frontend as needed. +This is a server-side API: authenticate with your API key. The client SDKs deliberately expose no history interface; fetch history from your backend and pass it to your frontend as needed. ## What gets stored -The session record — status, timing, caller attribution, name, and `metadata` — and the conversation transcript are always kept for production sessions. Audio recording is a per-agent choice, on by default: set `conversation.record_audio` in the console under your agent's **Settings** or in the `conversation` section of the [config API](/agents/build/configuration#configure-through-the-api), with a per-session override on the [session request](/agents/deploy/authenticated-sessions). Recording off means there is no audio to download. +The session record (status, timing, caller attribution, name, and `metadata`) and the conversation transcript are always kept for production sessions. Audio recording is a per-agent choice, on by default: set `conversation.record_audio` in the console under your agent's **Settings** or in the `conversation` section of the [config API](/agents/build/configuration#configure-through-the-api), with a per-session override on the [session request](/agents/deploy/authenticated-sessions). Recording off means there is no audio to download. The setting doesn't affect the live call: transcript events still stream to connected clients in real time. - Recording defaults to on. Call-recording laws vary by jurisdiction — make sure + Recording defaults to on. Call-recording laws vary by jurisdiction. Make sure callers are informed and consent where required before going live. ## List sessions -`GET /v1/agent/sessions` returns session facts, newest first. The list is intentionally thin — no transcripts or tool details — so it stays fast at any volume. +`GET /v1/agent/sessions` returns session facts, newest first. The list is intentionally thin (no transcripts or tool details) so it stays fast at any volume. ```bash API (curl) @@ -75,12 +75,12 @@ Each row carries the session facts: | Parameter | Behavior | | ---------------------------------- | ----------------------------------------------------------------------------------------------------- | | `agent_id` | Sessions of a single agent | -| `status` | One of `pending`, `active`, `completed`, `failed`, `unknown` — comma-separate values to match several | +| `status` | One of `pending`, `active`, `completed`, `failed`, `unknown`; comma-separate values to match several | | `caller_number` | Exact match on the caller's E.164 number; a bare number gets `+` prepended automatically | | `created_after` / `created_before` | ISO 8601 timestamps | - `pending` sessions — a session was created but no participant ever connected — + `pending` sessions (a session was created but no participant ever connected) are excluded by default. Pass `status=pending` explicitly to see them. Preview calls made from the Builder never appear in this API. @@ -94,7 +94,7 @@ Two modes, mutually exclusive (combining them returns `400`): | Cursor | Pass the previous response's `next_cursor` as `cursor`; stop when `has_more` is `false` | Crawling, exports, syncing | | Page | Pass `page` (1-based); the response always includes `total` | Paged UIs with jump-to-page | -`page_size` defaults to 30, maximum 100; values out of range are rejected. In cursor mode `total` is `null` unless you pass `include_total=true`. Page mode is capped at an offset of 100,000 rows — requests beyond it return `400` — and every page response still includes `next_cursor`, so you can switch to cursor crawling from any page for deep scans. +`page_size` defaults to 30, maximum 100; values out of range are rejected. In cursor mode `total` is `null` unless you pass `include_total=true`. Page mode is capped at an offset of 100,000 rows (requests beyond it return `400`), and every page response still includes `next_cursor`, so you can switch to cursor crawling from any page for deep scans. ## Get a session @@ -159,43 +159,43 @@ curl "https://api.fish.audio/v1/agent/sessions/SESSION_ID" \ | `type` | What it is | | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | `message` | One utterance. `role` is `user` or `assistant` (system prompt content is never exposed). `turn_id` groups items belonging to the same conversational turn. | -| `tool_call` | A tool invocation as it started. `input` is a JSON string, stored in full — no truncation. | +| `tool_call` | A tool invocation as it started. `input` is a JSON string, stored in full with no truncation. | | `tool_result` | The paired outcome, matched by `call_id`. `status` is `completed` or `failed`. `latency_ms` is the execution time. | Details worth knowing: -- **Role vocabulary** — the history API's `assistant` is the same speaker the [SDK's live events](/agents/deploy/web-sdk) call `agent`. -- **Order** — items are sorted by `created_at` ascending; a `tool_result`'s timestamp is its completion time, so long-running tools appear where they actually finished, with messages in between. Items sharing a timestamp order `message`, then `tool_call`, then `tool_result`. -- **`tool_source`** — where the tool ran: `client`, `webhook`, `builtin` (platform tools such as call transfer and hang-up), `mcp` (tools from a connected MCP server), `background` (work the agent delegated to a background task), or `unknown` (calls recorded before source attribution). Treat it as an open set. See [Tools](/agents/build/tools). -- **Payloads** — `input` and `output` are JSON strings, symmetric with the live SDK events, so one parser covers both. `output` and `error` are stored up to 256 KB; beyond that the text is cut and `output_truncated` is `true`. -- **Mid-call queries** — you can fetch an `active` session; you get the items persisted so far, and `analysis` is `null` until [post-call analysis](/agents/monitor/post-call-analysis) completes. +- **Role vocabulary**: the history API's `assistant` is the same speaker the [SDK's live events](/agents/deploy/web-sdk) call `agent`. +- **Order**: items are sorted by `created_at` ascending; a `tool_result`'s timestamp is its completion time, so long-running tools appear where they actually finished, with messages in between. Items sharing a timestamp order `message`, then `tool_call`, then `tool_result`. +- **`tool_source`**: where the tool ran, one of `client`, `webhook`, `builtin` (platform tools such as call transfer and hang-up), `mcp` (tools from a connected MCP server), `background` (work the agent delegated to a background task), or `unknown` (calls recorded before source attribution). Treat it as an open set. See [Tools](/agents/build/tools). +- **Payloads**: `input` and `output` are JSON strings, symmetric with the live SDK events, so one parser covers both. `output` and `error` are stored up to 256 KB; beyond that the text is cut and `output_truncated` is `true`. +- **Mid-call queries**: you can fetch an `active` session; you get the items persisted so far, and `analysis` is `null` until [post-call analysis](/agents/monitor/post-call-analysis) completes. - A `tool_call` without a matching `tool_result` means the execution never resolved. `items` is a discriminated union on `type`, and it evolves additively. Ignore - item types and fields you don't recognize — new modalities and item kinds will + item types and fields you don't recognize. New modalities and item kinds will appear without a version bump. ### Correlate with live events -`call_id` is the same identifier your client receives in the SDK's `toolCallStarted` / `toolCallCompleted` / `toolCallFailed` events. Log it live, and you can align in-call UI with the post-call record — jump from a tool chip in your interface straight to the matching `tool_call` / `tool_result` pair in history. See the [Web SDK](/agents/deploy/web-sdk). +`call_id` is the same identifier your client receives in the SDK's `toolCallStarted` / `toolCallCompleted` / `toolCallFailed` events. Log it live, and you can align in-call UI with the post-call record: jump from a tool chip in your interface straight to the matching `tool_call` / `tool_result` pair in history. See the [Web SDK](/agents/deploy/web-sdk). ## Download recordings -`GET /v1/agent/sessions/{session_id}/recording` returns the recording status and signed download URLs — one audio track per speaker, so you can play or process the agent and the user separately. Recordings exist only when the session was [recorded](#what-gets-stored): for sessions that never recorded — recording turned off, or no audio produced — the endpoint returns `404`. +`GET /v1/agent/sessions/{session_id}/recording` returns the recording status and signed download URLs: one audio track per speaker, so you can play or process the agent and the user separately. For sessions that never [recorded](#what-gets-stored) (recording turned off, or no audio produced), the endpoint returns `404`. ```bash API (curl) curl "https://api.fish.audio/v1/agent/sessions/SESSION_ID/recording" \ --header "Authorization: Bearer $FISH_API_KEY" ``` -Signed URLs are short-lived. Don't store them — store the `session_id` and request fresh URLs when you need the audio. For the same reason, recordings are not embedded in the session detail response. +Signed URLs are short-lived: store the `session_id` and request fresh URLs when you need the audio. For the same reason, recordings are not embedded in the session detail response. ## Access semantics - Your API key sees every production session across your team's workspaces, from all sources. -- A session that doesn't exist — or belongs to another team — always returns `404`, never `403`. The [recording endpoint](#download-recordings) additionally returns `404` for a session that exists but was never recorded. +- A session that doesn't exist, or belongs to another team, always returns `404`, never `403`. The [recording endpoint](#download-recordings) additionally returns `404` for a session that exists but was never recorded. ## Going further @@ -208,7 +208,7 @@ Signed URLs are short-lived. Don't store them — store the `session_id` and req What the embedded `analysis` object contains and how to configure it. - Push instead of poll — get notified when sessions end. + Push instead of poll: get notified when sessions end. The live `toolCall*` events that share `call_id` with history. diff --git a/agents/monitor/post-call-analysis.mdx b/agents/monitor/post-call-analysis.mdx index b6c48ab..133ecb1 100644 --- a/agents/monitor/post-call-analysis.mdx +++ b/agents/monitor/post-call-analysis.mdx @@ -4,7 +4,7 @@ description: "Automatic summaries, structured data extraction, and success evalu icon: "chart-simple" --- -After a conversation ends, the platform runs an LLM pass over the [stored transcript](/agents/monitor/conversation-history#what-gets-stored) and produces three results: a **summary**, values for the **data fields** you define, and a verdict for each **success criterion**. Summaries are on by default for every agent — extraction and evaluation are opt-in, configured per agent on the Builder's **Analysis** page. +After a conversation ends, the platform runs an LLM pass over the [stored transcript](/agents/monitor/conversation-history#what-gets-stored) and produces three results: a **summary**, values for the **data fields** you define, and a verdict for each **success criterion**. Summaries are on by default for every agent. Extraction and evaluation are opt-in, configured per agent on the Builder's **Analysis** page. - The summary language does **not** follow the conversation language. Summaries - and rationales are always written in the configured language, even when the - conversation happened in another one — so downstream systems see consistent - output. + Summaries and rationales are always written in the configured language, even + when the conversation happened in another one, so downstream systems see + consistent output. ### Data fields -Structured values extracted from the transcript — up to **20 fields** per agent. Each field is: +Structured values extracted from the transcript, up to **20 fields** per agent. Each field is: | Property | Description | | -------------- | ---------------------------------------------------------------------------------------------------------- | @@ -57,27 +56,27 @@ Structured values extracted from the transcript — up to **20 fields** per agen | `description` | Instruction telling the model what to extract (up to 500 characters) | | `enum_options` | For `enum` fields only: 2–20 allowed values | -When the conversation doesn't contain the information, the field's value is `null` with a rationale explaining why — the model never guesses. +When the conversation doesn't contain the information, the field's value is `null` with a rationale explaining why. ### Success criteria -Up to **10 criteria** per agent, each a `name` plus a `description` (up to 500 characters) stating what a successful conversation looks like — "The agent resolved the caller's issue or set clear next steps." +Up to **10 criteria** per agent, each a `name` plus a `description` (up to 500 characters) stating what a successful conversation looks like, such as "The agent resolved the caller's issue or set clear next steps." Every criterion gets a three-state verdict with a rationale: -- **`success`** — the transcript shows the expectation was met. -- **`failure`** — the transcript shows it was not. -- **`unknown`** — the transcript is incomplete, the answer was ambiguous, or the information needed to judge is missing. +- **`success`**: the transcript shows the expectation was met. +- **`failure`**: the transcript shows it was not. +- **`unknown`**: the transcript is incomplete, the answer was ambiguous, or the information needed to judge is missing. The `unknown` state is deliberate: rather than forcing an ambiguous call into - a binary verdict, the model tells you it couldn't judge — so your success-rate + a binary verdict, the model tells you it couldn't judge, so your success-rate numbers stay honest. ## Configure analysis -Open your agent's **Analysis** page in the Builder. Three cards — Summary, Data fields, and Criteria — autosave to the draft as you edit. You can also set the `analysis` section of the configuration via the API: +Open your agent's **Analysis** page in the Builder. Three cards (Summary, Data fields, and Criteria) autosave to the draft as you edit. You can also set the `analysis` section of the configuration via the API: ```bash curl --request PATCH "https://api.fish.audio/v1/agent/agents/$AGENT_ID/config" \ @@ -111,27 +110,26 @@ curl --request PATCH "https://api.fish.audio/v1/agent/agents/$AGENT_ID/config" \ Like all configuration, edits land in the [draft](/agents/deploy/versions-publishing). Preview calls use the draft immediately; production sessions pick up analysis changes after you **publish**. -To disable analysis entirely, turn the summary off and leave both lists empty — conversations then finish with analysis status `skipped` and no LLM call is made. +To disable analysis entirely, turn the summary off and leave both lists empty. Conversations then finish with analysis status `skipped` and no LLM call is made. ## When analysis runs -Analysis starts automatically when a conversation ends — production sessions from any source, and preview calls when you end them. A run moves through these statuses: +Analysis starts automatically when a conversation ends: production sessions from any source, and preview calls when you end them. A run moves through these statuses: | Status | Meaning | | ----------- | ----------------------------------------------------------------------------------- | | `queued` | Scheduled, waiting to run | | `running` | The LLM pass is in progress | | `completed` | Results are available | -| `skipped` | Nothing to analyze — the conversation had no user messages, or analysis is disabled | +| `skipped` | Nothing to analyze: the conversation had no user messages, or analysis is disabled | | `error` | The run failed; no results | Results are typically ready within seconds of the conversation ending. Production sessions are analyzed against the agent version pinned when the - session started — editing your criteria afterwards never changes how past - conversations were judged, and each result stays attributable to the - configuration that produced it. + session started; editing your criteria afterwards never changes how past + conversations were judged. ## Read the results @@ -173,7 +171,7 @@ curl "https://api.fish.audio/v1/agent/sessions/$SESSION_ID" \ } ``` -To push results into your CRM or data warehouse as soon as they're ready, configure a [post-call webhook](/agents/monitor/webhooks) — `call.analyzed` fires as soon as the run settles, whatever the outcome, with `analysis.status` telling you which of the terminal statuses above it reached. +To push results into your CRM or data warehouse as soon as they're ready, configure a [post-call webhook](/agents/monitor/webhooks). `call.analyzed` fires as soon as the run settles, whatever the outcome, with `analysis.status` telling you which of the terminal statuses above it reached. ## Going further diff --git a/agents/monitor/webhooks.mdx b/agents/monitor/webhooks.mdx index 7e69f9b..3647c48 100644 --- a/agents/monitor/webhooks.mdx +++ b/agents/monitor/webhooks.mdx @@ -1,22 +1,25 @@ --- title: "Webhooks" -description: "Get an HTTP POST when a call ends and when its analysis settles — no polling" +description: "Get an HTTP POST when a call ends and when its analysis settles, with no polling" icon: "bell" --- -Point your agent at one or more endpoints on your server and Fish Audio calls them when things happen: `call.ended` the moment a session reaches a terminal state, `call.analyzed` when [post-call analysis](/agents/monitor/post-call-analysis) settles. Use them to write results into your CRM, ticketing system, or data warehouse without polling the sessions API. +Point your agent at one or more endpoints on your server and Fish Audio calls them when things happen: `call.ended` the moment a session reaches a terminal state, `call.analyzed` when [post-call analysis](/agents/monitor/post-call-analysis) settles, and, on outbound phone calls, `phone_call.dial_finished` as soon as the dial attempt resolves. Use them to write results into your CRM, ticketing system, or data warehouse without polling the sessions API. ## Events -| Event | Fires | Payload | -| --------------- | ------------------------------------------------------------------------------------------------ | --------------------------------------------- | -| `call.ended` | When a session reaches a terminal state — every real call, whether or not analysis is configured | Session facts plus the reason the call ended | -| `call.analyzed` | When post-call analysis settles — completed, skipped, or error | Session facts plus the full `analysis` entity | +| Event | Fires | Payload | +| -------------------------- | ------------------------------------------------------------------------------------------------ | --------------------------------------------- | +| `call.ended` | When a session reaches a terminal state: every real call, whether or not analysis is configured | Session facts plus the reason the call ended | +| `call.analyzed` | When post-call analysis settles: completed, skipped, or error | Session facts plus the full `analysis` entity | +| `phone_call.dial_finished` | When an outbound call's dial attempt resolves: answered, busy, no answer, or failed | Session facts plus `dial_status` and `answered_by` | -For a given session, `call.ended` is delivered before `call.analyzed`, and every production call produces both events. When analysis has nothing to work with (the caller never spoke, or analysis is disabled) or fails, `call.analyzed` still arrives — with `analysis.status` set to `skipped` or `error` and empty result sections. You never have to infer from silence whether a result is still coming: check `analysis.status`. +For a given session, `call.ended` is delivered before `call.analyzed`, and every answered production call produces both events. When analysis has nothing to work with (the caller never spoke, or analysis is disabled) or fails, `call.analyzed` still arrives, with `analysis.status` set to `skipped` or `error` and empty result sections. + +`phone_call.dial_finished` fires only for outbound calls, and it fires whether or not anyone picks up. An answered outbound call produces all three events, `phone_call.dial_finished` first; an outbound call that is never answered (a `dial_status` of `busy`, `no_answer`, or `failed`) produces `phone_call.dial_finished` and nothing else. Unanswered calls are not billed and not analyzed. - Transcripts are deliberately excluded from webhook payloads — fetch them on + Transcripts are deliberately excluded from webhook payloads. Fetch them on demand with `GET /v1/agent/sessions/{session_id}`. See [conversation history](/agents/monitor/conversation-history). @@ -71,23 +74,23 @@ httpx.patch( -The list is replaced as a whole on each update — send every endpoint you want to keep, including the ones you aren't changing. Set it to `null` or `[]` to stop deliveries. A single `post_call` object is still accepted and is treated as a one-element list. +The list is replaced as a whole on each update, so send every endpoint you want to keep. Set it to `null` or `[]` to stop deliveries. A single `post_call` object is still accepted and is treated as a one-element list. | Field | Rules | | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `post_call` | Up to 5 entries, and their `url` values must be unique — a longer list or a repeated URL is rejected with `422` | -| `url` | Required on every entry. Up to 4000 characters. Must resolve to a public address — localhost and private-network hosts are rejected with `422`, and the check is repeated at every delivery | +| `post_call` | Up to 5 entries, and their `url` values must be unique. A longer list or a repeated URL is rejected with `422` | +| `url` | Required on every entry. Up to 4000 characters. Must resolve to a public address. Localhost and private-network hosts are rejected with `422`, and the check is repeated at every delivery | | `secret` | Optional signing key, up to 256 characters, set per entry. **Write-only**: reads return `has_secret: true`, never the value | Webhook settings are part of the agent configuration, so they follow the - draft-and-publish flow — [publish](/agents/deploy/versions-publishing) for + draft-and-publish flow: [publish](/agents/deploy/versions-publishing) for changes to apply to production calls. ## Payload -Every payload carries the `event` name and a `session` object with a fixed set of session facts. Note the field names differ from the [sessions API](/agents/monitor/conversation-history), which uses `session_id` / `started_at` / `ended_at` for the same facts. `call.ended` adds `ended_reason`; `call.analyzed` adds the `analysis` entity: +Every payload carries the `event` name and a `session` object with a fixed set of session facts. Note the field names differ from the [sessions API](/agents/monitor/conversation-history), which uses `session_id` / `started_at` / `ended_at` for the same facts. `call.ended` adds `ended_reason`; `call.analyzed` adds the `analysis` entity; `phone_call.dial_finished` adds top-level `dial_status` and `answered_by`: @@ -147,13 +150,41 @@ Every payload carries the `event` name and a `session` object with a fixed set o } ``` +```json phone_call.dial_finished +{ + "event": "phone_call.dial_finished", + "dial_status": "answered", + "answered_by": "human", + "session": { + "id": "5f3e…", + "agent_id": "a1b2…", + "branch_id": "b7d4…", + "source": "phone", + "status": "active", + "direction": "outbound", + "dial_status": "answered", + "answered_by": "human", + "batch_call_id": null, + "conversation_started_at": "2026-07-23T12:01:12Z", + "conversation_ended_at": null, + "duration_seconds": null, + "end_user_id": "", + "metadata": { "lead_ref": "L-2041" }, + "agent_name": "Follow-up agent", + "config_hash": "sha256:9c41…" + } +} +``` + -`ended_reason` is `hangup` for a call that terminated normally and `error` when the session failed. New values may be added as richer end causes ship — treat unrecognized values as informational rather than rejecting the event. +`ended_reason` is `hangup` for a call that terminated normally and `error` when the session failed. New values may be added as richer end causes ship. Treat unrecognized values as informational rather than rejecting the event. + +`dial_status` is `answered`, `busy`, `no_answer`, or `failed`. `answered_by` reports what answering-machine detection heard (`human`, `voicemail`, or `unknown`) and is `null` unless the call was answered. Both facts also appear on the session, alongside `direction` (`inbound` or `outbound`) and `batch_call_id` (reserved, always `null` today); these session fields are present in every webhook payload, with the dial fields `null` on inbound sessions. The `phone_call.dial_finished` snapshot is taken when the dial resolves, so on an answered call `conversation_ended_at` and `duration_seconds` are still `null`. The final numbers arrive with `call.ended`. -`end_user_id` and `metadata` are echoed exactly as you set them when creating the session — use them to correlate the event with records in your own system. `branch_id` is an internal configuration-lineage identifier — safe to ignore. What lands in `summary`, `data`, and `criteria_results` is defined by your [analysis configuration](/agents/monitor/post-call-analysis). +`end_user_id` and `metadata` are echoed exactly as you set them when creating the session. Use them to correlate the event with records in your own system. `branch_id` is an internal configuration-lineage identifier and is safe to ignore. What lands in `summary`, `data`, and `criteria_results` is defined by your [analysis configuration](/agents/monitor/post-call-analysis). -The example shows a `completed` analysis. `analysis.status` can also be `skipped` (nothing to analyze — the caller never spoke, or analysis is disabled) or `error` (the run failed, with the cause in `analysis.error`); both arrive with `summary: null` and empty `data` / `criteria_results`. Check the status before reading results. +The example shows a `completed` analysis. `analysis.status` can also be `skipped` (nothing to analyze: the caller never spoke, or analysis is disabled) or `error` (the run failed, with the cause in `analysis.error`); both arrive with `summary: null` and empty `data` / `criteria_results`. Check the status before reading results. ## Verify the signature @@ -163,10 +194,10 @@ When an entry has a `secret`, every request to that endpoint carries a signature X-Fish-Webhook-Signature: t=1784808000,v1=8693a4b9… ``` -`t` is the Unix time (seconds) the request was sent — each retry is signed fresh. `v1` is the hex HMAC. Verify in three steps: +`t` is the Unix time (seconds) the request was sent; each retry is signed fresh. `v1` is the hex HMAC. Verify in three steps: 1. Parse `t` and `v1` from the header. -2. Recompute HMAC-SHA256 over the string `{t}.` followed by the **raw request body bytes** — before any JSON parsing or re-serialization — and compare against `v1` in constant time. +2. Recompute HMAC-SHA256 over the string `{t}.` followed by the **raw request body bytes** (before any JSON parsing or re-serialization) and compare against `v1` in constant time. 3. Reject requests whose `t` is more than 5 minutes from your clock. Because the timestamp is inside the MAC, a replayed capture can't be refreshed. @@ -230,8 +261,8 @@ def verify_webhook(raw_body: bytes, signature_header: str, secret: str) -> bool: - Future scheme revisions would ship under a new element (`v2=…`) alongside `v1` - — parse the elements you know and ignore the rest, as the snippets above do. + Future scheme revisions would ship under a new element (`v2=…`) alongside `v1`. + Parse the elements you know and ignore the rest, as the snippets above do. ## Delivery semantics @@ -242,23 +273,23 @@ def verify_webhook(raw_body: bytes, signature_header: str, secret: str) -> bool: | Guarantee | At-least-once, per endpoint | | Timeout | 10 seconds per attempt | | Retries | 2 after the first attempt (3 attempts total) per endpoint, with backoff of 1s / 5s, then that delivery is dropped | -| Ordering | `call.ended` before `call.analyzed` for the same session | +| Ordering | `phone_call.dial_finished` (outbound only) before `call.ended` before `call.analyzed` for the same session | Endpoints are delivered in parallel and independently: each gets its own attempts, its own retry budget, and its own signature keyed with its own secret. An endpoint that is down and exhausts all three attempts has no effect on the others. -Respond with a `2xx` status within the timeout; a `500` response or a timed-out request counts as a failed attempt. Acknowledge first and process asynchronously — slow handlers burn their own retry budget. +Respond with a `2xx` status within the timeout; a `500` response or a timed-out request counts as a failed attempt. Acknowledge first and process asynchronously. Slow handlers burn their own retry budget. -**Idempotency.** At-least-once delivery means the same event can arrive more than once. Retries of one delivery carry an identical body, so dedupe `call.ended` on (`event`, `session.id`) and `call.analyzed` on (`event`, `session.id`, `analysis.finished_at`). The extra element matters because a skipped or failed analysis can be re-run from the console: the recovered result arrives as a fresh `call.analyzed` with a newer `finished_at`, superseding the earlier one. +**Idempotency.** At-least-once delivery means the same event can arrive more than once. Retries of one delivery carry an identical body, so dedupe `call.ended` and `phone_call.dial_finished` on (`event`, `session.id`), and `call.analyzed` on (`event`, `session.id`, `analysis.finished_at`). The extra element matters because a skipped or failed analysis can be re-run from the console: the recovered result arrives as a fresh `call.analyzed` with a newer `finished_at`, superseding the earlier one. [Preview calls](/agents/test/preview-calls) made from the Builder never - trigger webhooks — debugging sessions don't reach your production endpoint. To - test end to end, run a real session against your published agent. + trigger webhooks. To test end to end, run a real session against your + published agent. ## Auto-ticket unresolved calls -`call.analyzed` closes the loop on conversations the agent couldn't: judge every call with a success criterion, and open a ticket in your helpdesk whenever the verdict isn't `success` — no mid-call decision, no dashboard watching. +`call.analyzed` closes the loop on conversations the agent couldn't: judge every call with a success criterion, and open a ticket in your helpdesk whenever the verdict isn't `success`. First give the agent's [analysis configuration](/agents/monitor/post-call-analysis) a criterion that captures resolution: @@ -319,7 +350,7 @@ app.post( subject: `Unresolved call ${payload.session.id}`, body: `${payload.analysis.summary}\n\n` + - `issue_resolved: ${verdict.result} — ${verdict.rationale}`, + `issue_resolved: ${verdict.result}. ${verdict.rationale}`, customer: payload.session.end_user_id, }); } @@ -371,7 +402,7 @@ async def fish_webhooks( open_ticket( subject=f"Unresolved call {payload['session']['id']}", body=f"{analysis['summary']}\n\n" - f"issue_resolved: {verdict['result']} — {verdict['rationale']}", + f"issue_resolved: {verdict['result']}. {verdict['rationale']}", customer=payload["session"]["end_user_id"], ) return {} @@ -379,16 +410,16 @@ async def fish_webhooks( -`verifyWebhook` is the function from [Verify the signature](#verify-the-signature); `openTicket` stands in for your helpdesk's API. Escalating on anything but `success` includes `unknown` verdicts — the model couldn't judge the call, which usually deserves human eyes too. Tighten the check to `failure` only if unknowns prove noisy. +`verifyWebhook` is the function from [Verify the signature](#verify-the-signature); `openTicket` stands in for your helpdesk's API. Escalating on anything but `success` includes `unknown` verdicts: the model couldn't judge the call, which usually deserves human eyes too. Tighten the check to `failure` only if unknowns prove noisy. Edges worth handling: -- Calls with nothing to analyze arrive with `analysis.status: "skipped"` — the handler above tickets them as unjudged, so every call reaches the helpdesk without also watching `call.ended`. Drop that branch if silent calls don't belong in your queue. -- Payloads carry no transcript. To include one in the ticket, fetch `GET /v1/agent/sessions/{session_id}` from your handler — see [conversation history](/agents/monitor/conversation-history). +- Calls with nothing to analyze arrive with `analysis.status: "skipped"`. The handler above tickets them as unjudged, so every call reaches the helpdesk without also watching `call.ended`. Drop that branch if silent calls don't belong in your queue. +- Payloads carry no transcript. To include one in the ticket, fetch `GET /v1/agent/sessions/{session_id}` from your handler. See [conversation history](/agents/monitor/conversation-history). - Set `end_user_id` and `metadata` when creating sessions so tickets attach to the right customer record without a lookup. - Post-call ticketing is silent — the caller has already hung up. When the + Post-call ticketing is silent: the caller has already hung up. When the caller should leave the call holding a ticket number, have the agent open it mid-call with a [webhook tool](/agents/build/webhook-tools#escalate-to-a-ticket-mid-call), and keep @@ -417,7 +448,7 @@ Edges worth handling: icon="code-branch" href="/agents/deploy/versions-publishing" > - How configuration changes — including webhooks — go live. + How configuration changes (including webhooks) go live. Create an agent and have your first conversation in minutes. - Agents, drafts, versions, sessions — the mental model. + Agents, drafts, versions, sessions: the mental model. Pick the right surface for your app. @@ -23,11 +23,11 @@ Fish Agents is a hosted platform for real-time voice agents. Define an agent's p | Surface | What it does | Start here | | --------------- | ------------------------------------------------------------------------------------------------------- | ----------------------------------------------- | -| **Console** | Design, test, and publish agents visually — no code. | [Configuration](/agents/build/configuration) | +| **Console** | Design, test, and publish agents visually, with no code. | [Configuration](/agents/build/configuration) | | **REST API** | Manage agents, sessions, tools, and knowledge programmatically under `/v1/agent/*`. | [API introduction](/api-reference/introduction) | | **Client SDKs** | Run live voice conversations in the browser with `@fishaudio/agent-client` or `@fishaudio/agent-react`. | [Web SDK](/agents/deploy/web-sdk) | -All three operate on the same agents. The core console workflow — creating agents, editing configuration, publishing versions — is also available over the API with your standard Fish Audio API key: +All three operate on the same agents. The core console workflow (creating agents, editing configuration, publishing versions) is also available over the API with your standard Fish Audio API key: ```bash curl https://api.fish.audio/v1/agent/agents \ @@ -110,7 +110,7 @@ curl https://api.fish.audio/v1/agent/agents \ icon="clock-rotate-left" href="/agents/monitor/conversation-history" > - Transcripts, tool timelines, and per-speaker recordings — you choose what + Transcripts, tool timelines, and per-speaker recordings. You choose what each agent stores. - Automatic summaries, structured data extraction, and success criteria — + Automatic summaries, structured data extraction, and success criteria, delivered to your systems via [webhooks](/agents/monitor/webhooks). diff --git a/agents/quickstart.mdx b/agents/quickstart.mdx index b8e3093..6deb78c 100644 --- a/agents/quickstart.mdx +++ b/agents/quickstart.mdx @@ -9,29 +9,29 @@ Build a voice agent and talk to it in a few minutes. Use the console for a no-co **Prerequisites** - A Fish Audio account -- For the API path: a Fish Audio API key — create one under [API keys](https://fish.audio/app/api-keys/) in the console +- For the API path: a Fish Audio API key. Create one under [API keys](https://fish.audio/app/api-keys/) in the console - Go to [Agents](https://fish.audio/app/agents) and click **New agent**. Give it a name — you land in the Builder immediately. + Go to [Agents](https://fish.audio/app/agents) and click **New agent**. Give it a name. You land in the Builder immediately. On the **Configuration** page, write the system prompt that defines who your agent is and how it should behave (up to 4,000 tokens). Optionally set a **First message** so the agent opens the conversation. - Edits save automatically as a draft — there is no Save button. + Edits save automatically as a draft. Choose a voice and a speaking language for your agent. You can pick from the featured voices or browse the full library. See [Voice & language](/agents/build/voice-language) for details. - Click **Test call** in the top bar, then **Start call** in the panel. Your browser asks for microphone access, then you talk to the agent directly — with a live transcript, call timer, and mute and hang-up controls in the side panel. + Click **Test call** in the top bar, then **Start call** in the panel. Your browser asks for microphone access, then you talk to the agent directly, with a live transcript, call timer, and mute and hang-up controls in the side panel. Preview calls always run against your current draft, so you can iterate on the prompt and immediately hear the difference. See [Preview calls](/agents/test/preview-calls). - Click **Publish** to turn the draft into an immutable version. Published versions are what real sessions connect to — drafts stay private to the Builder. + Click **Publish** to turn the draft into an immutable version. Published versions are what real sessions connect to. You can keep editing the draft afterwards; nothing goes live until you publish again. See [Versions & publishing](/agents/deploy/versions-publishing). @@ -41,7 +41,7 @@ Build a voice agent and talk to it in a few minutes. Use the console for a no-co - Create the agent and configure it in one call — the `config` section is optional at creation time, and you can update it later with `PATCH /v1/agent/agents/{agent_id}/config`. + Create the agent and configure it in one call. The `config` section is optional at creation time, and you can update it later with `PATCH /v1/agent/agents/{agent_id}/config`. ```bash Create an agent curl --request POST https://api.fish.audio/v1/agent/agents \ @@ -60,7 +60,7 @@ Build a voice agent and talk to it in a few minutes. Use the console for a no-co }' ``` - The response includes the agent's `agent_id` — use it as `$AGENT_ID` below. `voice_id` accepts any voice model id from the [Voice Library](/features/manage-voices). + The response includes the agent's `agent_id`. Use it as `$AGENT_ID` below. `voice_id` accepts any voice model id from the [Voice Library](/features/manage-voices). `system_prompt` is limited to 4,000 tokens; longer prompts return `422`. Keeping it under 2,000 tokens is recommended for latency and cost. @@ -77,7 +77,7 @@ Build a voice agent and talk to it in a few minutes. Use the console for a no-co Each publish creates an immutable version with an auto-incremented `version_number`. - From your backend, exchange your API key for a short-lived session token. This is the credential the browser uses to join the call — your API key never leaves your server. + From your backend, exchange your API key for a short-lived session token. This is the credential the browser uses to join the call. Your API key never leaves your server. ```bash Create a session curl --request POST https://api.fish.audio/v1/agent/sessions \ @@ -97,7 +97,7 @@ Build a voice agent and talk to it in a few minutes. Use the console for a no-co } ``` - Return this response to your frontend as-is — the SDK consumes it unchanged. `expires_at` is the deadline for joining the call. + Return this response to your frontend as-is. `expires_at` is the deadline for joining the call. Install the client SDK: @@ -106,7 +106,7 @@ Build a voice agent and talk to it in a few minutes. Use the console for a no-co npm install @fishaudio/agent-client ``` - Pass the session token to `AgentSession.start` — the SDK requests the microphone, connects, and streams audio both ways: + Pass the session token to `AgentSession.start`. The SDK requests the microphone, connects, and streams audio both ways: ```typescript Connect and talk import { AgentSession } from "@fishaudio/agent-client"; @@ -125,10 +125,10 @@ Build a voice agent and talk to it in a few minutes. Use the console for a no-co await session.end(); ``` - Start talking — you hear the agent reply and both sides of the conversation arrive as transcript events. See the [Web SDK](/agents/deploy/web-sdk) for the full event and method surface, or the [React SDK](/agents/deploy/react-sdk) for hooks. + Start talking. You hear the agent reply and both sides of the conversation arrive as transcript events. See the [Web SDK](/agents/deploy/web-sdk) for the full event and method surface, or the [React SDK](/agents/deploy/react-sdk) for hooks. - The whole loop is two files: a backend route that creates the session with your API key, and frontend code that starts the call. Run the server with `FISH_API_KEY` and `AGENT_ID` set, serve the frontend from your app's dev server (Vite, Next — anything that bundles npm imports), and talk. + The whole loop is two files: a backend route that creates the session with your API key, and frontend code that starts the call. Run the server with `FISH_API_KEY` and `AGENT_ID` set, serve the frontend from your app's dev server (Vite, Next, or anything that bundles npm imports), and talk. ```javascript server.mjs import express from "express"; @@ -180,7 +180,7 @@ Build a voice agent and talk to it in a few minutes. Use the console for a no-co - Agents, drafts, versions, and sessions — how the pieces fit together. + Agents, drafts, versions, and sessions: how the pieces fit together. Events, transcripts, client tools, and audio controls in the browser. diff --git a/agents/telephony/inbound-calls.mdx b/agents/telephony/inbound-calls.mdx index 4ab3e62..5fa9fe4 100644 --- a/agents/telephony/inbound-calls.mdx +++ b/agents/telephony/inbound-calls.mdx @@ -30,7 +30,7 @@ Point a phone number at an agent and it picks up every inbound call. Phone calls - Add a number to your workspace — see [Phone numbers](/agents/telephony/phone-numbers). Numbers are stored and displayed in E.164 format (for example `+15551234567`). + Add a number to your workspace; see [Phone numbers](/agents/telephony/phone-numbers). Numbers are stored and displayed in E.164 format (for example `+15551234567`). Calls connect to the agent's published configuration, not the draft. [Publish](/agents/deploy/versions-publishing) before pointing traffic at the number. @@ -59,7 +59,7 @@ Point a phone number at an agent and it picks up every inbound call. Phone calls The number-to-agent binding is resolved on each incoming call, so a rebind - takes effect on the next call — useful for moving a number from a staging + takes effect on the next call, useful for moving a number from a staging agent to a production agent. @@ -102,17 +102,17 @@ curl --request GET "https://api.fish.audio/v1/agent/sessions?agent_id=YOUR_AGENT } ``` -You can combine `caller_number` with the other list filters (`agent_id`, `status`, `created_after`, `created_before`) — see [Conversation history](/agents/monitor/conversation-history) for the full parameter list. +You can combine `caller_number` with the other list filters (`agent_id`, `status`, `created_after`, `created_before`); see [Conversation history](/agents/monitor/conversation-history) for the full parameter list. ## Phone sessions behave like any session Nothing about a phone call needs special handling downstream: -- **Transcript and tool timeline** — `GET /v1/agent/sessions/{session_id}` returns the conversation, including tool calls and results, when the agent [stores transcripts](/agents/monitor/conversation-history#what-gets-stored). See [Conversation history](/agents/monitor/conversation-history). -- **Recording** — `GET /v1/agent/sessions/{session_id}/recording` returns signed URLs, one track per speaker, when the session was recorded. -- **Post-call analysis** — summaries, data fields, and evaluation criteria run on phone sessions like any other. See [Post-call analysis](/agents/monitor/post-call-analysis). -- **Webhooks** — `call.ended` and `call.analyzed` fire for phone sessions too. See [Webhooks](/agents/monitor/webhooks). -- **Hang up via API** — end an in-progress call with `POST /v1/agent/sessions/{session_id}/end`. +- **Transcript and tool timeline**: `GET /v1/agent/sessions/{session_id}` returns the conversation, including tool calls and results, when the agent [stores transcripts](/agents/monitor/conversation-history#what-gets-stored). See [Conversation history](/agents/monitor/conversation-history). +- **Recording**: `GET /v1/agent/sessions/{session_id}/recording` returns signed URLs, one track per speaker, when the session was recorded. +- **Post-call analysis**: summaries, data fields, and evaluation criteria run on phone sessions like any other. See [Post-call analysis](/agents/monitor/post-call-analysis). +- **Webhooks**: `call.ended` and `call.analyzed` fire for phone sessions too. See [Webhooks](/agents/monitor/webhooks). +- **Hang up via API**: end an in-progress call with `POST /v1/agent/sessions/{session_id}/end`. ## Going further diff --git a/agents/telephony/phone-numbers.mdx b/agents/telephony/phone-numbers.mdx index 8e587b6..74a449b 100644 --- a/agents/telephony/phone-numbers.mdx +++ b/agents/telephony/phone-numbers.mdx @@ -4,7 +4,7 @@ description: "Search, purchase, bind, and release phone numbers for your agents icon: "phone" --- -Phone numbers connect your agents to the telephone network. Each number lives in a workspace within your team (purchases land in your default workspace) and is bound to at most one agent — calls to the number are answered by that agent. The `/v1/agent/phone-numbers` API covers the whole lifecycle: search the purchasable inventory, buy a number, bind it to an agent, and release it when you no longer need it. +Phone numbers connect your agents to the telephone network. Each number lives in a workspace within your team (purchases land in your default workspace) and is bound to at most one agent: calls to the number are answered by that agent. The `/v1/agent/phone-numbers` API covers the whole lifecycle: search the purchasable inventory, buy a number, bind it to an agent, and release it when you no longer need it. ## Search available numbers @@ -15,18 +15,17 @@ curl "https://api.fish.audio/v1/agent/available-phone-numbers?country_code=US&ar --header "Authorization: Bearer $FISH_API_KEY" ``` -The response is an `available_phone_numbers` array of inventory entries forwarded from the provider — number and region. +The response is an `available_phone_numbers` array of inventory entries forwarded from the provider: number and region. | Parameter | Description | | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | `country_code` | ISO 3166-1 alpha-2 country code, `US` (default) or `CA`. | | `area_code` | Restrict results to one area code, like `415`. | | `number_type` | `local` (default). The managed inventory is US and Canada local numbers only; `toll_free` returns `400`. | -| `provider` | Inventory to search. Only `twilio` (the default) is available — managed numbers with [call transfer](/agents/telephony/transfers) support. | +| `provider` | Inventory to search. Only `twilio` (the default) is available: managed numbers with [call transfer](/agents/telephony/transfers) support. | - Availability is not a reservation — a listed number can still be claimed by - someone else before you buy it. + A listed number can still be claimed by someone else before you buy it. ## Purchase a number @@ -45,20 +44,20 @@ curl --request POST https://api.fish.audio/v1/agent/phone-numbers \ }' ``` -Returns `201` with the number object — the same shape the list endpoint returns. The number lands in your default workspace, and its monthly price is billed in daily slices. +Returns `201` with the number object, the same shape the list endpoint returns. The number lands in your default workspace, and its monthly price is billed in daily slices. | Field | Description | | -------------- | ------------------------------------------------------------------------ | -| `provider` | Required — `twilio`, the inventory the number came from. | -| `phone_number` | Required — an E.164 number from the search response. | +| `provider` | Required: `twilio`, the inventory the number came from. | +| `phone_number` | Required: an E.164 number from the search response. | | `label` | Optional free-form label, up to 120 characters. | -| `agent_id` | Optional — bind an agent so the number answers inbound calls right away. | +| `agent_id` | Optional: bind an agent so the number answers inbound calls right away. | -A `409` means the number is already on the platform; a `502` means the provider refused the purchase — the number stays visible with status `error` and is safe to release. +A `409` means the number is already on the platform; a `502` means the provider refused the purchase; the number stays visible with status `error` and is safe to release. ## List your numbers -Returns your team's phone numbers across its workspaces, newest first — each carries its `workspace_id`. +Returns your team's phone numbers across its workspaces, newest first; each carries its `workspace_id`. ```bash Request curl https://api.fish.audio/v1/agent/phone-numbers \ @@ -83,7 +82,7 @@ curl https://api.fish.audio/v1/agent/phone-numbers \ | Field | Description | | ----------------- | ---------------------------------------------------------------- | -| `phone_number_id` | Unique identifier — use it in URL paths | +| `phone_number_id` | Unique identifier; use it in URL paths | | `phone_number` | The number in E.164 format | | `provider` | Telephony provider backing the number | | `label` | Free-form label you assign | @@ -101,7 +100,7 @@ Returns the same number object as the list endpoint. ## Bind or unbind an agent -`PATCH` the number with an `agent_id` to change which agent answers it. Binding is resolved per call, so the change applies from the next inbound call — nothing to redeploy. +`PATCH` the number with an `agent_id` to change which agent answers it. Binding is resolved per call, so the change applies from the next inbound call. ```bash Bind @@ -120,7 +119,7 @@ curl --request PATCH https://api.fish.audio/v1/agent/phone-numbers/$PHONE_NUMBER -Omitted fields are left unchanged — only an explicit `"agent_id": null` unbinds the number. +Omitted fields are left unchanged. Only an explicit `"agent_id": null` unbinds the number. Re-binding is how you promote an agent behind a stable number: point the @@ -130,7 +129,7 @@ Omitted fields are left unchanged — only an explicit `"agent_id": null` unbind ## Label your numbers -Labels are free-form text for keeping an inventory readable — by team, region, campaign, or environment. +Labels are free-form text for keeping an inventory readable: by team, region, campaign, or environment. ```bash Request curl --request PATCH https://api.fish.audio/v1/agent/phone-numbers/$PHONE_NUMBER_ID \ @@ -151,7 +150,7 @@ curl --request DELETE https://api.fish.audio/v1/agent/phone-numbers/$PHONE_NUMBE ``` - Releasing is irreversible. Anyone — including other platforms — can buy the + Releasing is irreversible. Anyone, including other platforms, can buy the number afterwards, so callers who saved it may reach a stranger. diff --git a/agents/telephony/transfers.mdx b/agents/telephony/transfers.mdx index 2fe0bbb..be8df08 100644 --- a/agents/telephony/transfers.mdx +++ b/agents/telephony/transfers.mdx @@ -1,6 +1,6 @@ --- title: "Call Transfers" -description: "Let the agent hand a phone call to a human — a cold carrier handoff, or a warm transfer with hold music and a private briefing" +description: "Let the agent hand a phone call to a human: a cold carrier handoff, or a warm transfer with hold music and a private briefing" icon: "phone-arrow-right" --- @@ -12,22 +12,22 @@ Two modes decide what the handoff feels like: | ------------------------------ | ----------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Handoff** | The agent announces the transfer and drops out; the carrier connects the caller to the destination. | The caller waits on hold while the agent briefs the human on a private call, then the two are connected. | | **What the human hears first** | The caller, directly. | The agent's briefing: who is calling and what they need. | -| **Recording & analysis** | Stop at the handoff — nothing after it appears in [post-call analysis](/agents/monitor/post-call-analysis). | Recording continues after the human joins, capturing the full call audio; the transcript — and therefore analysis — covers only the agent's segment, up to the merge. | +| **Recording & analysis** | Stop at the handoff: nothing after it appears in [post-call analysis](/agents/monitor/post-call-analysis). | Recording continues after the human joins, capturing the full call audio; the transcript (and therefore analysis) covers only the agent's segment, up to the merge. | ## Prerequisites -- A phone number bound to your agent, answering inbound calls — see [Inbound calls](/agents/telephony/inbound-calls). -- The number must support transfers — numbers purchased from the platform inventory (`provider: "twilio"`) do; see [Phone numbers](/agents/telephony/phone-numbers). +- A phone number bound to your agent, answering inbound calls. See [Inbound calls](/agents/telephony/inbound-calls). +- The number must support transfers. Numbers purchased from the platform inventory (`provider: "twilio"`) do; see [Phone numbers](/agents/telephony/phone-numbers). ## Configure a destination ### In the console -On the agent's **Phone** page, under **Inbound calls**, turn on **Call transfer**. Give the destination a name, enter its phone number, and pick the transfer mode — choosing **Warm** adds a handoff choice (ask the human to accept first, or connect immediately after the briefing). Like every config edit, the change lands in the agent's draft and takes effect on live calls after you [publish](/agents/deploy/versions-publishing). +On the agent's **Phone** page, under **Inbound calls**, turn on **Call transfer**. Give the destination a name, enter its phone number, and pick the transfer mode. Choosing **Warm** adds a handoff choice (ask the human to accept first, or connect immediately after the briefing). Like every config edit, the change lands in the agent's draft and takes effect on live calls after you [publish](/agents/deploy/versions-publishing). ### Through the API -Transfer destinations live in the `conversation` section of the [agent config](/agents/build/configuration), as `transfer_destinations`. An empty list disables transfers; only one destination is supported for now — a longer list is rejected. +Transfer destinations live in the `conversation` section of the [agent config](/agents/build/configuration), as `transfer_destinations`. An empty list disables transfers; only one destination is supported for now, and a longer list is rejected. ```bash Enable @@ -62,19 +62,19 @@ curl --request PATCH "https://api.fish.audio/v1/agent/agents/$AGENT_ID/config" \ | `type` | Destination type; `phone` is the only value today and the default. | | `label` | Name for the destination, shown in the Builder. At most 64 characters. | | `phone_number` | The destination in E.164 format, for example `+14155550123`. | -| `mode` | `cold` (default) or `warm` — see the comparison above. | +| `mode` | `cold` (default) or `warm`; see the comparison above. | | `warm_connect` | Warm only: `confirm` (default) waits for the human's go-ahead before connecting; `direct` connects right after the briefing. | -Destination numbers must be E.164 and are limited to a set of supported countries — a number outside the list is rejected with `422 Unprocessable Entity` and an error naming the allowed country codes. The transfer card in the console shows the current list. +Destination numbers must be E.164 and are limited to a set of supported countries. A number outside the list is rejected with `422 Unprocessable Entity` and an error naming the allowed country codes. The transfer card in the console shows the current list. ## How the agent decides to transfer -Configuring a destination is what enables the built-in `transfer_call` tool — there is no separate switch, and an empty destination list removes the tool. Unlike the toggled [system tools](/agents/build/system-tools), you won't find it in the `tools` config section; it ships automatically on phone sessions whenever a valid destination exists. +Configuring a destination is what enables the built-in `transfer_call` tool: an empty destination list removes it. Unlike the toggled [system tools](/agents/build/system-tools), you won't find it in the `tools` config section; it ships automatically on phone sessions whenever a valid destination exists. -The built-in instructions are conservative: the agent transfers only when the caller asks for a person, or when the request clearly needs one and the agent cannot help further — never preemptively for questions it can answer itself. +The built-in instructions are conservative: the agent transfers only when the caller asks for a person, or when the request clearly needs one and the agent cannot help further. - Use the system prompt to sharpen the escalation policy — for example, + Use the system prompt to sharpen the escalation policy, for example "Transfer to a human whenever the caller mentions a refund." See [Tools](/agents/build/tools) for how the agent chooses between its tools. @@ -83,13 +83,13 @@ The built-in instructions are conservative: the agent transfers only when the ca The agent tells the caller it is transferring them, and once that sentence finishes playing the call is handed to the carrier. The caller hears a dial tone while the destination rings; the agent drops out and its session ends at the handoff. -If the handoff is refused — the destination is unreachable, or the number doesn't support transfers — the agent stays on the line, apologizes, and keeps helping. It can attempt the transfer again later. +If the handoff is refused (the destination is unreachable, or the number doesn't support transfers), the agent stays on the line, apologizes, and keeps helping. It can attempt the transfer again later. ## What happens on a warm transfer - The agent asks the caller to hold for a moment — in the session's language — + The agent asks the caller to hold for a moment, in the session's language, and hold music starts. @@ -109,18 +109,18 @@ If the handoff is refused — the destination is unreachable, or the number does -If the human cannot be reached, declines, or the consult call hits voicemail, the caller comes off hold and the agent apologizes and continues helping — it can retry if the caller asks. If the caller hangs up while on hold, the agent briefly tells the human what happened and ends the consult call. +If the human cannot be reached, declines, or the consult call hits voicemail, the caller comes off hold and the agent apologizes and continues helping. It can retry if the caller asks. If the caller hangs up while on hold, the agent briefly tells the human what happened and ends the consult call. ## Limitations -- **Phone calls only** — web and SDK sessions have no phone leg to hand off, so the `transfer_call` tool never ships for them. -- **Transfer-capable numbers** — the number the caller dialed must be a `twilio`-provider number. -- **One destination** — each agent supports a single transfer destination for now. -- **Supported countries** — destination numbers are limited to an allowlist of country codes. +- **Phone calls only**: web and SDK sessions have no phone leg to hand off, so the `transfer_call` tool never ships for them. +- **Transfer-capable numbers**: the number the caller dialed must be a `twilio`-provider number. +- **One destination**: each agent supports a single transfer destination for now. +- **Supported countries**: destination numbers are limited to an allowlist of country codes. ## Billing -Transferred call time is metered separately from agent time, with cold and warm transfers each billed at their own per-minute rate. The standard agent rate applies only while the agent itself is on the call — it stops at the handoff for cold transfers and at the merge for warm transfers, and the connected call is metered as transfer minutes from that point on. +Transferred call time is metered separately from agent time, with cold and warm transfers each billed at their own per-minute rate. The standard agent rate stops at the handoff for cold transfers and at the merge for warm transfers; from that point the call is metered as transfer minutes. ## Going further diff --git a/agents/test/agent-tests.mdx b/agents/test/agent-tests.mdx index d4cf1e7..b58c097 100644 --- a/agents/test/agent-tests.mdx +++ b/agents/test/agent-tests.mdx @@ -1,10 +1,10 @@ --- title: "Agent Tests" -description: "Script conversations and let an LLM judge score your agent's replies — catch regressions before you publish" +description: "Script conversations and let an LLM judge score your agent's replies to catch regressions before you publish" icon: "vial" --- -Write a conversation once, run it against any agent, and get a **Pass** or **Fail** verdict with the judge's reasoning. Tests live in a shared workspace library, run against your agent's current draft, and never touch what's published — so you can iterate on a prompt and re-run in seconds. +Write a conversation once, run it against any agent, and get a **Pass** or **Fail** verdict with the judge's reasoning. Tests live in a shared workspace library, run against your agent's current draft, and never touch what's published, so you can iterate on a prompt and re-run in seconds. ## How a test works @@ -15,7 +15,7 @@ A **Single Turn** test hands your agent a scripted conversation and asks it to p 3. An LLM judge scores the reply against your **Expectation**, optionally calibrated by success and failure examples, and returns a verdict with its reasoning. - Tests run as text — no audio is synthesized — but the agent uses its full + Tests run as text (no audio is synthesized), but the agent uses its full draft configuration: the [knowledge base](/agents/build/knowledge-base) is consulted, and the agent can invoke its attached tools while generating the reply. [Webhook tools](/agents/build/webhook-tools) send real HTTP requests @@ -34,11 +34,11 @@ Tests are workspace-level resources, managed under **Library → Tests** in the Under **Conversation**, click **Add message** to build the history the agent - sees — each message is either an **Agent** or **User** turn. When the test + sees. Each message is either an **Agent** or **User** turn. When the test runs, the agent generates the reply that comes next. - Under **Judging**, write the **Expectation** — what a correct reply must do. + Under **Judging**, write the **Expectation**: what a correct reply must do. Optionally click **Add example** to provide success and failure examples; they calibrate the judge but aren't required. @@ -57,8 +57,8 @@ Tests are workspace-level resources, managed under **Library → Tests** in the A test only runs against agents it's attached to. Attach from either side: -- **From the library** — open the test's **Access** tab and toggle it on for each agent. -- **From the Builder** — on the agent's **Tests** page, click **Add tests** and pick from the library. +- **From the library**: open the test's **Access** tab and toggle it on for each agent. +- **From the Builder**: on the agent's **Tests** page, click **Add tests** and pick from the library. One test can be attached to many agents, and each agent keeps its own last result. **Remove from agent** detaches the test from that agent only; **Delete** in the library removes the test from all agents. @@ -77,22 +77,22 @@ If the run can't complete, the verdict shows **Error** with the error message in ## Run every test for an agent -On the agent's **Tests** page in the Builder, click **Run all**. Each row moves through **queued → running → Pass/Fail** — or **Error** if a run can't complete — and the page header summarizes the latest batch — for example, `4 passed, 1 failed on last run`. Use the row menu to re-run a single test, edit it, or remove it from the agent. +On the agent's **Tests** page in the Builder, click **Run all**. Each row moves through **queued → running → Pass/Fail** (or **Error** if a run can't complete), and the page header summarizes the latest batch, for example `4 passed, 1 failed on last run`. Use the row menu to re-run a single test, edit it, or remove it from the agent. ## Tests run against the draft -Tests always exercise the agent's latest **draft** configuration — including unpublished changes to the system prompt. That makes the loop fast: +Tests always exercise the agent's latest **draft** configuration, including unpublished changes to the system prompt. That makes the loop fast: Change the system prompt or first message in [Configuration](/agents/build/configuration). - Click **Run all** — no publish needed. + Click **Run all**. Once results look right, [publish the draft](/agents/deploy/versions-publishing). Running tests never publishes - anything and doesn't affect your agent's publish state. + anything. diff --git a/agents/test/preview-calls.mdx b/agents/test/preview-calls.mdx index aa041ed..560be79 100644 --- a/agents/test/preview-calls.mdx +++ b/agents/test/preview-calls.mdx @@ -4,13 +4,13 @@ description: "Talk to your draft agent in a live voice call, right in the Builde icon: "headset" --- -Preview calls let you hold a live voice conversation with your agent without leaving the Builder. Every call runs against your current **draft** configuration — edit the system prompt, pick a different voice, and call again to hear the difference immediately, no publish required. +Preview calls let you hold a live voice conversation with your agent without leaving the Builder. Every call runs against your current **draft** configuration: edit the system prompt, pick a different voice, and call again to hear the difference immediately, no publish required. ## Start a preview call - Preview works from any Builder section — Configuration, Tools, Knowledge + Preview works from any Builder section: Configuration, Tools, Knowledge base, and so on. @@ -19,7 +19,7 @@ Preview calls let you hold a live voice conversation with your agent without lea Your browser asks for microphone access the first time. The call cannot - start without a microphone — if you deny permission, the panel tells you to + start without a microphone: if you deny permission, the panel tells you to allow access in your browser settings and try again. @@ -29,7 +29,7 @@ Preview calls let you hold a live voice conversation with your agent without lea -The panel stays connected while you switch between Builder sections, so you can read your configuration mid-call. Navigating away from the Builder — back to the agent list, for example — hangs up. If the call fails to connect, the panel shows an error state and lets you retry; a connection drop mid-call simply ends the call. +The panel stays connected while you switch between Builder sections, so you can read your configuration mid-call. Navigating away from the Builder (back to the agent list, for example) hangs up. If the call fails to connect, the panel shows an error state and lets you retry; a connection drop mid-call simply ends the call. ## During the call @@ -46,8 +46,8 @@ When the agent invokes a [tool](/agents/build/tools), the call appears inline in Preview calls run your real configuration. [Webhook - tools](/agents/build/webhook-tools) send real HTTP requests to your endpoints - — point them at a staging environment if you want to avoid side effects while + tools](/agents/build/webhook-tools) send real HTTP requests to your endpoints. + Point them at a staging environment if you want to avoid side effects while testing. @@ -57,7 +57,7 @@ When the agent invokes a [tool](/agents/build/tools), the call appears inline in | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | Mute | Toggles your microphone without ending the call | | End call | Ends the call and returns the panel to its ready state | -| Timer | Shows elapsed time; preview calls are capped at 10 minutes (600 seconds) — or the agent's configured maximum call duration, whichever is lower | +| Timer | Shows elapsed time; preview calls are capped at 10 minutes (600 seconds), or the agent's configured maximum call duration, whichever is lower | ## Drafts, not published versions @@ -68,7 +68,7 @@ Preview calls and production sessions read different configurations: | Preview call | Latest draft, autosaved as you edit in the Builder | | Production session | The published version, unchanged until you publish again | -This makes preview the fast inner loop: tweak the prompt, call, listen, repeat. Your published version — and every live integration using it — is untouched until you decide the draft is ready and [publish it](/agents/deploy/versions-publishing). +This makes preview the fast inner loop: tweak the prompt, call, listen, repeat, and [publish](/agents/deploy/versions-publishing) when the draft is ready. After a config change, hang up and start a new call. A preview call reflects @@ -79,7 +79,7 @@ This makes preview the fast inner loop: tweak the prompt, call, listen, repeat. Preview sessions are built for iteration, not record-keeping: -- **Post-call analysis runs in the panel.** If you have [post-call analysis](/agents/monitor/post-call-analysis) configured, ending the call triggers it and the results — summary, extracted data fields, and per-criterion success / failure / unknown verdicts — appear directly in the panel. Analysis runs however the call ends — including the automatic time limit or closing the tab mid-call — but results are viewable only in the panel, so close the tab and you won't see them. +- **Post-call analysis runs in the panel.** If you have [post-call analysis](/agents/monitor/post-call-analysis) configured, ending the call triggers it and the results (summary, extracted data fields, and per-criterion success / failure / unknown verdicts) appear directly in the panel. Analysis runs however the call ends (including the automatic time limit or closing the tab mid-call), but results are viewable only in the panel. - **Nothing enters conversation history.** Preview calls never appear in [conversation history](/agents/monitor/conversation-history), and the transcript is not preserved after you hang up. - **No post-call webhook fires.** Preview sessions never trigger the `call.analyzed` [post-call webhook](/agents/monitor/webhooks), so Builder debugging doesn't push analysis results to your production systems. @@ -87,7 +87,7 @@ Preview sessions are built for iteration, not record-keeping: - Scripted conversations with automated Pass/Fail judging — catch regressions + Scripted conversations with automated Pass/Fail judging: catch regressions without listening by ear. - Configure summaries, data extraction, and success criteria — results show up + Configure summaries, data extraction, and success criteria. Results show up in the preview panel. diff --git a/api-reference/agent-errors.mdx b/api-reference/agent-errors.mdx index 2490e85..10756b6 100644 --- a/api-reference/agent-errors.mdx +++ b/api-reference/agent-errors.mdx @@ -32,11 +32,11 @@ The exceptions to that shape: | Status | Meaning | What to do | | ------ | ------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------- | -| `400` | The request is well-formed JSON but semantically wrong | Read the `message` — it names the field or rule. | +| `400` | The request is well-formed JSON but semantically wrong | Read the `message`; it names the field or rule. | | `401` | Missing, invalid, or expired API key | Send `Authorization: Bearer `; check it on [API Keys](https://fish.audio/app/api-keys). | -| `402` | Out of API credit | Top up. Only session creation and phone-number purchase charge up front — a `402` never leaves a partial charge. | +| `402` | Out of API credit | Top up. Only session creation and phone-number purchase charge up front, so a `402` never leaves a partial charge. | | `403` | The action is not allowed for this caller | See [403 causes](#403-what-was-refused) below. | -| `404` | The resource doesn't exist — or isn't yours | Cross-team access returns the same `404` as a missing id, so treat both identically. | +| `404` | The resource doesn't exist, or isn't yours | Cross-team access returns the same `404` as a missing id, so treat both identically. | | `409` | A state conflict blocks the action | See [409 conflicts](#409-conflicts) below. | | `415` | Missing body or wrong `Content-Type` | JSON endpoints need `application/json`; knowledge-source uploads need `multipart/form-data`. | | `422` | Field-level validation failed | Fix the fields listed in the error array. | @@ -50,23 +50,23 @@ The exceptions to that shape: Two quirks worth coding around: -- **Explicit `null` is rejected** on `PATCH` endpoints with `422` — omit a field to keep its value, send `""` to clear a text field. -- **Path-parameter validation renders `404`, not `422`** — a non-integer `{version_number}` looks like a missing version. +- **Explicit `null` is rejected** on `PATCH` endpoints with `422`: omit a field to keep its value, send `""` to clear a text field. +- **Path-parameter validation renders `404`, not `422`**: a non-integer `{version_number}` looks like a missing version. ## 403: what was refused | Message | Cause | | -------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | | `Agent is not public` | Anonymous session/widget request against an agent whose public switch is off. | -| `Origin header is required for public agents` | Anonymous request without an `Origin` header — browsers send it automatically, curl must set it. | +| `Origin header is required for public agents` | Anonymous request without an `Origin` header. Browsers send it automatically, curl must set it. | | `Origin not allowed` | The page's origin is not on the agent's [allowed origins](/agents/deploy/public-agents#origin-matching). | | `Agent is unavailable` | The agent's owner can't serve public sessions right now; visitors deliberately can't tell why. | -| `Agent platform is in private beta. Apply for access at https://fish.audio/app/agents` | Creating agent resources requires beta access on your account — sign in and submit the application form at that link. | +| `Agent platform is in private beta. Apply for access at https://fish.audio/app/agents` | Creating agent resources requires beta access on your account. Sign in and submit the application form at that link. | | `Phone number limit reached for this team (N)` | The team holds its maximum of live numbers; release one first. | ## 404: anti-enumeration -A resource that exists but belongs to another team returns the **identical** `404` body as one that never existed — for agents, sessions, tools, knowledge sources, phone numbers, and versions. Released phone numbers and deleted knowledge sources behave the same. Never use `404` vs `403` to probe for existence; there is no distinction to find. +A resource that exists but belongs to another team returns the **identical** `404` body as one that never existed. This applies to agents, sessions, tools, knowledge sources, phone numbers, and versions. Released phone numbers and deleted knowledge sources behave the same. Never use `404` vs `403` to probe for existence. The recording endpoint adds one more meaning: `Recording not found` on a real session means it was [never recorded](/agents/monitor/conversation-history#what-gets-stored). @@ -74,20 +74,20 @@ The recording endpoint adds one more meaning: `Recording not found` on a real se | Message | Cause and fix | | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | -| `Agent has no published version` | Session creation against a never-published agent — [publish](/agents/deploy/versions-publishing) first. | -| `Knowledge source is attached to N agent(s)…` | Deletion is blocked while any agent references the source in its draft **or published** config — detach everywhere, republish, then delete. | -| `Tool is attached to N agent(s)…` | Deletion is blocked while any agent's **draft** references the tool (published snapshots are frozen copies and don't block) — detach, then delete. | -| `This number is already on the platform` | The number is held by a team already — pick another from [search](/api-reference/endpoint/agent/search-available-phone-numbers). | +| `Agent has no published version` | Session creation against a never-published agent: [publish](/agents/deploy/versions-publishing) first. | +| `Knowledge source is attached to N agent(s)…` | Deletion is blocked while any agent references the source in its draft **or published** config: detach everywhere, republish, then delete. | +| `Tool is attached to N agent(s)…` | Deletion is blocked while any agent's **draft** references the tool (published snapshots are frozen copies and don't block): detach, then delete. | +| `This number is already on the platform` | The number is held by a team already; pick another from [search](/api-reference/endpoint/agent/search-available-phone-numbers). | ## Billing and rate limits -Exactly two endpoints charge before acting, and both can return `402 Out of API credit`: [`POST /v1/agent/sessions`](/api-reference/endpoint/agent/create-agent-session) (API-key callers) and [`POST /v1/agent/phone-numbers`](/api-reference/endpoint/agent/purchase-phone-number) (the first rental day). Anonymous public sessions never see `402` — an out-of-credit owner surfaces to visitors as `403 Agent is unavailable`. Usage is settled after each call and never fails a request. +Exactly two endpoints charge before acting, and both can return `402 Out of API credit`: [`POST /v1/agent/sessions`](/api-reference/endpoint/agent/create-agent-session) (API-key callers) and [`POST /v1/agent/phone-numbers`](/api-reference/endpoint/agent/purchase-phone-number) (the first rental day). Anonymous public sessions never see `402`. An out-of-credit owner surfaces to visitors as `403 Agent is unavailable`. Usage is settled after each call and never fails a request. -`429` applies only to anonymous session creation on [public agents](/agents/deploy/public-agents), in fixed per-minute windows per agent and per client IP. API-key traffic is not rate limited on this surface — your backend is the gate. +`429` applies only to anonymous session creation on [public agents](/agents/deploy/public-agents), in fixed per-minute windows per agent and per client IP. API-key traffic is not rate limited on this surface; your backend is the gate. ## 5xx -`502` names the failing upstream in the message: the conversation gateway (`Agent gateway is unreachable`) or the telephony provider (`Twilio refused the request: …`). Retrying is safe — a failed phone-number purchase leaves the row visible with status `error`, refunds the day charge, and can be released; a failed release keeps the number live so releasing again retries. `503` means a platform dependency was briefly unreachable; retry with backoff. +`502` names the failing upstream in the message: the conversation gateway (`Agent gateway is unreachable`) or the telephony provider (`Twilio refused the request: …`). Retrying is safe. A failed phone-number purchase leaves the row visible with status `error`, refunds the day charge, and can be released; a failed release keeps the number live so releasing again retries. `503` means a platform dependency was briefly unreachable; retry with backoff. ## Going further @@ -97,7 +97,7 @@ Exactly two endpoints charge before acting, and both can return `402 Out of API icon="server" href="/agents/deploy/authenticated-sessions" > - The session token flow — request fields, lifetime, errors. + The session token flow: request fields, lifetime, errors. Origin allowlists and the errors unique to keyless access. diff --git a/api-reference/endpoint/openapi-v1/speech-to-text.mdx b/api-reference/endpoint/openapi-v1/speech-to-text.mdx index 4c51d08..3475e1a 100644 --- a/api-reference/endpoint/openapi-v1/speech-to-text.mdx +++ b/api-reference/endpoint/openapi-v1/speech-to-text.mdx @@ -7,5 +7,5 @@ iconType: "solid" --- -This BETA endpoint accepts audio only as `multipart/form-data` (a file upload) or `application/msgpack`. JSON is not supported — the audio cannot be sent as a base64 string. +This BETA endpoint accepts audio only as `multipart/form-data` (a file upload) or `application/msgpack`. JSON with base64-encoded audio is not supported. diff --git a/api-reference/errors.mdx b/api-reference/errors.mdx index 10d2648..54a3501 100644 --- a/api-reference/errors.mdx +++ b/api-reference/errors.mdx @@ -16,7 +16,7 @@ Every Fish Audio error comes back as JSON with a `message` and a `status`: | Status | Meaning | What to do | |---|---|---| -| `400` | Bad request — invalid parameters, or a `reference_id` / voice that doesn't exist | Fix the request; read the `message`. | +| `400` | Bad request: invalid parameters, or a `reference_id` / voice that doesn't exist | Fix the request; read the `message`. | | `401` | Invalid or missing API key | Send `Authorization: Bearer `; check it on [API Keys](https://fish.audio/app/api-keys). | | `402` | Insufficient credits | Top up on [Billing](https://fish.audio/app/billing). | | `403` | Not permitted for this key/resource | Check the key's scope and the resource owner. | @@ -26,7 +26,7 @@ Every Fish Audio error comes back as JSON with a `message` and a `status`: ## Retries -Retry `429` and `5xx` with exponential backoff. Don't retry other `4xx` codes — they won't succeed without a change to the request. +Retry `429` and `5xx` with exponential backoff. Don't retry other `4xx` codes; they won't succeed without a change to the request. ```python import time @@ -45,7 +45,7 @@ for attempt in range(5): if e.status >= 500: time.sleep(2 ** attempt) else: - raise # 4xx — fix the request + raise # 4xx: fix the request ``` ## Handling errors in the SDKs @@ -58,7 +58,7 @@ from fishaudio.exceptions import ( AuthenticationError, # 401 RateLimitError, # 429 NotFoundError, # 404 - APIError, # any other HTTP error — has .status and .message + APIError, # any other HTTP error; has .status and .message FishAudioError, # base class for all SDK errors ) @@ -81,7 +81,7 @@ import { NotFoundError, // 404 BadRequestError, // 400 UnprocessableEntityError, // 422 - FishAudioError, // base — has .statusCode and .body + FishAudioError, // base; has .statusCode and .body } from "fish-audio"; try { @@ -100,5 +100,5 @@ try { Audio playback via `play()` needs `ffmpeg`. If it's missing, the Python SDK raises - `DependencyError` — install `ffmpeg` or save the audio to a file instead. + `DependencyError`. Install `ffmpeg` or save the audio to a file instead. diff --git a/api-reference/openapi.json b/api-reference/openapi.json index 1f81f06..d22e5ba 100644 --- a/api-reference/openapi.json +++ b/api-reference/openapi.json @@ -491,7 +491,7 @@ "/v1/agent/sessions": { "get": { "summary": "List Agent Sessions", - "description": "List your team's sessions, newest first. Filter by agent, status, caller\nnumber, or creation time. Paginate with `cursor` (recommended; follow\n`next_cursor` while `has_more` is true) or with `page` for offset pagination\nwith a `total` count — the two are mutually exclusive.", + "description": "List your team's sessions, newest first. Filter by agent, status, caller\nnumber, or creation time. Paginate with `cursor` (recommended; follow\n`next_cursor` while `has_more` is true) or with `page` for offset pagination\nwith a `total` count; the two are mutually exclusive.", "security": [ { "BearerAuth": [] @@ -2128,7 +2128,7 @@ "/v1/agent/agents": { "get": { "summary": "List Agents", - "description": "List your team's agents, newest first. Paginate with `cursor` (follow\n`next_cursor` while `has_more` is true) or with `page` for offset\npagination with a `total` count — the two are mutually exclusive.", + "description": "List your team's agents, newest first. Paginate with `cursor` (follow\n`next_cursor` while `has_more` is true) or with `page` for offset\npagination with a `total` count; the two are mutually exclusive.", "security": [ { "BearerAuth": [] @@ -2420,7 +2420,7 @@ }, "post": { "summary": "Create Agent", - "description": "Create an agent, optionally with its full initial configuration inline —\none call provisions a ready-to-publish agent. The agent starts as a draft:\npublish it before creating sessions with it.", + "description": "Create an agent, optionally with its full initial configuration inline.\nThe agent starts as a draft: publish it before creating sessions with it.", "security": [ { "BearerAuth": [] @@ -2921,7 +2921,7 @@ }, "patch": { "summary": "Update Agent", - "description": "Update agent-level fields (name, description, status, public access and\nsession-override policy). Omitted fields keep their value. Conversation\nbehavior — voice, prompt, recording and the rest — is draft configuration:\nuse `PATCH /v1/agent/agents/{agent_id}/config`.", + "description": "Update agent-level fields (name, description, status, public access and\nsession-override policy). Omitted fields keep their value. Conversation\nbehavior (voice, prompt, recording and the rest) is draft configuration:\nuse `PATCH /v1/agent/agents/{agent_id}/config`.", "security": [ { "BearerAuth": [] @@ -3341,7 +3341,7 @@ "/v1/agent/agents/{agent_id}/widget": { "get": { "summary": "Get Widget Config", - "description": "Unauthenticated display configuration for the embeddable ``\nwidget. Only agents published as public are reachable, and the request\n`Origin` must match the agent's allowed origins — the same gate as\nanonymous session creation. Attributes set on the embed tag override\nevery field returned here.", + "description": "Unauthenticated display configuration for the embeddable ``\nwidget. Only agents published as public are reachable, and the request\n`Origin` must match the agent's allowed origins, the same gate as\nanonymous session creation. Attributes set on the embed tag override\nevery field returned here.", "parameters": [ { "in": "path", @@ -3381,7 +3381,7 @@ "content": { "application/json": { "schema": { - "description": "Display config for the embeddable `` widget.\n\nServed without authentication for public agents; HTML attributes on the\nembed tag override every field. Grows alongside the Builder widget tab\n(greeting, policy switches, styles) — additions only, never renames.", + "description": "Display config for the embeddable `` widget.\n\nServed without authentication for public agents; HTML attributes on the\nembed tag override every field. Grows alongside the Builder widget tab\n(greeting, policy switches, styles): additions only, never renames.", "properties": { "agent_name": { "title": "Agent Name", @@ -3479,7 +3479,7 @@ "/v1/agent/agents/{agent_id}/config": { "get": { "summary": "Get Draft Config", - "description": "Read the agent's current draft configuration — the state the next publish\nwill freeze. Sessions run the latest published version, not the draft; use\nthe versions endpoints to see what is live. Webhook signing secrets are\nwrite-only and reported as `has_secret`.", + "description": "Read the agent's current draft configuration, the state the next publish\nwill freeze. Sessions run the latest published version, not the draft; use\nthe versions endpoints to see what is live. Webhook signing secrets are\nwrite-only and reported as `has_secret`.", "security": [ { "BearerAuth": [] @@ -4495,7 +4495,7 @@ "/v1/agent/agents/{agent_id}/versions/{version_number}": { "get": { "summary": "Get Agent Version", - "description": "One published version with its full frozen configuration snapshot —\nincluding the current live version. Secrets inside the snapshot are\nredacted the same way as the draft config.", + "description": "One published version with its full frozen configuration snapshot,\nincluding the current live version. Secrets inside the snapshot are\nredacted the same way as the draft config.", "security": [ { "BearerAuth": [] @@ -4964,7 +4964,7 @@ "/v1/agent/knowledge-sources": { "get": { "summary": "List Knowledge Sources", - "description": "List your team's knowledge sources, newest first. Paginate with `cursor`\n(follow `next_cursor` while `has_more` is true) or with `page` for offset\npagination with a `total` count — the two are mutually exclusive.", + "description": "List your team's knowledge sources, newest first. Paginate with `cursor`\n(follow `next_cursor` while `has_more` is true) or with `page` for offset\npagination with a `total` count; the two are mutually exclusive.", "security": [ { "BearerAuth": [] @@ -5751,7 +5751,7 @@ }, "patch": { "summary": "Update Knowledge Source", - "description": "Rename the source and/or replace its content by uploading a new file in\n`source` (multipart). Replacing content increments `revision_number` while\nthe id stays stable, and every attached agent's draft picks it up\nimmediately; published versions pin the revision they were published with,\nso republish each affected agent to put the new content live — that\ntwo-step (sync content, republish) is the nightly content-sync recipe.", + "description": "Rename the source and/or replace its content by uploading a new file in\n`source` (multipart). Replacing content increments `revision_number` while\nthe id stays stable, and every attached agent's draft picks it up\nimmediately; published versions pin the revision they were published with,\nso republish each affected agent to put the new content live.", "security": [ { "BearerAuth": [] @@ -6086,7 +6086,7 @@ }, "delete": { "summary": "Delete Knowledge Source", - "description": "Delete a knowledge source. Returns 409 while any agent still references it\n— in its draft or in its currently published version (sessions resolve\nsources at call time, so deleting a published reference would change a\nrunning agent). Check `GET /v1/agent/knowledge-sources/{source_id}/agents`,\ndetach via the agent config, and republish if needed before deleting.", + "description": "Delete a knowledge source. Returns 409 while any agent still references it,\nin its draft or in its currently published version (sessions resolve\nsources at call time, so deleting a published reference would change a\nrunning agent). Check `GET /v1/agent/knowledge-sources/{source_id}/agents`,\ndetach via the agent config, and republish if needed before deleting.", "security": [ { "BearerAuth": [] @@ -6267,7 +6267,7 @@ "/v1/agent/knowledge-sources/{source_id}/agents": { "get": { "summary": "List Agents Using Knowledge Source", - "description": "Every agent that references this source in its draft or currently\npublished configuration — the pre-flight check before a delete.", + "description": "Every agent that references this source in its draft or currently\npublished configuration: the pre-flight check before a delete.", "security": [ { "BearerAuth": [] @@ -6436,7 +6436,7 @@ "/v1/agent/tools": { "get": { "summary": "List Tools", - "description": "List your team's tools, newest first. Filter with `agent_id` to see one\nagent's attached tools. Paginate with `cursor` (follow `next_cursor` while\n`has_more` is true) or with `page` for offset pagination with a `total`\ncount — the two are mutually exclusive.", + "description": "List your team's tools, newest first. Filter with `agent_id` to see one\nagent's attached tools. Paginate with `cursor` (follow `next_cursor` while\n`has_more` is true) or with `page` for offset pagination with a `total`\ncount; the two are mutually exclusive.", "security": [ { "BearerAuth": [] @@ -7038,7 +7038,7 @@ "/v1/agent/tools/{tool_id}": { "get": { "summary": "Get Tool", - "description": "Fetch one tool's full definition. Credential header values are never\nreturned — each credential header reports `has_secret` instead.", + "description": "Fetch one tool's full definition. Credential header values are never\nreturned; each credential header reports `has_secret` instead.", "security": [ { "BearerAuth": [] @@ -7315,7 +7315,7 @@ }, "patch": { "summary": "Update Tool", - "description": "Patch tool fields; omitted fields keep their value (null is rejected —\nsend an empty string to clear a text field). `headers` replaces the header\nlist wholesale — include credential values again whenever you send it,\nsince reads never return them. Attached agents' drafts pick up the change\nimmediately; published versions stay frozen until re-published.", + "description": "Patch tool fields; omitted fields keep their value (null is rejected;\nsend an empty string to clear a text field). `headers` replaces the header\nlist wholesale. Include credential values again whenever you send it,\nsince reads never return them. Attached agents' drafts pick up the change\nimmediately; published versions stay frozen until re-published.", "security": [ { "BearerAuth": [] @@ -7676,7 +7676,7 @@ }, "delete": { "summary": "Delete Tool", - "description": "Delete a tool. Returns 409 while any agent's draft configuration still\nreferences it — check `GET /v1/agent/tools/{tool_id}/agents` and detach it\nvia the agent config first, so a delete can never silently change agent\nbehavior.", + "description": "Delete a tool. Returns 409 while any agent's draft configuration still\nreferences it. Check `GET /v1/agent/tools/{tool_id}/agents` and detach it\nvia the agent config first, so a delete can never silently change agent\nbehavior.", "security": [ { "BearerAuth": [] @@ -7857,7 +7857,7 @@ "/v1/agent/tools/{tool_id}/agents": { "get": { "summary": "List Agents Using Tool", - "description": "Every agent whose draft configuration references this tool — the\npre-flight check before a delete. Published versions keep executing their\nfrozen tool snapshot, so only draft references block deletion.", + "description": "Every agent whose draft configuration references this tool: the\npre-flight check before a delete. Published versions keep executing their\nfrozen tool snapshot, so only draft references block deletion.", "security": [ { "BearerAuth": [] @@ -8026,7 +8026,7 @@ "/v1/agent/phone-numbers": { "get": { "summary": "List Phone Numbers", - "description": "List your team's phone numbers, newest first. Released numbers are gone\nfor good and never appear. Look an id up by E.164 with `phone_number`, or\nfilter with `agent_id` to see one agent's numbers. Paginate with `cursor`\n(follow `next_cursor` while `has_more` is true) or with `page` for offset\npagination with a `total` count — the two are mutually exclusive.", + "description": "List your team's phone numbers, newest first. Released numbers are gone\nfor good and never appear. Look an id up by E.164 with `phone_number`, or\nfilter with `agent_id` to see one agent's numbers. Paginate with `cursor`\n(follow `next_cursor` while `has_more` is true) or with `page` for offset\npagination with a `total` count; the two are mutually exclusive.", "security": [ { "BearerAuth": [] @@ -8345,7 +8345,7 @@ }, "post": { "summary": "Purchase Phone Number", - "description": "Buy a number from the inventory. The number lands in your default\nworkspace, and any `agent_id` you bind must live there too. Billing is the\nmonthly price charged in daily slices: the first day is charged before\nanything is bought (402 costs you nothing), and the daily run advances it\nfrom there. 409 means the number is already on the platform; 502 means the\nprovider refused the purchase — the number stays visible with status\n`error` and is safe to release.", + "description": "Buy a number from the inventory. The number lands in your default\nworkspace, and any `agent_id` you bind must live there too. Billing is the\nmonthly price charged in daily slices: the first day is charged before\nanything is bought (402 costs you nothing), and the daily run advances it\nfrom there. 409 means the number is already on the platform; 502 means the\nprovider refused the purchase; the number stays visible with status\n`error` and is safe to release.", "security": [ { "BearerAuth": [] @@ -8979,7 +8979,7 @@ }, "patch": { "summary": "Update Phone Number", - "description": "Change the label and/or repoint the number at another agent — the\ndeployment-pipeline move (rebind from the staging agent to the production\none). Send `agent_id: null` to unbind; unbound numbers ring busy. The\nagent must live in the number's workspace. Rebinding is a routing-table\nupdate resolved on the next inbound call; nothing about the number itself\nis reprovisioned.", + "description": "Change the label and/or repoint the number at another agent: the\ndeployment-pipeline move (rebind from the staging agent to the production\none). Send `agent_id: null` to unbind; unbound numbers ring busy. The\nagent must live in the number's workspace. Rebinding is a routing-table\nupdate resolved on the next inbound call; nothing about the number itself\nis reprovisioned.", "security": [ { "BearerAuth": [] @@ -9251,7 +9251,7 @@ }, "delete": { "summary": "Release Phone Number", - "description": "Release a number back to the provider's inventory and stop its daily\nbilling. This is irreversible: anyone — including other platforms — can\nbuy the number afterwards, so callers who saved it may reach a stranger.\nThe number disappears from this API immediately.", + "description": "Release a number back to the provider's inventory and stop its daily\nbilling. This is irreversible: anyone (including other platforms) can\nbuy the number afterwards, so callers who saved it may reach a stranger.\nThe number disappears from this API immediately.", "security": [ { "BearerAuth": [] @@ -13026,7 +13026,7 @@ }, "schemas": { "AgentSessionSummary": { - "description": "One list row: lifecycle facts only — the conversation timeline lives on\nthe detail endpoint (Retell-style thin list).", + "description": "One list row: lifecycle facts only; the conversation timeline lives on\nthe detail endpoint (Retell-style thin list).", "properties": { "session_id": { "title": "Session Id", @@ -16303,7 +16303,7 @@ "type": "object" }, "TTSRequest": { - "description": "Request body for text-to-speech synthesis. Supports single-speaker synthesis on all compatible TTS models. Multi-speaker dialogue synthesis is only available with the S2 family (`s2-pro`, `s2.1-pro`, `s2.1-pro-free`), not `s1`.\n\n## Single Speaker\nProvide either `reference_id` (string) pointing to a voice model, or `references` (array of ReferenceAudio) for zero-shot cloning.\n\n## Multiple Speakers (Dialogue — S2 family only)\nFor multi-speaker synthesis, provide:\n- `reference_id`: array of voice model IDs, e.g., [\"speaker-0-id\", \"speaker-1-id\"]\n- `text`: use speaker tags `<|speaker:0|>`, `<|speaker:1|>`, etc. to indicate speaker changes, e.g., \"<|speaker:0|>Hello!<|speaker:1|>Hi there!\"\n\nAlternatively, for zero-shot multi-speaker:\n- `references`: 2D array where each inner array contains references for one speaker\n- `reference_id`: array of identifiers (can be arbitrary strings for zero-shot)\n\n## Example (Multi-Speaker with Model IDs)\n```json\n{\n \"text\": \"<|speaker:0|>Good morning!<|speaker:1|>Good morning! How are you?<|speaker:0|>I'm great, thanks!\",\n \"reference_id\": [\"model-id-alice\", \"model-id-bob\"]\n}\n```", + "description": "Request body for text-to-speech synthesis. Supports single-speaker synthesis on all compatible TTS models. Multi-speaker dialogue synthesis is only available with the S2 family (`s2-pro`, `s2.1-pro`, `s2.1-pro-free`), not `s1`.\n\n## Single Speaker\nProvide either `reference_id` (string) pointing to a voice model, or `references` (array of ReferenceAudio) for zero-shot cloning.\n\n## Multiple Speakers (Dialogue, S2 family only)\nFor multi-speaker synthesis, provide:\n- `reference_id`: array of voice model IDs, e.g., [\"speaker-0-id\", \"speaker-1-id\"]\n- `text`: use speaker tags `<|speaker:0|>`, `<|speaker:1|>`, etc. to indicate speaker changes, e.g., \"<|speaker:0|>Hello!<|speaker:1|>Hi there!\"\n\nAlternatively, for zero-shot multi-speaker:\n- `references`: 2D array where each inner array contains references for one speaker\n- `reference_id`: array of identifiers (can be arbitrary strings for zero-shot)\n\n## Example (Multi-Speaker with Model IDs)\n```json\n{\n \"text\": \"<|speaker:0|>Good morning!<|speaker:1|>Good morning! How are you?<|speaker:0|>I'm great, thanks!\",\n \"reference_id\": [\"model-id-alice\", \"model-id-bob\"]\n}\n```", "properties": { "text": { "description": "Text to convert to speech.", diff --git a/api-reference/sdk/python/overview.mdx b/api-reference/sdk/python/overview.mdx index a555021..69030c0 100644 --- a/api-reference/sdk/python/overview.mdx +++ b/api-reference/sdk/python/overview.mdx @@ -185,7 +185,7 @@ async def text_chunks(): async def main(): async with AsyncFishAudio() as client: - # stream_websocket is an async generator — iterate it, don't await the call + # stream_websocket is an async generator: iterate it, don't await the call audio_stream = client.tts.stream_websocket(text_chunks(), latency="balanced") with open("out.mp3", "wb") as f: async for chunk in audio_stream: diff --git a/developer-guide/compat/capabilities.mdx b/developer-guide/compat/capabilities.mdx index 2dbea1f..9e63fdc 100644 --- a/developer-guide/compat/capabilities.mdx +++ b/developer-guide/compat/capabilities.mdx @@ -10,18 +10,18 @@ page. ## How parameters are handled -**Unsupported options return an explicit error — never a silent no-op.** Every +**Unsupported options return an explicit error, never a silent no-op.** Every parameter is either **mapped** to the native equivalent (`stability` → temperature, `speed` → speech rate, bitrates snap to a tier), **ignored** when it doesn't affect synthesis (`similarity_boost`, `style`, `seed`), or **refused** with a 4xx in the protocol's own error format when honoring it would -change the response content or structure — -[full list](#what-is-explicitly-refused). Unknown JSON fields are ignored, +change the response content or structure +([full list](#what-is-explicitly-refused)). Unknown JSON fields are ignored, matching OpenAI behavior. ## What each protocol supports -Two protocols: OpenAI at `/v1` (equivalently `/api/v1` and `/openai/v1` — the +Two protocols: OpenAI at `/v1` (equivalently `/api/v1` and `/openai/v1`; the OpenAI, OpenRouter, and Groq SDKs all speak it), and ElevenLabs at `/elevenlabs`. @@ -38,10 +38,10 @@ OpenAI, OpenRouter, and Groq SDKs all speak it), and ElevenLabs at | Voice library listing | – (not part of the protocol) | Yes (`/v1/voices`, `/v2/voices`) | | Model catalog | Yes (list; no retrieve-by-id) | Yes | -Voices are always Fish voice IDs — empty or `default` selects the model's +Voices are always Fish voice IDs. Empty or `default` selects the model's default voice, and anything that isn't a real voice ID, preset names (`nova`, `echo`, `Rachel`, …) included, is a 400. One exception: lowercase `alloy` -matches a real Fish voice and synthesizes — an unrelated voice, not OpenAI's +matches a real Fish voice and synthesizes as an unrelated voice, not OpenAI's Alloy (`Alloy` is refused). Where to find voice IDs: [Voices](/developer-guide/getting-started/migration#voices). @@ -64,10 +64,10 @@ reset, is in the [Realtime protocol reference](/developer-guide/compat/realtime-protocol#audio-formats). - **WAV responses carry a placeholder header — trust the byte count, not the + **WAV responses carry a placeholder header: trust the byte count, not the header.** The RIFF size field reads `4294967076` and the `data` chunk size `4294967040` however short the clip is, on buffered responses and streams - alike. Anything that reads the header gets the duration wrong — Python's + alike. Anything that reads the header gets the duration wrong: Python's `wave` module reports a one-second clip as 13.5 hours; players that decode to end-of-stream are unaffected. For a correct duration use the response length, re-mux (`ffmpeg -i in.wav -c copy out.wav`), or request MP3 or Opus. @@ -78,14 +78,14 @@ reset, is in the MP3 is encoded at 64, 128, or 192 kbps. An out-of-tier request snaps **up** to the nearest tier (`mp3_32000_32` → 64 kbps at 32 kHz); anything above 192 kbps is a 400. To pick the tier exactly, set it as a -[Fish-native parameter](#reaching-fish-native-parameters) — a non-tier value +[Fish-native parameter](#reaching-fish-native-parameters); a non-tier value there is refused too. Opus bitrate is automatic: requested values are accepted and ignored. ## What is explicitly refused -Each returns a 4xx in the protocol's own error format — or, on WebSockets, an -in-band error event: +Each returns a 4xx in the protocol's own error format (or, on WebSockets, an +in-band error event): | Endpoint | Refused options | | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | @@ -95,7 +95,7 @@ in-band error event: | OpenAI TTS | `response_format` of `aac` or `flac`; `instructions`; any `stream_format` other than `audio` | | OpenAI STT | `include`, `known_speaker_names`, `known_speaker_references` | | OpenAI chat | `n ≠ 1`; audio input combined with audio output in one call | -| OpenAI Realtime | `audio/pcmu`, `audio/pcma`; any `turn_detection` with a type (`server_vad`, `semantic_vad`, anything else — only `null` and `"none"` pass); `output_modalities` that omits `"audio"`; a second concurrent response (`response_already_active`) | +| OpenAI Realtime | `audio/pcmu`, `audio/pcma`; any `turn_detection` with a type (`server_vad`, `semantic_vad`, anything else; only `null` and `"none"` pass); `output_modalities` that omits `"audio"`; a second concurrent response (`response_already_active`) | On OpenAI transcription, `languages`, `keywords`, `prompt`, and `temperature` are in the [ignored tier](#how-parameters-are-handled). @@ -105,7 +105,7 @@ and `off` are treated as unset, so SDK defaults never trigger a refusal. ## Placeholder values -Protocol fields with no Fish counterpart are filled with neutral constants — +Protocol fields with no Fish counterpart are filled with neutral constants; don't build logic on them: - OpenAI `verbose_json`: the Whisper-engine internals (`avg_logprob`, @@ -115,16 +115,16 @@ don't build logic on them: is always `0`. Neither is a confidence score. - `speaker_id` is always `null`; requesting diarization returns an explicit error instead. -- `language` in responses echoes your request — it is never a detection result. +- `language` in responses echoes your request, never a detection result. Send none and you get `language: ""` alongside `language_probability: 1.0`. ## Reaching Fish-native parameters -The full native TTS parameter set — `temperature`, `top_p`, `chunk_length`, -`latency`, `normalize`, `prosody`, reference audio — is reachable on +The full native TTS parameter set (`temperature`, `top_p`, `chunk_length`, +`latency`, `normalize`, `prosody`, reference audio) is reachable on **synthesis endpoints**, in two ways. One exception: **reference audio is not available on the timestamped ElevenLabs endpoints** (`…/with-timestamps`, -`…/stream/with-timestamps`, the `stream-input` WebSocket) — a 400 on the REST +`…/stream/with-timestamps`, the `stream-input` WebSocket): a 400 on the REST variants, an in-band error frame at flush time on the WebSocket. ### `provider.options` in the request body @@ -173,14 +173,14 @@ curl https://api.fish.audio/compat/v1/audio/speech \ -Inline `references` — base64 audio plus its transcript — enable zero-shot +Inline `references` (base64 audio plus its transcript) enable zero-shot voice cloning through any OpenAI SDK. ### `X-Fish-*` request headers -On every synthesis endpoint — OpenAI `POST /v1/audio/speech` and chat +On every synthesis endpoint (OpenAI `POST /v1/audio/speech` and chat synthesis, the ElevenLabs endpoints, and OpenAI Realtime synthesis sessions, -including their WebSocket handshakes — an `X-Fish-` header sets the +including their WebSocket handshakes), an `X-Fish-` header sets the corresponding native TTS field: @@ -227,7 +227,7 @@ Values parse as JSON when possible, and headers win over mapped values. Neither channel applies to transcription: every STT endpoint returns a 400 when sent `provider.options.fish-audio` or `X-Fish-*`. That includes a -Realtime socket whose handshake carried `X-Fish-*` headers — the +Realtime socket whose handshake carried `X-Fish-*` headers: the `input_audio_buffer.commit` there is refused with `unsupported_native_options`, so open a second socket without the headers to transcribe. On the native `/v1/tts/live` WebSocket, put native fields in @@ -236,7 +236,7 @@ each `start.request` payload rather than in handshake headers. Synthesis on the Compatible APIs defaults to `latency: "low"` (streaming-optimized). For maximum quality on non-interactive workloads, set - `X-Fish-Latency: normal` — your explicit value always wins. + `X-Fish-Latency: normal`; your explicit value always wins. ## Limits and operational contract @@ -253,22 +253,22 @@ each `start.request` payload rather than in handshake headers. | Realtime: buffered audio awaiting commit | 100 MiB | error event, audio not buffered | | ElevenLabs `stream-input`: buffered text | 1 MiB | error frame `code: 1009`, then close | | ElevenLabs realtime STT: buffered audio | 100 MiB | `input_error`; append dropped, socket stays open | -| Single connection (SSE / WebSocket) | 3600 s | closed — reconnect before expiry | +| Single connection (SSE / WebSocket) | 3600 s | closed; reconnect before expiry | | Time to first response byte | 5 min | 502 (streaming bodies are not time-capped) | The 32 MiB frame limit bounds one WebSocket message, not a session. On a Realtime socket, item-borne audio and `input_audio_buffer` bytes share the same 100 MiB budget. -- **Billing** — like native usage, against your key. Billable requests are - never retried automatically; retrying is your call. -- **Rate limits** — the Compatible APIs add none of their own today; a 429 +- **Billing**: Like native usage, against your key. Billable requests are + never retried automatically. +- **Rate limits**: The Compatible APIs add none of their own today; a 429 means you reached your account's concurrency limit, the same one native calls count against. Every 429 carries a `Retry-After` header. -- **Truncation is an error** — a stream that fails mid-response returns an +- **Truncation is an error**: A stream that fails mid-response returns an error frame or 502, never a truncated 200. - Every synthesis or transcription response carries an `X-Generation-Id` - header — include it in support requests. + header; include it in support requests. - During service incidents, requests may return 503; back off exponentially. ## Known client-side limitations @@ -277,7 +277,7 @@ same 100 MiB budget. (`response.output_audio.*`). Current `openai` packages emit these; very old beta-era clients expecting `response.audio.*` will not match. - **ElevenLabs Python `convert_realtime`**: pass `model_id` and - `voice_settings` explicitly — left to the SDK's `OMIT` sentinel, `model_id` + `voice_settings` explicitly. Left to the SDK's `OMIT` sentinel, `model_id` reaches the handshake URL as the literal `model_id=Ellipsis` (a 400) and `voice_settings` raises `AttributeError` on the first frame. The SDK also force-upgrades `http://` base URLs to `wss://`. diff --git a/developer-guide/compat/migrate-from-elevenlabs.mdx b/developer-guide/compat/migrate-from-elevenlabs.mdx index b12c923..ba1c567 100644 --- a/developer-guide/compat/migrate-from-elevenlabs.mdx +++ b/developer-guide/compat/migrate-from-elevenlabs.mdx @@ -36,13 +36,13 @@ The two changes that matter when migrating: 1. **Voices are your Fish voices.** ElevenLabs preset names (`Rachel`, …) don't exist here. Pass the ID of a voice from your - [Fish voice library](/developer-guide/getting-started/migration#voices) — - `client.voices.get_all()` lists them ([Voice library](#voice-library)) — or + [Fish voice library](/developer-guide/getting-started/migration#voices), or `default` (also `-`) for the model's default voice. + `client.voices.get_all()` lists your voices ([Voice library](#voice-library)). 2. **Keep your `model_id`, or pick a Fish model.** Existing values like `eleven_multilingual_v2` are accepted as-is and served by Fish `s2.1-pro`, so your code runs unchanged. To choose a different model, pass its Fish - name, e.g. `fish-audio/s2-pro` — see + name, e.g. `fish-audio/s2-pro`; see [Models](/developer-guide/getting-started/migration#models). | Feature | Supported | @@ -95,14 +95,14 @@ all work. `voice_settings` maps onto Fish's synthesis controls: `speed` sets the speech rate, and `stability` becomes Fish `temperature = 1 − stability`. The -vendor-specific tuning knobs — `similarity_boost`, `style`, -`use_speaker_boost`, `language_code`, `seed` — are accepted and ignored, never +vendor-specific tuning knobs (`similarity_boost`, `style`, +`use_speaker_boost`, `language_code`, `seed`) are accepted and ignored, never an error. ### Output formats `mp3_*`, `pcm_*`, `opus_*`, and `wav_*` (a Fish Audio extension) work at the -supported sample rates — 32000 or 44100 for `mp3_*`, 48000 for `opus_*`. +supported sample rates: 32000 or 44100 for `mp3_*`, 48000 for `opus_*`. The 22.05 kHz family (`mp3_22050_32`, `pcm_22050`, …) is refused. The sample-rate segment of the format name is honored; the `mp3_*` bitrate segment snaps **up** to the nearest Fish tier @@ -117,16 +117,15 @@ keys in **seconds**; the WebSocket uses camelCase keys in **milliseconds** timings are interpolated from Fish segment timestamps at millisecond precision. - **`normalizedAlignment` is identical to `alignment`**, field for field. Fish - synthesizes the text as given, so there is no normalization pass to report; - the same alignment is sent under both keys. Pick either — don't diff them - expecting to recover normalization. + synthesizes the text as given, so there is no normalization pass to report. + Pick either; don't diff them expecting to recover normalization. - **On the WebSocket, the milliseconds are absolute on the session timeline** - (alignment frames need `?sync_alignment=true` — see + (alignment frames need `?sync_alignment=true`; see [Realtime input streaming](#realtime-input-streaming-websocket)), not relative to the current turn. Turn two of a `stream-input` session starts where turn one's audio ended (~1022 ms, say), not back at 0. That is what lets a client concatenate every frame's audio and alignment and have the captions - line up. Don't add your own per-turn offset — you would double-count it. + line up. Don't add your own per-turn offset; you would double-count it. ## Realtime input streaming (WebSocket) @@ -134,13 +133,13 @@ timings are interpolated from Fish segment timestamps at millisecond precision. The official protocol, unchanged: -1. Send the init message — a single-space `text`, plus optional +1. Send the init message: a single-space `text`, plus optional `voice_settings`. 2. Send text chunks; `try_trigger_generation` or `flush` forces synthesis now. 3. Send an empty-string `text` to finish. The server streams audio frames, then sends `{"isFinal": true}` and closes the -connection — so SDK iterators that read until close terminate correctly. +connection, so SDK iterators that read until close terminate correctly. Frame shapes, precisely, because they are not all the same: @@ -148,23 +147,23 @@ Frame shapes, precisely, because they are not all the same: `?sync_alignment=true` to the handshake URL and frames become `{"audio": …, "alignment": {…}, "normalizedAlignment": {…}}`. With alignment on, a turn's first audio arrives only after its first text chunk - has fully synthesized — expect a later first byte than the default mode. + has fully synthesized, so expect a later first byte than the default mode. Both alignment keys are omitted when there is nothing to report for that chunk, so treat them as optional rather than indexing into them blindly. - With `sync_alignment`, a turn can end with an **audio-less frame** carrying - only `alignment` and `normalizedAlignment` — trailing punctuation and + only `alignment` and `normalizedAlignment`: trailing punctuation and whitespace that no audio segment spoke. Code that assumes every frame has `audio` will throw on it. -- Errors arrive in band as `{"error": "…", "code": …}` on the open socket — - the code is an HTTP status, or a WebSocket close code for transport-level +- Errors arrive in band as `{"error": "…", "code": …}` on the open socket. + The code is an HTTP status, or a WebSocket close code for transport-level limits (an over-1 MiB text buffer closes the socket with `1009`). -- The stream ends with `{"isFinal": true}` — a frame with no `audio` — followed +- The stream ends with `{"isFinal": true}` (a frame with no `audio`), followed by a close. -An `output_format` that is invalid on its face — `ulaw_*`, or a name that -can't be parsed — fails with HTTP 400 before the upgrade. The 22.05 kHz family +An `output_format` that is invalid on its face (`ulaw_*`, or a name that +can't be parsed) fails with HTTP 400 before the upgrade. The 22.05 kHz family fails after it instead: the socket opens, then carries a `{"code": 400, …}` -frame and closes with a normal close code. Neither case is a silent no-op. +frame and closes with a normal close code. ```python from elevenlabs import VoiceSettings @@ -181,10 +180,9 @@ audio = b"".join(client.text_to_speech.convert_realtime( )) ``` -`text` is typed `Iterator[str]`. A plain string does run — the SDK re-chunks it -character by character — but a generator is what the signature asks for and it -shows what "streaming input" actually means: you yield text as it becomes -available, rather than having it all up front. +`text` is typed `Iterator[str]`. A plain string does run (the SDK re-chunks it +character by character), but a generator is what the signature asks for: you +yield text as it becomes available, rather than having it all up front. Pass both `model_id` and `voice_settings` explicitly with `convert_realtime`. @@ -205,7 +203,7 @@ available, rather than having it all up front. ```python Python result = client.speech_to_text.convert( file=open("meeting.wav", "rb"), - model_id="scribe_v1", # accepted and ignored — Fish's ASR model is used + model_id="scribe_v1", # accepted and ignored; Fish's ASR model is used language_code="zh", ) ``` @@ -216,7 +214,7 @@ import fs from "node:fs"; const buf = fs.readFileSync("meeting.wav"); const result = await client.speechToText.convert({ file: new File([buf], "meeting.wav"), - modelId: "scribe_v1", // accepted and ignored — Fish's ASR model is used + modelId: "scribe_v1", // accepted and ignored; Fish's ASR model is used }); console.log(result.text); ``` @@ -235,7 +233,7 @@ The response matches the official shape: `text`, plus `words` entries of entries come back whether or not you pass `timestamps_granularity="word"`; `timestamps_granularity="character"` is not supported and returns a 400. -Three fields are constants, not measurements — don't build quality gates or +Three fields are constants, not measurements; don't build quality gates or speaker logic on them: - `language_probability` is always `1.0`. @@ -243,14 +241,14 @@ speaker logic on them: - `words[].speaker_id` is always `null`; requesting diarization returns an explicit error instead. -And `language_code` in the response **echoes what you sent** — it is never a +And `language_code` in the response **echoes what you sent**, never a detection result. The value is still forwarded to Fish's ASR; only the reported field is an echo rather than a measurement. Fish needs no language hint, so sending none is normal; you then get `language_code: ""` alongside `language_probability: 1.0`, which reads as total confidence in an empty answer. Code branching on `language_probability > 0.9` will act on that empty string. -(On the *synthesis* side `language_code` is a different story — there it is +(On the *synthesis* side `language_code` is a different story: there it is accepted and dropped, along with the other vendor tuning knobs.) Options that would change the response contract are **rejected with 400** @@ -274,29 +272,26 @@ rather than silently ignored: **`tag_audio_events` is the one that bites on migration.** ElevenLabs defaults it to `true` and their examples pass it explicitly, so code copied from an - ElevenLabs project frequently sends it — and gets a 400 here. Drop the + ElevenLabs project frequently sends it, and gets a 400 here. Drop the parameter; the transcript itself is unaffected. -Booleans count as requested only when true — `False`, `"False"`, `0`, and `off` -are treated as unset, so default SDK serialization never trips any of these. A -refusal means you asked for the behavior, not that your SDK filled in a default. +Booleans count as requested only when true: `False`, `"False"`, `0`, and `off` +are treated as unset, so default SDK serialization never trips any of these. ### Realtime STT The realtime WebSocket (`/v1/speech-to-text/realtime`) supports `commit_strategy=manual` only: stream `input_audio_chunk` messages and set -`commit: true` to receive the committed transcript. The session uses explicit -commits — your application decides the segment boundaries — so -`commit_strategy=vad` is refused at the handshake instead of leaving a session -waiting. +`commit: true` to receive the committed transcript. Your application decides +the segment boundaries, so `commit_strategy=vad` is refused at the handshake +instead of leaving a session waiting. **Set `audio_format` to match your audio.** This socket carries bare PCM samples with no container, so the `audio_format` query parameter is the only thing that says how to interpret them. It defaults to **`pcm_16000`** (the - ElevenLabs SDK's own default) — and nothing in the audio can contradict - that. + ElevenLabs SDK's own default). Feed 44.1 kHz samples to a session that says 16000 and you get a **`200` with a fluent, confident, completely wrong transcript**, word timestamps and all: @@ -309,7 +304,7 @@ waiting. fail on every commit. -Audio goes in the `audio_base_64` field of an `input_audio_chunk` message — +Audio goes in the `audio_base_64` field of an `input_audio_chunk` message; other message shapes are ignored by design. Each commit answers with a `committed_transcript` message followed by `committed_transcript_with_timestamps`. @@ -319,10 +314,10 @@ other message shapes are ignored by design. Each commit answers with a Your Fish voice library and the model catalog are available in the ElevenLabs shapes: -- `GET /v1/voices`, `GET /v2/voices`, `GET /v1/voices/{id}` — your voices in +- `GET /v1/voices`, `GET /v2/voices`, `GET /v1/voices/{id}`: your voices in the ElevenLabs `Voice` shape (`category: "cloned"`, tags mapped to `labels`). -- `GET /v1/models` — the Fish model catalog in the ElevenLabs shape. -- `GET /v1/user` — a stub for SDK health checks. +- `GET /v1/models`: the Fish model catalog in the ElevenLabs shape. +- `GET /v1/user`: a stub for SDK health checks. @@ -351,23 +346,21 @@ Errors on this protocol use the ElevenLabs envelope: `provider_error`. `provider_error` is the one that tells you where the failure happened: the -request reached the Fish Audio API and its message is passed through verbatim — -an *invalid* key surfaces this way too, as a 401 `provider_error` carrying +request reached the Fish Audio API and its message is passed through verbatim. +An *invalid* key surfaces this way too, as a 401 `provider_error` carrying "No permission", while `unauthorized` is reserved for requests that sent no key at all. Every other status is raised by the compatibility layer itself. - + `voice_not_found` and `provider_error` are the easy pair to confuse. Looking - up a voice that isn't yours — `GET /v1/voices/{id}` — gives you + up a voice that isn't yours (`GET /v1/voices/{id}`) gives you `voice_not_found` with a 404. Naming that same voice in a *synthesis* request - instead fails as a 400 `provider_error` carrying "Reference not found". Same - mistake, two different statuses depending on which endpoint you made it on. + instead fails as a 400 `provider_error` carrying "Reference not found". So `{"status": "invalid_request", "message": "unsupported output_format \"ulaw_8000\""}` reports a parameter that is never accepted, while `{"status": "provider_error", "message": "Invalid sample rate 22050 for - format audio/mpeg…"}` comes from synthesis itself — a distinction worth - logging separately. + format audio/mpeg…"}` comes from synthesis itself. ## Going further diff --git a/developer-guide/compat/migrate-from-groq.mdx b/developer-guide/compat/migrate-from-groq.mdx index 9c03e28..ed74dc4 100644 --- a/developer-guide/compat/migrate-from-groq.mdx +++ b/developer-guide/compat/migrate-from-groq.mdx @@ -6,7 +6,7 @@ icon: "bolt" --- Configure the client once. The Groq SDK speaks the OpenAI protocol and appends -`/openai/v1` to its base URL itself — give it the bare `/compat` base: +`/openai/v1` to its base URL itself, so give it the bare `/compat` base: @@ -89,21 +89,21 @@ curl https://api.fish.audio/compat/openai/v1/audio/speech \ -Audio streams as it is synthesized. Set `response_format` explicitly — it +Audio streams as it is synthesized. Set `response_format` explicitly; it defaults to `pcm` on this endpoint, not `mp3`. `voice` is a Fish voice ID, not -a vendor preset — see +a vendor preset; see [Voices](/developer-guide/getting-started/migration#voices). -The accepted formats are `mp3`, `pcm`, `pcm16`, `wav`, and `opus` — the same +The accepted formats are `mp3`, `pcm`, `pcm16`, `wav`, and `opus`, the same set as the rest of `/openai/v1`; the `flac`, `mulaw`, and `ogg` the SDK types also offer are formats Fish cannot produce and return a 400. `sample_rate` (Hz) is accepted as an extension field on the request body -(untyped — `extra_body` in Python, an extra property in Node; defaults to +(untyped: `extra_body` in Python, an extra property in Node; defaults to 44100); asking for a rate the codec doesn't take is a 400, and the per-codec rate table is in [Audio formats](/developer-guide/compat/capabilities#audio-formats). - + The SDK's `response_format` types omit `pcm`, `pcm16`, and `opus`, and the same `Literal` narrows the Python SDK, so type checkers flag them there too. Cast at the call site: @@ -121,7 +121,7 @@ rate table is in The transcription side types `response_format` as - `json | text | verbose_json`, so `srt`/`vtt` need the same cast — and the + `json | text | verbose_json`, so `srt`/`vtt` need the same cast, and the SDK hands the subtitle document back as a plain string even though the method is typed to return `Transcription`. @@ -159,7 +159,7 @@ curl https://api.fish.audio/compat/openai/v1/audio/transcriptions \ `verbose_json`, word-level timestamp granularity, and the subtitle formats -(`srt`, `vtt`) work as they do on the OpenAI protocol — details and placeholder-field +(`srt`, `vtt`) work as they do on the OpenAI protocol; details and placeholder-field caveats in [Migrate from OpenAI](/developer-guide/compat/migrate-from-openai#transcription); `srt`/`vtt` need the SDK type cast shown above. diff --git a/developer-guide/compat/migrate-from-openai.mdx b/developer-guide/compat/migrate-from-openai.mdx index dfe999e..8b6cdad 100644 --- a/developer-guide/compat/migrate-from-openai.mdx +++ b/developer-guide/compat/migrate-from-openai.mdx @@ -34,7 +34,7 @@ const client = new OpenAI({ The same change works in any framework that takes an OpenAI-compatible base URL. Authentication is the standard `Authorization: Bearer` header; on WebSockets, `?api_key=` in the URL also works. The -Groq SDK speaks the same protocol — see +Groq SDK speaks the same protocol; see [Migrate from Groq](/developer-guide/compat/migrate-from-groq). | Feature | Supported | @@ -96,20 +96,20 @@ Differences from OpenAI: - **Set `response_format` explicitly.** It defaults to `pcm` here, not `mp3` as on OpenAI. Code that omits it and writes the bytes to `out.mp3` produces a file that won't play. -- **`voice` is a Fish voice ID** — see +- **`voice` is a Fish voice ID**; see [Voices](/developer-guide/getting-started/migration#voices). Preset names (`nova`, `echo`, …) are a 400, except `alloy`, which happens to synthesize in - an unrelated voice — see + an unrelated voice; see [capabilities](/developer-guide/compat/capabilities#what-each-protocol-supports). - **`instructions` is refused with a 400.** Use `X-Fish-*` headers or `provider.options` for delivery control instead. - **`stream_format` must be `audio`.** `stream_format: "sse"` is a first-class - parameter in current OpenAI SDKs and is refused with a 400 here — the + parameter in current OpenAI SDKs and is refused with a 400 here; the response is always the raw audio stream. `speed` works as on OpenAI. `sample_rate` (Hz) triggers a real resample, but each codec accepts only certain rates and asking for one it doesn't take is a -**400, not a silent fallback** — the per-codec rate table is in +**400, not a silent fallback**; the per-codec rate table is in [Audio formats](/developer-guide/compat/capabilities#audio-formats). One asymmetry between the two SDKs: `sample_rate` is not a first-class @@ -130,7 +130,7 @@ with client.audio.speech.with_streaming_response.create( Your existing model names carry over: `tts-1`, `tts-1-hd`, and `gpt-4o-mini-tts` are accepted as aliases for `fish-audio/s2.1-pro`. A name -that is neither a known alias nor a Fish model is refused with a 400 — never +that is neither a known alias nor a Fish model is refused with a 400, never silently substituted. ## Transcription @@ -172,17 +172,16 @@ curl https://api.fish.audio/compat/v1/audio/transcriptions \ `timestamp_granularities=["word"]` is what adds the top-level `words` array; without it `verbose_json` returns `segments[]` alone. (Fish times every word, so -`segments[]` is per-word either way — the flag controls the `words` array, not +`segments[]` is per-word either way; the flag controls the `words` array, not the precision.) Subtitle formats (`srt`, `vtt`) are aggregated from Fish's word-level timestamps into phrase-length cues, so they won't line up one-to-one with the per-word entries `verbose_json` returns in `segments[]`. -In `verbose_json`, the `language` field echoes the language **you** sent — it is -not a detection result, and `transcribe-1` needs no hint to work. Send none and -the field comes back as an empty string, which is the expected response, not a -failure. Don't route on it. (The value you send is still passed to Fish's ASR; +In `verbose_json`, the `language` field echoes the language **you** sent, not a +detection result; `transcribe-1` needs no hint to work. Send none and the field +comes back as an empty string, not a failure. Don't route on it. (The value you send is still passed to Fish's ASR; it's only the reported field that is an echo rather than a measurement.) Transcription model names carry over the same way: `whisper-1`, @@ -191,18 +190,18 @@ Transcription model names carry over the same way: `whisper-1`, In `verbose_json`, the Whisper-engine internals (`avg_logprob`, - `no_speech_prob`, `compression_ratio`, `temperature`) are neutral constants — + `no_speech_prob`, `compression_ratio`, `temperature`) are neutral constants; don't build quality filters on them. -The OpenAI SDK's `client.audio.translations` has no counterpart here — +The OpenAI SDK's `client.audio.translations` has no counterpart here: `POST /v1/audio/translations` is a 404. Transcribe in the source language and translate the text downstream. ## Chat-modality audio `POST /v1/chat/completions` serves TTS and STT in the chat-completions shape, -for SDKs and frameworks that only speak chat — LangChain, Vercel AI SDK, and +for SDKs and frameworks that only speak chat: LangChain, Vercel AI SDK, and similar. For **speech output**, request the audio modality; the last user message's text @@ -273,7 +272,7 @@ const client = new OpenAI({ apiKey: process.env.FISH_AUDIO_API_KEY, }); const rt = new OpenAIRealtimeWS({ model: "fish-audio/s2.1-pro" }, client); -// Stand-in for your audio pipeline — deltas are base64 chunks. +// Stand-in for your audio pipeline; deltas are base64 chunks. const playChunk = b64 => process.stdout.write(`audio: ${b64.length} b64 chars\n`); rt.on("session.created", () => { @@ -291,7 +290,7 @@ rt.on("response.output_audio.delta", ev => playChunk(ev.delta)); ``` The essentials: the endpoint speaks the Realtime **GA** event names, you drive -turn detection yourself (`input_audio_buffer.commit` ends an utterance — +turn detection yourself (`input_audio_buffer.commit` ends an utterance; server-side VAD is refused, not ignored), and transcription runs over the same socket or in a dedicated transcription session. The full event list, format and voice rules, and transcription-session handshakes are in the @@ -313,17 +312,17 @@ an integer `code`: } ``` -Errors from the Fish Audio API keep their original status — for example 402 -when your account is out of credit — and carry `type: "provider_error"` plus +Errors from the Fish Audio API keep their original status (for example 402 +when your account is out of credit) and carry `type: "provider_error"` plus `"metadata": {"provider_name": "fish-audio"}`. Errors from the compatibility layer itself carry no `metadata`. Rate limits return 429 with a `Retry-After` header. Authentication produces both kinds, and the `type` tells you which problem you have. No key, or a header that can't be parsed, is a 401 with -`type: "authentication_error"` and no `metadata` — nothing was checked against +`type: "authentication_error"` and no `metadata`: nothing was checked against your account. A key that fails validation is a 401 with -`type: "provider_error"` and `metadata.provider_name: "fish-audio"` — the key +`type: "provider_error"` and `metadata.provider_name: "fish-audio"`: the key itself is invalid. Both surface as your SDK's `AuthenticationError`. `GET /v1/models` is unauthenticated and answers even without a key, so it is @@ -344,6 +343,6 @@ not a way to check whether a key is valid. icon="sliders" href="/developer-guide/compat/capabilities#reaching-fish-native-parameters" > - `latency`, `temperature`, zero-shot cloning — through the OpenAI SDK. + `latency`, `temperature`, and zero-shot cloning through the OpenAI SDK. diff --git a/developer-guide/compat/migrate-from-openrouter.mdx b/developer-guide/compat/migrate-from-openrouter.mdx index 0c6ab37..b51825f 100644 --- a/developer-guide/compat/migrate-from-openrouter.mdx +++ b/developer-guide/compat/migrate-from-openrouter.mdx @@ -5,7 +5,7 @@ description: "Point the official OpenRouter TypeScript SDK at Fish Audio: TTS, S icon: "route" --- -Configure the client once — `serverURL` takes the `/api/v1` prefix: +Configure the client once. `serverURL` takes the `/api/v1` prefix: ```typescript import { OpenRouter } from "@openrouter/sdk"; @@ -16,7 +16,7 @@ const client = new OpenRouter({ }); ``` -From Python, OpenRouter's own SDK works the same way — `pip install openrouter`, +From Python, OpenRouter's own SDK works the same way: `pip install openrouter`, then pass `server_url`: ```python @@ -37,7 +37,7 @@ res = client.tts.create_speech( open("hello.mp3", "wb").write(res.read()) ``` -The OpenAI SDK works against the same base URL too — the Python tabs below +The OpenAI SDK works against the same base URL too; the Python tabs below use it. | Feature | Supported | @@ -104,18 +104,18 @@ curl https://api.fish.audio/compat/api/v1/audio/speech \ The response is a `ReadableStream`: audio bytes arrive as they are synthesized. The optional `voice` field takes a voice ID from your Fish voice library -(omitted or empty = model default — see +(omitted or empty = model default; see [Voices](/developer-guide/getting-started/migration#voices)). -`input_references` — OpenRouter's first-class field for stateless voice -cloning, reference audio plus its transcript — is supported: no saved voice +`input_references` (OpenRouter's first-class field for stateless voice +cloning, reference audio plus its transcript) is supported: no saved voice model required. Both SDKs carry the field on their `SpeechRequest` type (`inputReferences` in TypeScript, `input_references` in Python). ### `temperature`, `top_p`, `repetition_penalty` These three sit **at the top level of the speech request body**, next to -`model` and `input` — no `provider.options` wrapper required — matching what +`model` and `input` (no `provider.options` wrapper required), matching what the catalog advertises in each model's `supported_parameters`. ```json @@ -130,14 +130,14 @@ the catalog advertises in each model's `supported_parameters`. ``` Neither SDK's speech type declares them, so they still have to go around the -typed request — as raw JSON via `fetch`, or through the OpenAI SDK's +typed request: as raw JSON via `fetch`, or through the OpenAI SDK's `extra_body`. The same three are also reachable through `provider.options.fish-audio` and the `X-Fish-*` headers, and if you set a value two ways the more specific one wins: **`X-Fish-*` header beats `provider.options`, and `provider.options` beats the -top level.** Everything else in the Fish parameter set — `latency`, -`chunk_length`, `normalize`, `prosody`, references — is available only through +top level.** Everything else in the Fish parameter set (`latency`, +`chunk_length`, `normalize`, `prosody`, references) is available only through those two channels, not at the top level. See [Fish-native parameters](/developer-guide/compat/capabilities#reaching-fish-native-parameters). @@ -220,13 +220,13 @@ curl https://api.fish.audio/compat/api/v1/models Entries validate against the OpenRouter `Model` schema: **every required field is present, and the ones with no Fish counterpart are explicitly `null` rather -than missing** — SDK model validation passes and `model.context_length` reads +than missing**, so SDK model validation passes and `model.context_length` reads as `None`/`null` instead of raising. Four fields are always `null`: `context_length`, `default_parameters`, -`per_request_limits`, and `supported_voices`. Six optional ones — -`alias_target`, `benchmarks`, `expiration_date`, `hugging_face_id`, -`knowledge_cutoff`, and `reasoning` — are simply absent; don't write code that +`per_request_limits`, and `supported_voices`. Six optional ones +(`alias_target`, `benchmarks`, `expiration_date`, `hugging_face_id`, +`knowledge_cutoff`, and `reasoning`) are simply absent; don't write code that expects the key to exist. Filter with `output_modalities=speech` or `text` to list one direction only. @@ -236,7 +236,7 @@ Errors use the OpenRouter envelope, with the HTTP status as an integer in `error.code`. Failures from the Fish Audio API surface as `type: "provider_error"` with their original status and `metadata.provider_name: "fish-audio"`; errors from the compatibility layer -itself carry no `metadata` — just `code`, `message`, and `type`. +itself carry no `metadata`, just `code`, `message`, and `type`. ## Going further diff --git a/developer-guide/compat/realtime-protocol.mdx b/developer-guide/compat/realtime-protocol.mdx index 2216c98..dbe1746 100644 --- a/developer-guide/compat/realtime-protocol.mdx +++ b/developer-guide/compat/realtime-protocol.mdx @@ -11,21 +11,22 @@ wss://api.fish.audio/compat/v1/realtime?model=fish-audio/s2.1-pro Authentication is the standard `Authorization: Bearer` header, or `?api_key=` in the URL. The OpenAI SDK derives this URL from the same -client you configured for REST — a working example is in +client you configured for REST; a working example is in [Migrate from OpenAI](/developer-guide/compat/migrate-from-openai#realtime-websocket). ## Events The endpoint speaks the Realtime **GA** event names -(`response.output_audio.delta` and the rest of the GA set) — what current -`openai` packages emit, even though the Node module path still says `beta`. +(`response.output_audio.delta` and the rest of the GA set), which is what +current `openai` packages emit, even though the Node module path still says +`beta`. Supported client events: `session.update`, `transcription_session.update`, `conversation.item.create`, `conversation.item.delete`, `input_audio_buffer.append` / `commit` / `clear`, `response.create`, -`response.cancel`. `conversation.item.truncate` is accepted and does nothing — -it cuts a previous assistant message's audio, and none is retained server-side — so -it is the one client event that sends no acknowledgement back. Any other event +`response.cancel`. `conversation.item.truncate` is accepted and does nothing +(it cuts a previous assistant message's audio, and none is retained +server-side), so it is the one client event that sends no acknowledgement back. Any other event type comes back as an `unknown_event_type` error. One response at a time per session: a second `response.create` while one is @@ -33,14 +34,14 @@ running returns the `response_already_active` error. ## Audio formats -Output formats: `pcm16`, `wav`, `mp3`, `opus` — as a string (`"mp3"`) or the +Output formats: `pcm16`, `wav`, `mp3`, `opus`, as a string (`"mp3"`) or the GA object form (`{"type": "audio/mp3", "rate": 32000}`). `audio/pcmu` and `audio/pcma` are rejected explicitly. **Both spellings reset the rate when -they carry no `rate` of their own** — to that format's default (`audio/pcm` → +they carry no `rate` of their own**: to that format's default (`audio/pcm` → 24000, `wav` and `mp3` → 44100, `opus` → 48000), not to what the session was -using before — so set `rate` explicitly when you switch formats mid-session. -`session.updated` echoes the resulting output *and* input format — rate -included — so you can read back what the session settled on. Supported sample +using before, so set `rate` explicitly when you switch formats mid-session. +`session.updated` echoes the resulting output *and* input format, rate +included, so you can read back what the session settled on. Supported sample rates per codec are in [Audio formats](/developer-guide/compat/capabilities#audio-formats). @@ -49,14 +50,14 @@ rates per codec are in `voice` is a Fish voice ID, accepted both at the legacy top-level `session.voice` and at the GA `session.audio.output.voice` position; session events echo it at the GA position. `speed` follows the same two-position rule -and the REST endpoint's 0.25–4.0 range — out of range returns an +and the REST endpoint's 0.25–4.0 range; out of range returns an `invalid_speed` error event instead of synthesizing. ## Turn detection Turn detection is yours to drive: send `input_audio_buffer.commit` to end an -utterance. Any `turn_detection` with a type — `server_vad`, `semantic_vad`, or -anything else, in either the GA or the legacy top-level position — is refused +utterance. Any `turn_detection` with a type (`server_vad`, `semantic_vad`, or +anything else, in either the GA or the legacy top-level position) is refused with `unsupported_turn_detection` rather than accepted and ignored, and `session.created` reports `"turn_detection": null`. Likewise, an `output_modalities` that omits `"audio"` is refused: this socket always @@ -65,11 +66,11 @@ produces audio. ## Transcription Transcription works over the same socket: `input_audio_buffer.append` base64 -audio, then `input_audio_buffer.commit` — the transcript arrives as +audio, then `input_audio_buffer.commit`. The transcript arrives as `conversation.item.input_audio_transcription.completed`, and a failed transcription arrives as the matching `conversation.item.input_audio_transcription.failed` (the buffered audio is -kept, so you can retry). The socket returns the transcript text only — for +kept, so you can retry). The socket returns the transcript text only; for word timestamps use `POST /v1/audio/transcriptions`. Both official ways of opening a **dedicated** transcription session work: send @@ -82,8 +83,8 @@ TTS model. ## Limits -Session limits — text and audio caps per conversation item, item counts, frame -size — are in the +Session limits (text and audio caps per conversation item, item counts, frame +size) are in the [limits table](/developer-guide/compat/capabilities#limits-and-operational-contract). ## Related diff --git a/developer-guide/core-features/emotions.mdx b/developer-guide/core-features/emotions.mdx index f1b21ce..273f8b5 100644 --- a/developer-guide/core-features/emotions.mdx +++ b/developer-guide/core-features/emotions.mdx @@ -33,7 +33,7 @@ Fish Audio models support 64+ emotional expressions and voice styles that can be This page shows S2 usage with `[bracket]` cues. If you use the legacy S1 model, - wrap markers in parentheses instead — see [S1 (legacy) syntax](#s1-legacy-syntax) + wrap markers in parentheses instead; see [S1 (legacy) syntax](#s1-legacy-syntax) below for the full list, or the [Models Overview](/developer-guide/models-pricing/models-overview#s2-natural-language-control). @@ -62,7 +62,7 @@ The S2 TTS models will interpret these markers and adjust the voice accordingly. ## Sound & Delivery Markers -These markers aren't emotions — they shape _how_ a line is delivered, add natural human sounds, or layer in ambient effects. Combine them with the emotion cues above. +These markers aren't emotions. They shape _how_ a line is delivered, add natural human sounds, or layer in ambient effects. Combine them with the emotion cues above. ### Tone Markers (6 expressions) diff --git a/developer-guide/getting-started/api-key.mdx b/developer-guide/getting-started/api-key.mdx index 6c48152..b7e9477 100644 --- a/developer-guide/getting-started/api-key.mdx +++ b/developer-guide/getting-started/api-key.mdx @@ -4,7 +4,7 @@ description: "Create a Fish Audio account, generate an API key, and make your fi icon: "key" --- -Everything you build with Fish Audio — the API, the Python library, JavaScript — authenticates with a single **API key**. Here's how to get one and make your first call in a couple of minutes. +Everything you build with Fish Audio (the API, the Python library, JavaScript) authenticates with a single **API key**. Here's how to get one and make your first call in a couple of minutes. ## 1. Create an account and key @@ -16,7 +16,7 @@ Everything you build with Fish Audio — the API, the Python library, JavaScript Sign in and open [fish.audio/app/api-keys](https://fish.audio/app/api-keys). - Click **Create New Key**, give it a descriptive name (and an expiration if you want), then **copy the key and store it securely** — treat it like a password. + Click **Create New Key**, give it a descriptive name (and an expiration if you want), then **copy the key and store it securely**. Never commit your API key to version control or share it publicly. @@ -72,5 +72,5 @@ You just generated your first audio. Where to next: - **Building with an AI coding agent?** Install the Fish Audio skill so it writes correct SDK/API code — `npx skills add docs.fish.audio`. See [AI Coding Agents](/developer-guide/resources/coding-agents). + **Building with an AI coding agent?** Install the Fish Audio skill so it writes correct SDK/API code: `npx skills add docs.fish.audio`. See [AI Coding Agents](/developer-guide/resources/coding-agents). diff --git a/developer-guide/getting-started/migration.mdx b/developer-guide/getting-started/migration.mdx index 87dc8a9..1f5cdc7 100644 --- a/developer-guide/getting-started/migration.mdx +++ b/developer-guide/getting-started/migration.mdx @@ -40,7 +40,7 @@ tag: "Beta" with client.audio.speech.with_streaming_response.create( model="fish-audio/s2.1-pro", input="Hello from Fish Audio!", - voice="", # "" = default voice — see Voices below to pick one + voice="", # "" = default voice; see Voices below to pick one response_format="mp3", ) as response: response.stream_to_file("hello.mp3") @@ -58,7 +58,7 @@ tag: "Beta" const response = await client.audio.speech.create({ model: "fish-audio/s2.1-pro", input: "Hello from Fish Audio!", - voice: "", // "" = default voice — see Voices below to pick one + voice: "", // "" = default voice; see Voices below to pick one response_format: "mp3", }); fs.writeFileSync("hello.mp3", Buffer.from(await response.arrayBuffer())); @@ -78,9 +78,9 @@ tag: "Beta" - Full guide: [Migrate from OpenAI](/developer-guide/compat/migrate-from-openai) — - transcription, chat audio, the Realtime WebSocket, and what each feature - supports. + Full guide: [Migrate from OpenAI](/developer-guide/compat/migrate-from-openai), + covering transcription, chat audio, the Realtime WebSocket, and what each + feature supports. @@ -96,7 +96,7 @@ tag: "Beta" - From Python, use the OpenAI SDK against the same `/api/v1` base — that + From Python, use the OpenAI SDK against the same `/api/v1` base; that is how OpenRouter itself is used from Python. @@ -134,7 +134,7 @@ tag: "Beta" with client.audio.speech.with_streaming_response.create( model="fish-audio/s2.1-pro", input="Hello from Fish Audio!", - voice="", # "" = default voice — see Voices below to pick one + voice="", # "" = default voice; see Voices below to pick one response_format="mp3", ) as response: response.stream_to_file("hello.mp3") @@ -154,8 +154,8 @@ tag: "Beta" - Full guide: [Migrate from OpenRouter](/developer-guide/compat/migrate-from-openrouter) — - transcription, the model catalog, and what each feature supports. + Full guide: [Migrate from OpenRouter](/developer-guide/compat/migrate-from-openrouter), + covering transcription, the model catalog, and what each feature supports. @@ -185,7 +185,7 @@ tag: "Beta" ) audio = b"".join(client.text_to_speech.convert( - voice_id="default", # "default" = default voice — see Voices below + voice_id="default", # "default" = default voice; see Voices below text="Hello from Fish Audio!", model_id="fish-audio/s2.1-pro", output_format="mp3_44100_128", @@ -202,7 +202,7 @@ tag: "Beta" apiKey: process.env.FISH_AUDIO_API_KEY, }); - // "default" = default voice — see Voices below to pick one + // "default" = default voice; see Voices below to pick one const stream = await client.textToSpeech.convert("default", { text: "Hello from Fish Audio!", modelId: "fish-audio/s2.1-pro", @@ -226,9 +226,9 @@ tag: "Beta" - Full guide: [Migrate from ElevenLabs](/developer-guide/compat/migrate-from-elevenlabs) — - speech-to-text, timestamps, realtime streaming, your voice library, and - what each feature supports. + Full guide: [Migrate from ElevenLabs](/developer-guide/compat/migrate-from-elevenlabs), + covering speech-to-text, timestamps, realtime streaming, your voice + library, and what each feature supports. @@ -244,7 +244,7 @@ tag: "Beta" - The Groq SDK appends `/openai/v1` itself — give it the bare `/compat` + The Groq SDK appends `/openai/v1` itself, so give it the bare `/compat` base: @@ -261,7 +261,7 @@ tag: "Beta" resp = client.audio.speech.create( model="fish-audio/s2.1-pro", input="Hello from Fish Audio!", - voice="", # "" = default voice — see Voices below to pick one + voice="", # "" = default voice; see Voices below to pick one response_format="mp3", ) open("hello.mp3", "wb").write(resp.read()) @@ -279,7 +279,7 @@ tag: "Beta" const resp = await client.audio.speech.create({ model: "fish-audio/s2.1-pro", input: "Hello from Fish Audio!", - voice: "", // "" = default voice — see Voices below to pick one + voice: "", // "" = default voice; see Voices below to pick one response_format: "mp3", }); fs.writeFileSync("hello.mp3", Buffer.from(await resp.arrayBuffer())); @@ -299,15 +299,15 @@ tag: "Beta" - Full guide: [Migrate from Groq](/developer-guide/compat/migrate-from-groq) — - transcription, the model catalog, and what each feature supports. + Full guide: [Migrate from Groq](/developer-guide/compat/migrate-from-groq), + covering transcription, the model catalog, and what each feature supports. ## From a framework -Frameworks that speak the OpenAI protocol work the same way — point them at +Frameworks that speak the OpenAI protocol work the same way: point them at `/compat/v1`: @@ -358,25 +358,25 @@ Building a voice agent with Pipecat or LiveKit? Fish Audio has | Model | What it is | | -------------------------- | --------------------------------------------------------------- | -| `fish-audio/s2.1-pro` | Production multilingual TTS with voice cloning — recommended | +| `fish-audio/s2.1-pro` | Production multilingual TTS with voice cloning (recommended) | | `fish-audio/s2.1-pro-free` | Free tier of S2.1 Pro, for prototyping (no latency guarantee) | | `fish-audio/s2-pro` | Expressive TTS for narration and multi-speaker content | | `fish-audio/s1` | Multilingual TTS with emotion control | | `fish-audio/transcribe-1` | Multilingual speech-to-text with word-level timestamps | `transcribe-1` needs no language hint. The `language` field in a transcription -response echoes what you sent — empty when you sent none, never a detection +response echoes what you sent: empty when you sent none, never a detection result (the [ElevenLabs protocol](/developer-guide/compat/migrate-from-elevenlabs#speech-to-text) echoes `language_code` the same way). ## Voices The examples above use the model's default voice. To pick a specific one, pass -a **voice ID** — browse the [Voice Library](/overview/platform) and copy the id +a **voice ID**: browse the [Voice Library](/overview/platform) and copy the id of any voice, or make your own with [Voice Cloning](/features/voice-cloning) or [Voice Design](/features/voice-design). Vendor preset names (`nova`, `echo`, -`Rachel`, …) don't exist here — replace them with a voice ID. +`Rachel`, …) don't exist here. ```python voice="9a9cf47702da476aa4629e2506d4a857" # OpenAI / OpenRouter / Groq diff --git a/developer-guide/integrations/n8n.mdx b/developer-guide/integrations/n8n.mdx index 73f31c7..ae9855c 100644 --- a/developer-guide/integrations/n8n.mdx +++ b/developer-guide/integrations/n8n.mdx @@ -34,10 +34,10 @@ See the [n8n community nodes guide](https://docs.n8n.io/integrations/community-n The node supports: -- **Text-to-Speech** — Generate audio from text using any voice model -- **Speech-to-Text** — Transcribe audio files -- **Voice Models** — List, create, and manage custom voices -- **Account** — Check credit balance +- **Text-to-Speech**: Generate audio from text using any voice model +- **Speech-to-Text**: Transcribe audio files +- **Voice Models**: List, create, and manage custom voices +- **Account**: Check credit balance The node is also available as an AI tool for use with n8n's AI Agent nodes. diff --git a/developer-guide/integrations/telnyx.mdx b/developer-guide/integrations/telnyx.mdx index 883e9cf..f9dce42 100644 --- a/developer-guide/integrations/telnyx.mdx +++ b/developer-guide/integrations/telnyx.mdx @@ -4,7 +4,7 @@ description: "Use Fish Audio voices for real-time and in-call text-to-speech wit icon: "/images/telnyx-logo.svg" --- -[Telnyx](https://telnyx.com) is a carrier-owned global communications platform providing infrastructure for real-time agents — voice AI, SIP trunking, programmable voice, and messaging. Fish Audio is available on Telnyx as a hosted text-to-speech provider: you synthesize Fish Audio voices directly through the Telnyx API and play them in live phone calls via Call Control and TeXML, with no Fish Audio API key required. +[Telnyx](https://telnyx.com) is a carrier-owned global communications platform providing infrastructure for real-time agents: voice AI, SIP trunking, programmable voice, and messaging. Fish Audio is available on Telnyx as a hosted text-to-speech provider: you synthesize Fish Audio voices directly through the Telnyx API and play them in live phone calls via Call Control and TeXML, with no Fish Audio API key required. ## Prerequisites @@ -19,7 +19,7 @@ Telnyx exposes a curated shortlist of voices from the [Fish Audio Voice Library] FishAudio.s2.1-pro.933563129e564b19a115bedd57b7406a ``` -All curated voices are cross-lingual — any voice can speak any language present in the input text. On S2 models, you can also control delivery with inline emotion tags like `[happy]` or `[whispering]` — see [emotion control](/developer-guide/best-practices/emotion-control) for the full syntax. +All curated voices are cross-lingual: any voice can speak any language present in the input text. On S2 models, you can also control delivery with inline emotion tags like `[happy]` or `[whispering]`; see [emotion control](/developer-guide/best-practices/emotion-control) for the full syntax. See the [Telnyx Fish Audio provider page](https://developers.telnyx.com/docs/voice/tts/providers/fishaudio) for the current voice roster, supported audio formats, and sample rates. diff --git a/developer-guide/models-pricing/models-overview.mdx b/developer-guide/models-pricing/models-overview.mdx index 4ae4382..0ade880 100644 --- a/developer-guide/models-pricing/models-overview.mdx +++ b/developer-guide/models-pricing/models-overview.mdx @@ -22,7 +22,7 @@ Fish Audio offers state-of-the-art text-to-speech models optimized for different **Fish Audio S2.1-Pro** - Our recommended production TTS model and an improved version of S2-Pro - - Natural language control with `[bracket]` syntax — not limited to a fixed set (e.g., `[whispers sweetly]`, `[laughing nervously]`) + - Natural language control with `[bracket]` syntax, not limited to a fixed set (e.g., `[whispers sweetly]`, `[laughing nervously]`) - Multi-speaker dialogue support - 83 languages - Improved quality, latency, and throughput over S2-Pro @@ -44,7 +44,7 @@ Fish Audio offers state-of-the-art text-to-speech models optimized for different **Fish Audio S2-Pro** - Previous-generation S2 TTS model - - Natural language control with `[bracket]` syntax — not limited to a fixed set (e.g., `[whispers sweetly]`, `[laughing nervously]`) + - Natural language control with `[bracket]` syntax, not limited to a fixed set (e.g., `[whispers sweetly]`, `[laughing nervously]`) - Multi-speaker dialogue support - 80+ languages - 100ms time-to-first-audio @@ -100,7 +100,7 @@ Fish Audio models support emotional expressions and voice styles that can be con ### S2.1-Pro and S2-Pro Natural Language Control -S2.1-Pro and S2-Pro treat `[bracket]` tags as standard text rather than dedicated control tokens. Through training on massive datasets, the models learned implicit mappings between natural language descriptions and acoustic variations. This means you are not limited to a predefined set of tags — you can use any descriptive expression and the model will interpret it, such as `[whispers sweetly]` or `[laughing nervously]`. +S2.1-Pro and S2-Pro treat `[bracket]` tags as standard text rather than dedicated control tokens. Through training on massive datasets, the models learned implicit mappings between natural language descriptions and acoustic variations. You can use any descriptive expression and the model will interpret it, such as `[whispers sweetly]` or `[laughing nervously]`. Common examples include: diff --git a/developer-guide/resources/agent-quickstart.mdx b/developer-guide/resources/agent-quickstart.mdx index cf270de..3936242 100644 --- a/developer-guide/resources/agent-quickstart.mdx +++ b/developer-guide/resources/agent-quickstart.mdx @@ -1,11 +1,11 @@ --- title: "Coding Assistant Quickstart" sidebarTitle: "Coding Assistants" -description: "Build with Fish Audio using your AI coding assistant — install the skill and start prompting in a minute" +description: "Build with Fish Audio using your AI coding assistant: install the skill and start prompting in a minute" icon: "laptop-code" --- -Install the Fish Audio **agent skill** and your coding agent — Claude Code, Cursor, Codex, and others — writes correct, current Fish Audio code: right method names, units, and error types, instead of guessing. Here's the fastest path. +Install the Fish Audio **agent skill** and your coding agent (Claude Code, Cursor, Codex, and others) writes correct, current Fish Audio code: right method names, units, and error types, instead of guessing. Here's the fastest path. @@ -17,17 +17,17 @@ Install the Fish Audio **agent skill** and your coding agent — Claude Code, Cu - Python (`fish-audio-sdk`) and JavaScript (`fish-audio`) — exact method signatures, sync + async, model selection, and the real exception types. + Python (`fish-audio-sdk`) and JavaScript (`fish-audio`): exact method signatures, sync + async, model selection, and the real exception types. - Raw REST + WebSocket for any language — auth, endpoints, MessagePack/JSON/multipart rules, and the streaming protocol. + Raw REST + WebSocket for any language: auth, endpoints, MessagePack/JSON/multipart rules, and the streaming protocol. - [Create a key](/developer-guide/getting-started/api-key) and export it — the code your agent writes reads it from the environment: + [Create a key](/developer-guide/getting-started/api-key) and export it; the code your agent writes reads it from the environment: ```bash export FISH_API_KEY="your_api_key_here" @@ -36,7 +36,7 @@ Install the Fish Audio **agent skill** and your coding agent — Claude Code, Cu - Prompt in plain language — it uses the correct client, methods, and error types: + Prompt in plain language; it uses the correct client, methods, and error types: @@ -79,7 +79,7 @@ npx skills add https://docs.fish.audio --list diff --git a/developer-guide/resources/coding-agents.mdx b/developer-guide/resources/coding-agents.mdx index 4d2bd11..f453002 100644 --- a/developer-guide/resources/coding-agents.mdx +++ b/developer-guide/resources/coding-agents.mdx @@ -10,7 +10,7 @@ import { AudioTranscript } from "/snippets/audio-transcript.jsx"; -Install the Fish Audio **agent skill**, and your coding agent — Claude Code, Cursor, Codex, and others — writes correct, current Fish Audio code: right method names, units, and error types, instead of guessing. +Install the Fish Audio **agent skill**, and your coding agent (Claude Code, Cursor, Codex, and others) writes correct, current Fish Audio code: right method names, units, and error types, instead of guessing. ## Install the skill @@ -22,10 +22,10 @@ This installs both Fish Audio skills into your agent (a canonical copy in `.agen - Python (`fish-audio-sdk`) and JavaScript (`fish-audio`) — exact method signatures and defaults, sync + async, model selection, and the real exception types. + Python (`fish-audio-sdk`) and JavaScript (`fish-audio`): exact method signatures and defaults, sync + async, model selection, and the real exception types. - Raw REST + WebSocket for any language or edge runtime — auth, endpoints, MessagePack/JSON/multipart rules, and the streaming protocol. + Raw REST + WebSocket for any language or edge runtime: auth, endpoints, MessagePack/JSON/multipart rules, and the streaming protocol. @@ -54,7 +54,7 @@ Want to read them before installing? The skills are served at [/.well-known/agen ## Try it -Once installed, ask your agent in plain language — it will use the correct client, methods, and error types: +Once installed, ask your agent in plain language; it will use the correct client, methods, and error types: diff --git a/developer-guide/sdk-guide/cookbook/batch-transcribe-with-language-hint.mdx b/developer-guide/sdk-guide/cookbook/batch-transcribe-with-language-hint.mdx index 78d9ab6..5e6ac38 100644 --- a/developer-guide/sdk-guide/cookbook/batch-transcribe-with-language-hint.mdx +++ b/developer-guide/sdk-guide/cookbook/batch-transcribe-with-language-hint.mdx @@ -96,9 +96,8 @@ for (const row of results) { Each call returns an [`ASRResponse`](/api-reference/sdk/python/types#asrresponse-objects) with `.text`, a `.duration` in seconds, and per-phrase `.segments`. The loop keeps files independent, so one bad file does not block the rest of the batch. - Auto-detection (omit `language`) works well, but passing an explicit - `language` improves accuracy for similar-sounding languages. Use one - `language` per batch — split mixed-language files into separate lists. + Use one `language` per batch; split mixed-language files into separate + lists. ## Related diff --git a/developer-guide/sdk-guide/cookbook/clone-and-wait-until-ready.mdx b/developer-guide/sdk-guide/cookbook/clone-and-wait-until-ready.mdx index 756fcea..4dadbeb 100644 --- a/developer-guide/sdk-guide/cookbook/clone-and-wait-until-ready.mdx +++ b/developer-guide/sdk-guide/cookbook/clone-and-wait-until-ready.mdx @@ -125,7 +125,7 @@ await writeFile("out.mp3", Buffer.concat(chunks)); A voice moves through `created` → `training` → `trained`, or ends in `failed`. Always handle `failed` and the timeout so a stuck voice cannot loop forever. - Training a persistent voice takes time, so only create one when you will reuse the voice across many requests. For one-off synthesis, skip the wait entirely and pass a `ReferenceAudio` inline — see [Instant voice cloning](/developer-guide/sdk-guide/cookbook/instant-voice-cloning). + Training a persistent voice takes time, so only create one when you will reuse the voice across many requests. For one-off synthesis, skip the wait entirely and pass a `ReferenceAudio` inline. See [Instant voice cloning](/developer-guide/sdk-guide/cookbook/instant-voice-cloning). ## Related diff --git a/developer-guide/sdk-guide/cookbook/discover-library-voice.mdx b/developer-guide/sdk-guide/cookbook/discover-library-voice.mdx index 4982003..a939cc6 100644 --- a/developer-guide/sdk-guide/cookbook/discover-library-voice.mdx +++ b/developer-guide/sdk-guide/cookbook/discover-library-voice.mdx @@ -12,7 +12,7 @@ import Prerequisites from "/snippets/prerequisites.mdx"; ## Recipe -Set `self_only=False` on [`voices.list()`](/api-reference/sdk/python/resources#list) to search the public [Voice Library](/features/manage-voices) instead of only your own models. The response carries `total` (matches across all pages) and `items` (this page). Pick a result's `id` and pass it straight to [`tts.convert()`](/api-reference/sdk/python/resources#convert) as `reference_id` — no cloning, no model to manage. +Set `self_only=False` on [`voices.list()`](/api-reference/sdk/python/resources#list) to search the public [Voice Library](/features/manage-voices) instead of only your own models. The response carries `total` (matches across all pages) and `items` (this page). Pick a result's `id` and pass it straight to [`tts.convert()`](/api-reference/sdk/python/resources#convert) as `reference_id`. ```python Python @@ -72,7 +72,7 @@ await writeFile("out.mp3", Buffer.concat(chunks)); ``` -`page.total` is the full match count, so `total > len(page.items)` tells you there are more pages — bump `page_number` to walk them. Any public voice `id` is a ready-to-use `reference_id`; nothing is saved to your account. +`total > len(page.items)` tells you there are more pages; bump `page_number` to walk them. Any public voice `id` works as `reference_id`; nothing is saved to your account. You can hit the same endpoint directly: diff --git a/developer-guide/sdk-guide/cookbook/instant-voice-cloning.mdx b/developer-guide/sdk-guide/cookbook/instant-voice-cloning.mdx index ba681b1..23eed8e 100644 --- a/developer-guide/sdk-guide/cookbook/instant-voice-cloning.mdx +++ b/developer-guide/sdk-guide/cookbook/instant-voice-cloning.mdx @@ -12,7 +12,7 @@ import Prerequisites from "/snippets/prerequisites.mdx"; ## Recipe -Pass a [`ReferenceAudio`](/api-reference/sdk/python/types#referenceaudio-objects) (raw audio bytes + an exact transcript) on the `convert` call. Nothing is saved server-side — the clone applies to that request only. +Pass a [`ReferenceAudio`](/api-reference/sdk/python/types#referenceaudio-objects) (raw audio bytes + an exact transcript) on the `convert` call. Nothing is saved server-side; the clone applies to that request only. ```python Synchronous @@ -94,7 +94,7 @@ await writeFile("cloned.mp3", Buffer.concat(chunks)); ## Reuse a voice across many requests -If you'll use the voice repeatedly, create a persistent model once and pass its id as `reference_id` — see the [Voice Cloning guide](/features/voice-cloning). +If you'll use the voice repeatedly, create a persistent model once and pass its id as `reference_id`. See the [Voice Cloning guide](/features/voice-cloning). ```python with open("sample.wav", "rb") as f: diff --git a/developer-guide/sdk-guide/cookbook/oneshot-vs-persistent-cloning.mdx b/developer-guide/sdk-guide/cookbook/oneshot-vs-persistent-cloning.mdx index 2b9b386..685ec87 100644 --- a/developer-guide/sdk-guide/cookbook/oneshot-vs-persistent-cloning.mdx +++ b/developer-guide/sdk-guide/cookbook/oneshot-vs-persistent-cloning.mdx @@ -14,10 +14,10 @@ import Prerequisites from "/snippets/prerequisites.mdx"; There are two ways to clone a voice. Pick by how often you'll reuse it: -- **One-shot (instant)** — pass a [`ReferenceAudio`](/api-reference/sdk/python/types#referenceaudio-objects) (raw bytes + exact transcript) on each `convert` call. Nothing is stored server-side; the clone lives only for that request. -- **Persistent** — call `voices.create` once to train a model, then reuse its id as `reference_id` on every request. No reference upload per call, and the same voice is shared across processes. +- **One-shot (instant)**: pass a [`ReferenceAudio`](/api-reference/sdk/python/types#referenceaudio-objects) (raw bytes + exact transcript) on each `convert` call. Nothing is stored server-side; the clone lives only for that request. +- **Persistent**: call `voices.create` once to train a model, then reuse its id as `reference_id` on every request. No reference upload per call, and the same voice is shared across processes. -Start with one-shot. Below, a single reference clip is cloned inline with no model to manage: +Start with one-shot. Below, a single reference clip is cloned inline: ```python Synchronous @@ -104,7 +104,7 @@ Call [`voices.create`](/api-reference/sdk/python/resources#create) to train a mo with open("reference.wav", "rb") as f: voice = client.voices.create(title="My Narrator", voices=[f.read()]) -# reuse the same id on every later request — no reference upload +# reuse the same id on every later request; no reference upload audio = client.tts.convert( text="Reusing my saved voice across many requests.", reference_id=voice.id, diff --git a/developer-guide/sdk-guide/cookbook/realtime-llm-to-speech.mdx b/developer-guide/sdk-guide/cookbook/realtime-llm-to-speech.mdx index a877203..5f10f42 100644 --- a/developer-guide/sdk-guide/cookbook/realtime-llm-to-speech.mdx +++ b/developer-guide/sdk-guide/cookbook/realtime-llm-to-speech.mdx @@ -51,7 +51,7 @@ asyncio.run(main()) ## Force generation at a boundary -By default the engine buffers text until it has enough for natural prosody. Yield a [`FlushEvent`](/api-reference/sdk/python/types#flushevent-objects) to force synthesis of what's buffered — useful for turn-taking in a conversation: +By default the engine buffers text until it has enough for natural prosody. Yield a [`FlushEvent`](/api-reference/sdk/python/types#flushevent-objects) to force synthesis of what's buffered, useful for turn-taking in a conversation: ```python from fishaudio.types import TextEvent, FlushEvent @@ -62,7 +62,7 @@ def turns(): yield TextEvent(text="Let's begin.") ``` -The SDK sends the start/stop frames for you — you only supply text and optional flushes. +The SDK sends the start/stop frames for you. Errors mid-stream surface as `WebSocketError`. Reconnect with a fresh call diff --git a/developer-guide/sdk-guide/cookbook/streaming-to-file.mdx b/developer-guide/sdk-guide/cookbook/streaming-to-file.mdx index 8b24e71..49a964b 100644 --- a/developer-guide/sdk-guide/cookbook/streaming-to-file.mdx +++ b/developer-guide/sdk-guide/cookbook/streaming-to-file.mdx @@ -69,7 +69,7 @@ audio = client.tts.stream(text="Hello!").collect() # -> bytes ``` - `convert()` already returns the complete audio as `bytes` — reach for + `convert()` already returns the complete audio as `bytes`. Reach for `stream()` when you want to start writing/forwarding bytes before generation finishes, or to avoid buffering large files. diff --git a/developer-guide/sdk-guide/cookbook/telephony-8khz-audio.mdx b/developer-guide/sdk-guide/cookbook/telephony-8khz-audio.mdx index 3d80f63..a5354a9 100644 --- a/developer-guide/sdk-guide/cookbook/telephony-8khz-audio.mdx +++ b/developer-guide/sdk-guide/cookbook/telephony-8khz-audio.mdx @@ -12,7 +12,7 @@ import Prerequisites from "/snippets/prerequisites.mdx"; ## Recipe -Phone networks carry narrowband audio at 8 kHz. Generating at a higher rate just forces the carrier to downsample on the way through — wasting bandwidth and often softening the result. Synthesize at 8 kHz directly and the bytes are ready to hand to your IVR or SIP stack. +Phone networks carry narrowband audio at 8 kHz. Generating at a higher rate just forces the carrier to downsample on the way through, wasting bandwidth and often softening the result. Synthesize at 8 kHz directly and the bytes are ready to hand to your IVR or SIP stack. Set the sample rate on [`TTSConfig`](/api-reference/sdk/python/types#ttsconfig-objects) (it is not a top-level argument) and write the WAV to disk. @@ -71,7 +71,7 @@ await writeFile("out.wav", Buffer.concat(chunks)); ``` -The output is a mono 8 kHz WAV — the standard for G.711 PCM telephony. For a headerless stream to feed straight into a SIP or RTP pipeline, switch to raw PCM with `format="pcm"`; the sample rate stays on `TTSConfig`. +The output is a mono 8 kHz WAV, the standard for G.711 PCM telephony. For a headerless stream to feed straight into a SIP or RTP pipeline, switch to raw PCM with `format="pcm"`; the sample rate stays on `TTSConfig`. ```python audio = client.tts.convert( diff --git a/developer-guide/sdk-guide/cookbook/transcribe-to-captions.mdx b/developer-guide/sdk-guide/cookbook/transcribe-to-captions.mdx index a7fc18a..d137d2f 100644 --- a/developer-guide/sdk-guide/cookbook/transcribe-to-captions.mdx +++ b/developer-guide/sdk-guide/cookbook/transcribe-to-captions.mdx @@ -12,7 +12,7 @@ import Prerequisites from "/snippets/prerequisites.mdx"; ## Recipe -Call [`asr.transcribe()`](/api-reference/sdk/python/resources#transcribe) with `include_timestamps=True`, then turn each [`ASRSegment`](/api-reference/sdk/python/types#asrsegment-objects) into a numbered cue. Segment `start` / `end` are in **seconds**, so the only real work is formatting them — SRT wants `HH:MM:SS,mmm` (comma), WebVTT wants `HH:MM:SS.mmm` (dot). +Call [`asr.transcribe()`](/api-reference/sdk/python/resources#transcribe) with `include_timestamps=True`, then turn each [`ASRSegment`](/api-reference/sdk/python/types#asrsegment-objects) into a numbered cue. Segment `start` / `end` are in **seconds**, so the only real work is formatting them: SRT wants `HH:MM:SS,mmm` (comma), WebVTT wants `HH:MM:SS.mmm` (dot). ```python Python @@ -88,10 +88,10 @@ console.log(`Wrote ${result.segments.length} cues to captions.srt`); ``` -Both files share one timestamp helper — WebVTT is just the SRT formatting with `,` swapped for `.`, so there is no second formatter to keep in sync. +Both files share one timestamp helper; WebVTT just swaps `,` for `.`. - Pass `language=` (for example `"en"` or `"zh"`) when you know it — explicit + Pass `language=` (for example `"en"` or `"zh"`) when you know it. Explicit language selection sharpens segment boundaries, which keeps your cue timing tight. diff --git a/developer-guide/sdk-guide/cookbook/voice-agent-loop.mdx b/developer-guide/sdk-guide/cookbook/voice-agent-loop.mdx index f87fdcf..5de90a2 100644 --- a/developer-guide/sdk-guide/cookbook/voice-agent-loop.mdx +++ b/developer-guide/sdk-guide/cookbook/voice-agent-loop.mdx @@ -108,13 +108,13 @@ console.log("Agent:", reply); For the lowest latency, feed your LLM's token stream straight into [`stream_websocket()`](/api-reference/sdk/python/resources#stream_websocket) - instead of waiting for the full reply string — see + instead of waiting for the full reply string. See [Realtime: LLM tokens → speech](/developer-guide/sdk-guide/cookbook/realtime-llm-to-speech). ## Reply in the caller's voice -`reference_id` points the reply at a saved voice. Drop it to use the default voice, or clone the caller's voice from the same clip you just transcribed by passing `references` instead — see [Instant voice cloning](/developer-guide/sdk-guide/cookbook/instant-voice-cloning). +`reference_id` points the reply at a saved voice. Drop it to use the default voice, or clone the caller's voice from the same clip you just transcribed by passing `references` instead. See [Instant voice cloning](/developer-guide/sdk-guide/cookbook/instant-voice-cloning). ## Related diff --git a/developer-guide/sdk-guide/python/errors.mdx b/developer-guide/sdk-guide/python/errors.mdx index da52877..cece69a 100644 --- a/developer-guide/sdk-guide/python/errors.mdx +++ b/developer-guide/sdk-guide/python/errors.mdx @@ -26,9 +26,9 @@ Every SDK error inherits from [`FishAudioError`](/api-reference/sdk/python/excep | `DependencyError` | a required system tool is missing (e.g. ffmpeg for `play()`) | Carries `.dependency` and `.install_command` | - There is no separate `ValidationError` raised at runtime. Invalid request - parameters come back as an `APIError` with `status == 422` — catch `APIError`, - not `ValidationError`. + Invalid request parameters come back as an `APIError` with `status == 422`. + Catch `APIError`, not a separate `ValidationError` (none is raised at + runtime). ## Handling errors @@ -51,7 +51,7 @@ try: except AuthenticationError: print("Invalid API key") except RateLimitError: - print("Rate limited — back off and retry") + print("Rate limited: back off and retry") except NotFoundError: print("That voice model does not exist") except APIError as e: @@ -70,7 +70,7 @@ async def main(): try: audio = await client.tts.convert(text="Hello!") except RateLimitError: - print("Rate limited — back off and retry") + print("Rate limited: back off and retry") except APIError as e: print(f"API error {e.status}: {e.message}") except FishAudioError as e: @@ -82,7 +82,7 @@ asyncio.run(main()) ## Retries -The Python client does **not** retry automatically — each call makes a single request and raises on failure. Add your own backoff where it matters, typically around `RateLimitError` and `ServerError`: +The Python client does **not** retry automatically. Add your own backoff where it matters, typically around `RateLimitError` and `ServerError`: ```python import time @@ -104,7 +104,7 @@ def convert_with_retry(text: str, max_retries: int = 3) -> bytes: `RequestOptions` accepts a `max_retries` field, but the current client does - not act on it — use an explicit loop like the one above. + not act on it. Use an explicit loop like the one above. ## Timeouts @@ -129,12 +129,12 @@ audio = client.tts.convert( ``` - If you inject your own `httpx_client`, the SDK uses it as-is — the client-level - `timeout`, `base_url`, and the `Authorization` header are **not** applied to - it. Configure those on the client you pass in. + If you inject your own `httpx_client`, the client-level `timeout`, + `base_url`, and `Authorization` header are **not** applied to it. Configure + those on the client you pass in. ## Related - [Exceptions API reference](/api-reference/sdk/python/exceptions) -- [Real-time WebSocket](/features/realtime-streaming) — `WebSocketError` handling +- [Real-time WebSocket](/features/realtime-streaming): `WebSocketError` handling diff --git a/developer-guide/sdk-guide/quickstart.mdx b/developer-guide/sdk-guide/quickstart.mdx index 87688d3..294b37b 100644 --- a/developer-guide/sdk-guide/quickstart.mdx +++ b/developer-guide/sdk-guide/quickstart.mdx @@ -10,7 +10,7 @@ The fastest path from zero to playable audio with the official Fish Audio SDKs. The Python SDK is the recommended starting point and is fully covered below. - The JavaScript SDK is in early release — see the [JavaScript SDK + The JavaScript SDK is in early release. See the [JavaScript SDK guide](/api-reference/sdk/javascript/api-reference) for its current surface. @@ -62,7 +62,7 @@ await play(audio); ``` -Run it, and you'll have `output.mp3` (Python) or local playback (JavaScript). That's it — you're generating speech. +Run it, and you'll have `output.mp3` (Python) or local playback (JavaScript). Want async in Python? Every method mirrors onto `AsyncFishAudio`: `async with diff --git a/developer-guide/self-hosting/air-gapped.mdx b/developer-guide/self-hosting/air-gapped.mdx index 136cb06..6ef5ee5 100644 --- a/developer-guide/self-hosting/air-gapped.mdx +++ b/developer-guide/self-hosting/air-gapped.mdx @@ -11,7 +11,7 @@ deployment is about getting those artifacts across the boundary. Fish Audio does this with you as part of an air-gapped delivery, and the step-by-step - procedure is in the deployment runbook for the version you install — see + procedure is in the deployment runbook for the version you install. See [Releases](/developer-guide/self-hosting/enterprise-releases). This page is what to expect and what to plan for. @@ -24,7 +24,7 @@ there. There is nothing else to mirror. **The Helm chart** is more work, because the release is many images rather than one. Every image has to be mirrored into a registry the cluster can reach and the release -pointed at it, component by component — there is no single switch. Plan for the whole +pointed at it, component by component. Plan for the whole set, and expect to do it with your account team rather than alone. **What you give up either way:** images are pinned to the exact content Fish Audio @@ -49,7 +49,7 @@ Regulated deployments usually need evidence rather than a configuration review, runbook carries the exercise. Two things are worth knowing before you plan it. It answers two questions, and they need different methods: whether the deployment -*needs* the internet, and whether it *calls out* when allowed to — the second is what a +*needs* the internet, and whether it *calls out* when allowed to. The second is what a telemetry or data-residency review actually asks. It is also a cluster-level exercise rather than a namespace one. Confirm early that @@ -58,6 +58,6 @@ complete it. ## Next steps -- [Requirements](/developer-guide/self-hosting/requirements) — hardware, platform, and network baselines -- [All-in-One container](/developer-guide/self-hosting/all-in-one) — the single-container form -- [Kubernetes deployment](/developer-guide/self-hosting/kubernetes) — the Helm forms +- [Requirements](/developer-guide/self-hosting/requirements): hardware, platform, and network baselines +- [All-in-One container](/developer-guide/self-hosting/all-in-one): the single-container form +- [Kubernetes deployment](/developer-guide/self-hosting/kubernetes): the Helm forms diff --git a/developer-guide/self-hosting/all-in-one.mdx b/developer-guide/self-hosting/all-in-one.mdx index b3520dd..68173d1 100644 --- a/developer-guide/self-hosting/all-in-one.mdx +++ b/developer-guide/self-hosting/all-in-one.mdx @@ -4,8 +4,8 @@ description: "The single-container appliance: what it is, what it cannot do, and icon: "box" --- -The All-in-One image packages the whole speech stack — edge API, model API layer, -inference router and worker, vocoder, text normalizer, and Redis — into one container, +The All-in-One image packages the whole speech stack (edge API, model API layer, +inference router and worker, vocoder, text normalizer, and Redis) into one container, with every model weight baked in. Once the image is on the host it runs with no Kubernetes and no internet access, which makes it the turnkey option for single-node appliances and strict air gaps. @@ -13,7 +13,7 @@ appliances and strict air gaps. This page covers what the appliance is and what to plan for. The commands, tuning options, and troubleshooting are in the **All-in-One guide**, which ships in the - documentation bundle for the image version you run — see + documentation bundle for the image version you run. See [Releases](/developer-guide/self-hosting/enterprise-releases). @@ -21,7 +21,7 @@ appliances and strict air gaps. The appliance runs one inference worker and one vocoder, a GPU each. It does not autoscale, does not shard across more GPUs or nodes, and does not ship the forced -aligner, so it returns no word or segment timings. It is offline-only — there is no +aligner, so it returns no word or segment timings. It is offline-only: there is no hosted-billing variant. For elastic or higher-throughput deployments, use the [Kubernetes chart](/developer-guide/self-hosting/kubernetes), which scales replicas across all GPUs and nodes. @@ -55,7 +55,7 @@ of calling a billing service. Two consequences worth designing around: - **Any non-empty bearer token is accepted**; a missing or empty one is rejected. The appliance has nothing to validate a token against. - **The token is recorded verbatim as the billing identity.** Use a stable, distinct - token per tenant — two tenants sharing a token are indistinguishable in the ledger. + token per tenant. Two tenants sharing a token are indistinguishable in the ledger. See [Offline usage accounting](/developer-guide/self-hosting/air-gapped#offline-usage-accounting). @@ -72,6 +72,6 @@ per voice. ## Next steps -- [Requirements](/developer-guide/self-hosting/requirements#all-in-one-container-host) — host baseline -- [Registry access](/developer-guide/self-hosting/registry-access) — how your team gets the image -- [Air-gapped deployments](/developer-guide/self-hosting/air-gapped) — moving the image to a disconnected host +- [Requirements](/developer-guide/self-hosting/requirements#all-in-one-container-host): host baseline +- [Registry access](/developer-guide/self-hosting/registry-access): how your team gets the image +- [Air-gapped deployments](/developer-guide/self-hosting/air-gapped): moving the image to a disconnected host diff --git a/developer-guide/self-hosting/enterprise-releases.mdx b/developer-guide/self-hosting/enterprise-releases.mdx index 29c9583..842a168 100644 --- a/developer-guide/self-hosting/enterprise-releases.mdx +++ b/developer-guide/self-hosting/enterprise-releases.mdx @@ -25,7 +25,7 @@ A deployment stays on its version until you change it. Take the new version from procedure in the deployment runbook for the Helm forms, or pull the new tag and recreate the container against the same volume for [All-in-One](/developer-guide/self-hosting/all-in-one). Air-gapped deployments -mirror the new version first — see +mirror the new version first. See [Air-gapped deployments](/developer-guide/self-hosting/air-gapped). diff --git a/developer-guide/self-hosting/kubernetes.mdx b/developer-guide/self-hosting/kubernetes.mdx index a2efea8..8c89471 100644 --- a/developer-guide/self-hosting/kubernetes.mdx +++ b/developer-guide/self-hosting/kubernetes.mdx @@ -11,7 +11,7 @@ they differ only in how usage is accounted. This page covers what the deployment involves and what you need to decide. The commands, values, and troubleshooting are in the **deployment runbook**, which ships - in the documentation bundle alongside the chart version you install — see + in the documentation bundle alongside the chart version you install. See [Releases](/developer-guide/self-hosting/enterprise-releases). The runbook is versioned with the chart; this page is not, so follow the runbook when they differ. @@ -53,9 +53,9 @@ manage that layer. Three Kubernetes Secrets have to exist before the install: registry credentials, a JWT secret for the edge API, and one shared between the in-cluster model store and the -workers that read from it — that one authenticates nothing outside the cluster, and the +workers that read from it. That one authenticates nothing outside the cluster, and the value is yours to choose. Generate the JWT secret once -and keep it stable — changing it invalidates issued tokens. For production, prefer +and keep it stable, because changing it invalidates issued tokens. For production, prefer External Secrets, Sealed Secrets, or your cloud secret manager over plain Secrets. ## What to expect @@ -66,13 +66,13 @@ rather than by editing tags. The first install is slow: the inference worker compiles its GPU graphs and the vocoder builds its inference engine before either becomes ready, which takes far longer than any later rollout. Both artifacts are cached on shared storage, so subsequent -starts are quick — as long as that storage persists. +starts are quick, as long as that storage persists. Upgrades, rollback, and uninstall are all standard Helm operations. The runbook covers the order to do them in and what to check at each step. ## Next steps -- [Requirements](/developer-guide/self-hosting/requirements) — hardware, platform, and network baselines -- [Registry access](/developer-guide/self-hosting/registry-access) — how your team gets the chart -- [Operations](/developer-guide/self-hosting/operations) — running it once it is live +- [Requirements](/developer-guide/self-hosting/requirements): hardware, platform, and network baselines +- [Registry access](/developer-guide/self-hosting/registry-access): how your team gets the chart +- [Operations](/developer-guide/self-hosting/operations): running it once it is live diff --git a/developer-guide/self-hosting/operations.mdx b/developer-guide/self-hosting/operations.mdx index ec7ffc2..eb67288 100644 --- a/developer-guide/self-hosting/operations.mdx +++ b/developer-guide/self-hosting/operations.mdx @@ -56,7 +56,7 @@ the compile cost before it becomes ready. Back up: - Reference voice archives, if your traffic uses reference ids. In a self-hosted - deployment these are durable data, not cache — nothing can re-download them. + deployment these are durable data, not cache. Nothing can re-download them. - The offline usage ledger directories, before they are pruned by your own archival process. - The values file used for the production release. diff --git a/developer-guide/self-hosting/registry-access.mdx b/developer-guide/self-hosting/registry-access.mdx index ba47398..52edd52 100644 --- a/developer-guide/self-hosting/registry-access.mdx +++ b/developer-guide/self-hosting/registry-access.mdx @@ -13,16 +13,16 @@ dashboard. Sign in to fish.audio and open **Developer → Self Host**. Everything your team needs to start is there: -- **Your deployment** — the delivery forms your team is granted. If it is empty, or the +- **Your deployment**: the delivery forms your team is granted. If it is empty, or the page reports that self-host deployment is not enabled, contact your account manager. -- **The versions available to you**, and the documentation bundle for each — see +- **The versions available to you**, and the documentation bundle for each. See [Releases](/developer-guide/self-hosting/enterprise-releases). - **Install commands built for your team**, with the registry host, artifact references, and the version you pick already filled in. The registry host, the artifact references, and the versions available to you are - specific to your team and are shown only in the dashboard — copy them from the Self + specific to your team and are shown only in the dashboard. Copy them from the Self Host page. @@ -50,7 +50,7 @@ start is there: A team can hold up to five deploy tokens at a time. Tokens carry the grants of the team that owns them, not of the person who created them, and follow those grants as they -change — there is no token to recreate when your entitlement is updated. +change, so there is no token to recreate when your entitlement is updated. Authenticate with your account email as the username and the deploy token as the password. Docker, Helm, and the cluster's image pull secret all use the same pair; the @@ -66,7 +66,7 @@ exact commands are on the Self Host page and in the deployment runbook. Recommended practice: -- Issue one token per consumer — production cluster, staging cluster, CI mirror — so a +- Issue one token per consumer (production cluster, staging cluster, CI mirror) so a single revocation never takes down more than one of them. - Store tokens in your secret manager, not in values files or version control. - Rotate on your normal credential schedule and whenever someone with access to a token diff --git a/developer-guide/self-hosting/requirements.mdx b/developer-guide/self-hosting/requirements.mdx index 74c583f..60378c0 100644 --- a/developer-guide/self-hosting/requirements.mdx +++ b/developer-guide/self-hosting/requirements.mdx @@ -22,7 +22,7 @@ measured on your own hardware before you commit to a capacity plan. | Network | Low-latency east-west networking | GPU workload stability, service-to-service calls, and shared storage access. | Verify these against your own chart version by rendering the release with a deployment -profile applied — the chart defaults alone stop on a required value — and summing the +profile applied (the chart defaults alone stop on a required value) and summing the requests. ### GPUs diff --git a/features/manage-voices.mdx b/features/manage-voices.mdx index 8915f1f..41a1be2 100644 --- a/features/manage-voices.mdx +++ b/features/manage-voices.mdx @@ -4,11 +4,11 @@ description: "List, inspect, update, and delete your voice models" icon: "sliders" --- -Every voice you [clone](/features/voice-cloning) becomes a model you own. List your library, look up a model's details, rename or re-share it, and delete what you no longer need — all from the API directly, the Python library, or JavaScript. +Every voice you [clone](/features/voice-cloning) becomes a model you own. List your library, look up a model's details, rename or re-share it, and delete what you no longer need, all from the API directly, the Python library, or JavaScript. - No code — manage voices in the browser. + No code: manage voices in the browser. Every endpoint for voice models. diff --git a/features/realtime-streaming.mdx b/features/realtime-streaming.mdx index 2dd90bc..4624440 100644 --- a/features/realtime-streaming.mdx +++ b/features/realtime-streaming.mdx @@ -4,7 +4,7 @@ description: "Stream audio as it generates for the lowest latency" icon: "bolt" --- -Start playing audio before the whole clip is ready. Fish Audio streams speech in chunks, so your users hear the first words in a fraction of a second — essential for voice agents and live narration. Two modes: **HTTP streaming** for text you already have, and **WebSocket** for text that arrives incrementally (like LLM tokens). +Start playing audio before the whole clip is ready. Fish Audio streams speech in chunks, so your users hear the first words in a fraction of a second, essential for voice agents and live narration. Two modes: **HTTP streaming** for text you already have, and **WebSocket** for text that arrives incrementally (like LLM tokens). @@ -25,7 +25,7 @@ Start playing audio before the whole clip is ready. Fish Audio streams speech in Conversational AI where time-to-first-audio matters. - Speak tokens as your model produces them — no waiting for the full reply. + Speak tokens as your model produces them. Long-form content that should start playing immediately. @@ -69,7 +69,7 @@ import { createWriteStream } from "fs"; const client = new FishAudioClient({ apiKey: process.env.FISH_API_KEY }); -// convert() returns a ReadableStream — write each chunk the +// convert() returns a ReadableStream. Write each chunk the // moment it arrives instead of waiting for the whole clip. const stream = await client.textToSpeech.convert( { text: "Streaming keeps latency low.", format: "mp3" }, @@ -88,7 +88,7 @@ file.end(); ## Stream from an LLM -When text arrives token by token, feed a generator to `stream_websocket`. It opens a WebSocket, sends text as you produce it, and yields audio chunks back — so speech keeps pace with your model. +When text arrives token by token, feed a generator to `stream_websocket`. It opens a WebSocket, sends text as you produce it, and yields audio chunks back, so speech keeps pace with your model. ```python Python @@ -118,18 +118,18 @@ for chunk in client.tts.stream_websocket(llm_tokens(), reference_id="YOUR_VOICE_ ### Which mode to use -- **HTTP streaming (`tts.stream`)** — you have the full text up front and want low time-to-first-audio. Simplest option. -- **WebSocket (`tts.stream_websocket`)** — text is still being produced (LLM output, live captions). Lets you start speaking before the sentence is finished. +- **HTTP streaming (`tts.stream`)**: you have the full text up front and want low time-to-first-audio. Simplest option. +- **WebSocket (`tts.stream_websocket`)**: text is still being produced (LLM output, live captions). Lets you start speaking before the sentence is finished. ### Lower the latency further - Use a streaming-friendly format like `mp3` or `pcm`. - Keep the connection warm for back-to-back generations. -- Pair with a cloned voice via `reference_id` — see [Voice Cloning](/features/voice-cloning). +- Pair with a cloned voice via `reference_id` (see [Voice Cloning](/features/voice-cloning)). ## Control where audio generates -The WebSocket buffers incoming text and generates audio once it has enough context for natural-sounding speech, so you don't need to batch tokens yourself. When you *do* want a clean break — end of a sentence, a deliberate pause, or the end of a turn — yield a `FlushEvent` to force generation immediately. Wrap text in a `TextEvent` if you prefer explicit events over bare strings. +The WebSocket buffers incoming text and generates audio once it has enough context for natural-sounding speech, so you don't need to batch tokens yourself. When you *do* want a clean break (end of a sentence, a deliberate pause, or the end of a turn), yield a `FlushEvent` to force generation immediately. Wrap text in a `TextEvent` if you prefer explicit events over bare strings. ```python from fishaudio import FishAudio @@ -151,8 +151,8 @@ for chunk in client.tts.stream_websocket(script(), reference_id="YOUR_VOICE_ID") Both streaming paths take a `latency` mode: -- `latency="balanced"` (default) — lowest time-to-first-audio. Use it for voice agents and live LLM output. -- `latency="normal"` — slightly higher latency, best audio quality. Use it for narration where you can afford a beat. +- `latency="balanced"` (default): lowest time-to-first-audio. Use it for voice agents and live LLM output. +- `latency="normal"`: slightly higher latency, best audio quality. Use it for narration where you can afford a beat. ```python for chunk in client.tts.stream_websocket(llm_tokens(), latency="balanced"): @@ -189,7 +189,7 @@ async def main(): async for token in your_async_llm(): yield token - # stream_websocket is an async generator — iterate it, don't await the call + # stream_websocket is an async generator: iterate it, don't await the call async for chunk in client.tts.stream_websocket( llm_tokens(), reference_id="YOUR_VOICE_ID", latency="balanced" ): @@ -200,7 +200,7 @@ asyncio.run(main()) ## Direct API (no SDK) -Token-level streaming runs over the WebSocket endpoint — the SDK's `stream_websocket()` handles framing for you. To speak the protocol directly, send MessagePack frames over the socket; the same `application/msgpack` payload format also works for one-shot HTTP streaming, which is faster to serialize than JSON for large reference audio: +Token-level streaming runs over the WebSocket endpoint. The SDK's `stream_websocket()` handles framing for you. To speak the protocol directly, send MessagePack frames over the socket; the same `application/msgpack` payload format also works for one-shot HTTP streaming, which is faster to serialize than JSON for large reference audio: ```python import os diff --git a/features/speech-to-text.mdx b/features/speech-to-text.mdx index 455f316..5fbac02 100644 --- a/features/speech-to-text.mdx +++ b/features/speech-to-text.mdx @@ -4,11 +4,11 @@ description: "Transcribe audio to text with per-segment timestamps" icon: "waveform" --- -Turn spoken audio into accurate text — with timed segments — using Fish Audio's ASR model. Send an audio file, get back the transcript, its duration, and timestamped segments. Works the same from the API directly, the Python library, or JavaScript. +Turn spoken audio into accurate text using Fish Audio's ASR model. Send an audio file, get back the transcript, its duration, and timestamped segments. Works the same from the API directly, the Python library, or JavaScript. - No code — upload audio, get a transcript. + No code: upload audio, get a transcript. Every parameter for `POST /v1/asr`. @@ -77,7 +77,7 @@ The response gives you the full `text`, the audio `duration` in seconds, and tim ## Read the timestamps -Each segment carries `start` and `end` times in seconds — ideal for captions. With the API, ask for them explicitly with `ignore_timestamps=false`. +Each segment carries `start` and `end` times in seconds, ideal for captions. With the API, ask for them explicitly with `ignore_timestamps=false`. ```python Python @@ -100,14 +100,14 @@ curl --request POST https://api.fish.audio/v1/asr \ - In the Python SDK, segment timestamps are **on by default** — pass `include_timestamps=False` to skip them. That's the *inverse* of the API/JavaScript flag `ignore_timestamps`. + In the Python SDK, segment timestamps are **on by default**. Pass `include_timestamps=False` to skip them. That's the *inverse* of the API/JavaScript flag `ignore_timestamps`. ## Implementation details ### Language -`language` is optional — Fish Audio auto-detects it when you omit it. Pass an ISO code (`en`, `zh`, `ja`, …) to pin it and improve accuracy on short or noisy clips. +`language` is optional: Fish Audio auto-detects it when you omit it. Pass an ISO code (`en`, `zh`, `ja`, …) to pin it and improve accuracy on short or noisy clips. ```python Python @@ -129,7 +129,7 @@ curl --request POST https://api.fish.audio/v1/asr \ ### Input audio -Common formats work directly — `wav`, `mp3`, `opus`, and more. Send the raw file bytes; no pre-processing required. The endpoint accepts `multipart/form-data` (shown above) or `application/msgpack`. +Common formats work directly: `wav`, `mp3`, `opus`, and more. Send the raw file bytes; no pre-processing required. The endpoint accepts `multipart/form-data` (shown above) or `application/msgpack`. ### File limits @@ -137,7 +137,7 @@ One request transcribes one audio file. The endpoint accepts files up to **20 MB ### Async transcription -The Python SDK ships an async client with the same surface — useful when you're transcribing many files concurrently or already running inside an event loop. Use `AsyncFishAudio` and `await` the call: +The Python SDK ships an async client with the same surface, useful when you're transcribing many files concurrently or already running inside an event loop. Use `AsyncFishAudio` and `await` the call: ```python import asyncio @@ -171,7 +171,7 @@ for result in asyncio.run(transcribe_all(["speech.wav"])): ### Direct API (MessagePack) -`POST /v1/asr` also accepts a [MessagePack](https://msgpack.org) body instead of multipart form data — the same path the API reference links to for low-overhead, server-side calls. Pack the audio bytes and options into one payload and set `Content-Type: application/msgpack`: +`POST /v1/asr` also accepts a [MessagePack](https://msgpack.org) body instead of multipart form data, the same path the API reference links to for low-overhead, server-side calls. Pack the audio bytes and options into one payload and set `Content-Type: application/msgpack`: ```python import os @@ -201,7 +201,7 @@ The response shape is identical to the multipart path: `text`, `duration` (secon - The reverse direction — text to lifelike audio. + The reverse direction: text to lifelike audio. Every field and the raw response schema. diff --git a/features/text-to-speech.mdx b/features/text-to-speech.mdx index b2fee42..1db1227 100644 --- a/features/text-to-speech.mdx +++ b/features/text-to-speech.mdx @@ -1,15 +1,15 @@ --- title: "Text to Speech" sidebarTitle: "Overview" -description: "Turn text into lifelike speech — use it however you build" +description: "Turn text into lifelike speech" icon: "microphone" --- -Generate natural speech from text with the `s2.1-pro`, `s2-pro`, and `s1` models. Pick a voice, choose a format, and go — from the API directly, the Python library, or JavaScript. +Generate natural speech from text with the `s2.1-pro`, `s2-pro`, and `s1` models, from the API directly, the Python library, or JavaScript. - No code — type, pick a voice, generate. + No code: type, pick a voice, generate. Every parameter for `POST /v1/tts`. @@ -26,7 +26,7 @@ Generate natural speech from text with the `s2.1-pro`, `s2-pro`, and `s1` models Audiobooks, explainers, ads, and video narration. - Speak an assistant's replies — pair with [streaming](/features/realtime-streaming) for low latency. + Speak an assistant's replies. Pair with [streaming](/features/realtime-streaming) for low latency. Read content aloud, phone menus, notifications. @@ -107,10 +107,10 @@ curl --request POST https://api.fish.audio/v1/tts \ ### Models -- **`s2.1-pro`** — recommended for production, with improved quality, latency, and throughput over S2-Pro. -- **`s2.1-pro-free`** — the same model at $0 for testing, prototyping, development, and smaller businesses, without TTFA or DPA guarantees. -- **`s2-pro`** — previous-generation S2 model with multi-speaker and natural-language expression control. -- **`s1`** — previous generation, `(parenthesis)` emotion tags. +- **`s2.1-pro`**: recommended for production, with improved quality, latency, and throughput over S2-Pro. +- **`s2.1-pro-free`**: the same model at $0 for testing, prototyping, development, and smaller businesses, without TTFA or DPA guarantees. +- **`s2-pro`**: previous-generation S2 model with multi-speaker and natural-language expression control. +- **`s1`**: previous generation, `(parenthesis)` emotion tags. In the API, select with the `model` request header; requests without it use `s2.1-pro`. The Python SDK always sends a model and defaults to `model="s2-pro"`. See [Choosing a Model](/developer-guide/models-pricing/choosing-a-model). @@ -163,7 +163,7 @@ The Python SDK exposes three ways to generate, depending on whether you have the | Method | Returns | Use it for | |---|---|---| -| `tts.convert()` | complete audio `bytes` | most cases — you have the text, you want the file | +| `tts.convert()` | complete audio `bytes` | most cases: you have the text, you want the file | | `tts.stream()` | `AudioStream` (iterate chunks, or `.collect()`) | memory-efficient transfer of large audio; write chunks to disk as they arrive | | `tts.stream_websocket()` | iterator of audio `bytes` | text arriving in real time (LLM tokens, live captions) | @@ -179,7 +179,7 @@ For real-time text streaming with `stream_websocket()`, see [Realtime Streaming] ### Instant voice cloning (reference audio) -Instead of a saved `reference_id`, pass raw audio plus its transcript to clone a voice on the fly — no training step. Best with a clean 10–30s sample. +Instead of a saved `reference_id`, pass raw audio plus its transcript to clone a voice on the fly, with no training step. Best with a clean 10–30s sample. ```python from fishaudio.types import ReferenceAudio @@ -201,7 +201,7 @@ Pick a format for your delivery channel, and tune bitrate to trade size against |---|---| | `mp3` (default) | good size/quality balance; set `mp3_bitrate` to `64`, `128`, or `192` | | `wav` | uncompressed, highest quality; set `sample_rate` (e.g. `44100`) | -| `pcm` | raw samples, no container — for low-latency playback and telephony pipelines | +| `pcm` | raw samples, no container; for low-latency playback and telephony pipelines | | `opus` | efficient for streaming; bitrate is automatic (`opus_bitrate=-1000`) | ```python @@ -217,9 +217,9 @@ audio = client.tts.convert( `latency` trades stability for speed; `chunk_length` controls how much text the engine batches before it starts generating. -- `latency="balanced"` (default) — lower time-to-first-audio (~300ms). Good for interactive use. -- `latency="normal"` — most stable output, at slightly higher latency. -- `chunk_length` (`100`–`300`, default `200`) — smaller chunks start audio sooner; larger chunks are more efficient for long text. +- `latency="balanced"` (default): lower time-to-first-audio (~300ms). Good for interactive use. +- `latency="normal"`: most stable output, at slightly higher latency. +- `chunk_length` (`100`–`300`, default `200`): smaller chunks start audio sooner; larger chunks are more efficient for long text. ```python Python @@ -243,7 +243,7 @@ curl --request POST https://api.fish.audio/v1/tts \ ### Direct API (MessagePack) -`POST /v1/tts` also accepts a MessagePack body (`Content-Type: application/msgpack`) — the path the [API reference](/api-reference/endpoint/openapi-v1/text-to-speech) is built around. Use it to send binary reference audio in the request without base64 overhead, or when you don't want the SDK. +`POST /v1/tts` also accepts a MessagePack body (`Content-Type: application/msgpack`), the path the [API reference](/api-reference/endpoint/openapi-v1/text-to-speech) is built around. Use it to send binary reference audio in the request without base64 overhead, or when you don't want the SDK. ```python import os @@ -265,11 +265,11 @@ with open("out.mp3", "wb") as f: f.write(resp.content) ``` -The `model` header is optional — if omitted or set to an unrecognized value, the request falls back to `s2.1-pro`. JSON and MessagePack accept the same fields. +If the `model` header is omitted or set to an unrecognized value, the request falls back to `s2.1-pro`. JSON and MessagePack accept the same fields. ### Advanced generation tuning -For finer control, `TTSConfig` exposes the model's sampling parameters. The defaults are well-tuned — reach for these only when you need to dial in determinism or curb artifacts. +For finer control, `TTSConfig` exposes the model's sampling parameters. The defaults are well-tuned. Reach for these only when you need to dial in determinism or curb artifacts. ```python from fishaudio.types import TTSConfig, Prosody @@ -286,7 +286,7 @@ config = TTSConfig( audio = client.tts.convert(text="Carefully tuned output.", config=config) ``` -A `TTSConfig` is reusable — define it once and pass it to many `convert()` calls. See the [full field list](/api-reference/sdk/python/types#ttsconfig-objects) for every parameter and default. +Define a `TTSConfig` once and pass it to many `convert()` calls. See the [full field list](/api-reference/sdk/python/types#ttsconfig-objects) for every parameter and default. ## Going further diff --git a/features/voice-cloning.mdx b/features/voice-cloning.mdx index e4fc44f..b8d7e4b 100644 --- a/features/voice-cloning.mdx +++ b/features/voice-cloning.mdx @@ -4,11 +4,11 @@ description: "Create a custom voice from audio samples, then speak with it" icon: "clone" --- -Build a reusable voice model from your own audio, then use it anywhere you generate speech. You get back a voice **id** — pass it as `reference_id` to [Text to Speech](/features/text-to-speech) and every generation speaks in that voice. Works from the API directly, the Python library, or JavaScript. +Build a reusable voice model from your own audio, then use it anywhere you generate speech. You get back a voice **id**. Pass it as `reference_id` to [Text to Speech](/features/text-to-speech) and every generation speaks in that voice. Works from the API directly, the Python library, or JavaScript. - No code — clone a voice in the browser. + No code: clone a voice in the browser. Every field for `POST /model`. @@ -90,7 +90,7 @@ console.log(voice._id, voice.state); ## Use your cloned voice -Pass the voice **id** as `reference_id` to Text to Speech — exactly like any other voice. +Pass the voice **id** as `reference_id` to Text to Speech, exactly like any other voice. ```python Python @@ -141,14 +141,14 @@ curl --request POST https://api.fish.audio/model \ ### Visibility -Models are `private` by default. Set `unlist` for a shareable link, or `public` to publish to the [Voice Library](/overview/platform). You can change this later — see [Manage Voices](/features/manage-voices). +Models are `private` by default. Set `unlist` for a shareable link, or `public` to publish to the [Voice Library](/overview/platform). You can change this later in [Manage Voices](/features/manage-voices). ## Instant vs. persistent clones There are two ways to clone: -- **Persistent model** (above) — train once with `voices.create()`, get back a reusable `id`. Best when you'll use the same voice repeatedly. -- **Instant clone** — pass reference audio inline on each generation with no model to manage. Best for one-off or per-request voices. +- **Persistent model** (above): train once with `voices.create()`, get back a reusable `id`. Best when you'll use the same voice repeatedly. +- **Instant clone**: pass reference audio inline on each generation with no model to manage. Best for one-off or per-request voices. For an instant clone, send the reference audio (and its transcript) directly to Text to Speech via `references` instead of `reference_id`: diff --git a/overview/capabilities.mdx b/overview/capabilities.mdx index 13a54b7..5e1965a 100644 --- a/overview/capabilities.mdx +++ b/overview/capabilities.mdx @@ -1,7 +1,7 @@ --- title: "Overview" sidebarTitle: "Overview" -description: "Everything Fish Audio can do — and how to build with it" +description: "Everything Fish Audio can do, and how to build with it" icon: "house" --- @@ -23,7 +23,7 @@ Fish Audio is a voice AI platform. Every core feature is available three ways: i - Stream audio as it generates — for voice agents and live apps. + Stream audio as it generates, for voice agents and live apps. @@ -33,7 +33,7 @@ Fish Audio is a voice AI platform. Every core feature is available three ways: i ## Also in the web app -These run in the browser, no code required — see the [Platform guide](/overview/platform). +These run in the browser, no code required. See the [Platform guide](/overview/platform). @@ -41,7 +41,7 @@ These run in the browser, no code required — see the [Platform guide](/overvie - Produce multi-speaker, long-form audio — audiobooks and narration. + Produce multi-speaker, long-form audio: audiobooks and narration. @@ -57,10 +57,10 @@ These run in the browser, no code required — see the [Platform guide](/overvie These text-to-speech models power most capabilities: -- **`s2.1-pro`** — the recommended production model, with improved quality, latency, and throughput over S2-Pro. -- **`s2.1-pro-free`** — the same model at $0 for testing, prototyping, development, and smaller businesses, without TTFA or DPA guarantees. -- **`s2-pro`** — the previous-generation S2 model, with multi-speaker and natural-language expression control. -- **`s1`** — the previous generation, with `(parenthesis)` emotion tags. +- **`s2.1-pro`**: the recommended production model, with improved quality, latency, and throughput over S2-Pro. +- **`s2.1-pro-free`**: the same model at $0 for testing, prototyping, development, and smaller businesses, without TTFA or DPA guarantees. +- **`s2-pro`**: the previous-generation S2 model, with multi-speaker and natural-language expression control. +- **`s1`**: the previous generation, with `(parenthesis)` emotion tags. See [Models Overview](/developer-guide/models-pricing/models-overview) and [Choosing a Model](/developer-guide/models-pricing/choosing-a-model) for the full lineup, languages, and limits. @@ -68,7 +68,7 @@ See [Models Overview](/developer-guide/models-pricing/models-overview) and [Choo - No code — generate audio, clone voices, and produce projects in your browser. + No code: generate audio, clone voices, and produce projects in your browser. diff --git a/overview/mcp.mdx b/overview/mcp.mdx index 5c70f6a..f8147e1 100644 --- a/overview/mcp.mdx +++ b/overview/mcp.mdx @@ -1,13 +1,13 @@ --- title: "MCP Server" -description: "Connect Claude, Cursor, or any MCP client to your Fish Audio account — search voices, generate speech, and transcribe audio" +description: "Connect Claude, Cursor, or any MCP client to your Fish Audio account: search voices, generate speech, and transcribe audio" icon: "plug" --- The Fish Audio MCP server gives AI agents direct access to your Fish Audio account: they can browse the voice library, generate speech, and transcribe audio for you. - **Endpoint**: `https://api.fish.audio/mcp` (streamable HTTP) -- **Authentication**: OAuth. Your MCP client opens a browser window and you sign in with your Fish Audio account — no API key required. +- **Authentication**: OAuth. Your MCP client opens a browser window and you sign in with your Fish Audio account. - **Billing**: usage draws from your plan's package credits, exactly like the [web app](https://fish.audio/app). It does not consume developer API credits. ## Connect @@ -68,11 +68,11 @@ The Fish Audio MCP server gives AI agents direct access to your Fish Audio accou ## Try it -Once connected, ask in plain language — the agent picks the right tools: +Once connected, ask in plain language. The agent picks the right tools: - "Find a calm English narration voice and read intro.md aloud — give me the audio link." + "Find a calm English narration voice, read intro.md aloud, and give me the audio link." @@ -92,7 +92,7 @@ In coding agents like Claude Code and Codex, tools compose with the shell: the a ## Good to know -- Add audio tags in square brackets inside the text to control delivery — `[whispering]`, `[excited]`, `[laughing]`. Tags are performed, never spoken. +- Add audio tags in square brackets inside the text to control delivery: `[whispering]`, `[excited]`, `[laughing]`. Tags are performed, never spoken. - If no voice is specified, a curated default voice for the requested language is used. - Generated audio is returned as a permanent URL; failed generations are refunded. -- Local files can be transcribed too — the agent uploads them to a temporary slot that is deleted automatically after 7 days. +- Local files can be transcribed too. The agent uploads them to a temporary slot that is deleted automatically after 7 days. diff --git a/overview/platform.mdx b/overview/platform.mdx index 464844e..4425efe 100644 --- a/overview/platform.mdx +++ b/overview/platform.mdx @@ -1,6 +1,6 @@ --- title: "Platform (Web App)" -description: "Use Fish Audio in your browser — no code required" +description: "Use Fish Audio in your browser, no code required" icon: "browser" --- @@ -60,8 +60,8 @@ The [Fish Audio web app](https://fish.audio/app) gives you every capability with ## Produce projects - Assemble multi-speaker, long-form audio — audiobooks, dialogue, and narration - — in a project editor. + Assemble multi-speaker, long-form audio (audiobooks, dialogue, and narration) + in a project editor. ## Library & history diff --git a/snippets/compat-callout.mdx b/snippets/compat-callout.mdx index de8d47a..8f341ef 100644 --- a/snippets/compat-callout.mdx +++ b/snippets/compat-callout.mdx @@ -1,6 +1,6 @@ **Already using the OpenAI, OpenRouter, ElevenLabs, or Groq SDK?** Swap in a - Fish Audio base URL and API key and it keeps working — + Fish Audio base URL and API key and it keeps working. [Migrate to Fish Audio](/developer-guide/getting-started/migration) has the URL for each SDK. diff --git a/tests/cookbooks/README.md b/tests/cookbooks/README.md index 595c10a..2cb1ee9 100644 --- a/tests/cookbooks/README.md +++ b/tests/cookbooks/README.md @@ -17,11 +17,11 @@ pytest tests/cookbooks -v ``` If no key is found (env var, `.env`, or local keyfile), the whole suite **skips** rather -than fails — so it's safe in CI without secrets. +than fails, so it's safe in CI without secrets. ## Add a recipe -Append a spec to `specs.py` — no test code changes: +Append a spec to `specs.py` (no test code changes): ```python { @@ -34,17 +34,17 @@ Append a spec to `specs.py` — no test code changes: ``` Per-case keys: -- `block` — index of the `python` code block in the page (document order). -- `subs` — `{placeholder: replacement}` string substitutions. -- `file` — `(filename, format)` the recipe should write (validated by magic bytes). -- `var` — `(variable, format)` an audio-bytes variable the recipe should define. -- `consumed` — assert the injected `consume()` drained a non-empty stream. -- `postamble` — extra code run after the block (e.g. to drive a generator the block defines). +- `block`: index of the `python` code block in the page (document order). +- `subs`: `{placeholder: replacement}` string substitutions. +- `file`: `(filename, format)` the recipe should write (validated by magic bytes). +- `var`: `(variable, format)` an audio-bytes variable the recipe should define. +- `consumed`: assert the injected `consume()` drained a non-empty stream. +- `postamble`: extra code run after the block (e.g. to drive a generator the block defines). ## Tiers -- **T1 (here):** pure Fish Audio recipes — run fully live. -- **T2:** integration seams (e.g. the `fish-tts` CLI, framework plugins) — test the +- **T1 (here):** pure Fish Audio recipes, run fully live. +- **T2:** integration seams (e.g. the `fish-tts` CLI, framework plugins): test the Fish-facing component live; the external framework is contract-checked. -- **T3:** full external round-trips (Telegram/Discord/Twilio) — staging only; see each +- **T3:** full external round-trips (Telegram/Discord/Twilio): staging only; see each tutorial's manual checklist and any credential-guarded integration test.