diff --git a/docs/ecosystem.rst b/docs/ecosystem.rst index fa8e334fa..ae658b491 100644 --- a/docs/ecosystem.rst +++ b/docs/ecosystem.rst @@ -40,6 +40,9 @@ LLM & AI Frameworks * - **OpenAI** - Use OpenAI models (and any OpenAI API-compatible server) inside Burr actions. - `Example `__ + * - **OrcaRouter** + - Use OrcaRouter's gateway (OpenAI-compatible endpoint, ``https://api.orcarouter.ai/v1``) inside Burr actions for a wide range of models through one API key. + - `Example `__ * - **LangChain / LCEL** - Use LangChain chains and runnables as Burr actions. Includes a custom serialization plugin to persist LangChain objects in state. - :doc:`Reference ` | diff --git a/examples/README.md b/examples/README.md index 9d4447cc2..a9536a0e1 100644 --- a/examples/README.md +++ b/examples/README.md @@ -48,6 +48,7 @@ Note we have a few more in [other-examples](other-examples/), but those do not y - [multi-agent-collaboration](multi-agent-collaboration/) - This example shows how to use Burr to create a multi-agent collaboration. This is a clone of the following [LangGraph example](https://github.com/langchain-ai/langgraph/blob/main/examples/multi_agent/multi-agent-collaboration.ipynb). - [multi-modal-chatbot](multi-modal-chatbot/) - This example shows how to use Burr to create a multi-modal chatbot. This demonstrates how to use a model to delegate to other models conditionally. - [streaming-overview](streaming-overview/) - This example shows how we can use the streaming API to respond to return quicker results to the user and build a seamless experience +- [orcarouter-agent](orcarouter-agent/) - A small stateful chat agent backed by [OrcaRouter](https://www.orcarouter.ai), using its OpenAI-compatible API. - [integrations/bedrock](integrations/bedrock/) - Minimal graphs using Amazon Bedrock (`BedrockAction` and `BedrockStreamingAction`). - [tracing-and-spans](tracing-and-spans/) - This example shows how to use Burr to create a simple chatbot with additional visibility. This is a good starting point for understanding how to use Burr's tracing functionality. - [web-server](web-server/) - This example shows how to use Burr in a web server. This is a good starting point for understanding how to use Burr for interaction. diff --git a/examples/orcarouter-agent/README.md b/examples/orcarouter-agent/README.md new file mode 100644 index 000000000..71aa2b33f --- /dev/null +++ b/examples/orcarouter-agent/README.md @@ -0,0 +1,81 @@ + + +# A Burr agent backed by OrcaRouter + +This example shows how to build a small stateful chat agent with Burr and have it +talk to [OrcaRouter](https://www.orcarouter.ai) — a gateway that provides an +OpenAI-compatible API for a wide range of models through a single endpoint. + +It also runs gateway-level, zero-trust security for AI agents on the same endpoint — +screening every prompt/response and governing every tool call on a default-deny basis, +with no application code changes. + +OrcaRouter exposes the OpenAI-compatible API at: + +``` +https://api.orcarouter.ai/v1 +``` + +Because the API is OpenAI-compatible, we can use the `openai` Python client and +simply point `base_url` at OrcaRouter. The `orcarouter/auto` model alias routes to a +default capable model. + +## Setup + +```bash +pip install "apache-burr[start]" openai +``` + +Then set your OrcaRouter API key: + +```bash +export ORCAROUTER_API_KEY="your-orca-key" +``` + +Optionally override the endpoint or model: + +```bash +export ORCAROUTER_BASE_URL="https://api.orcarouter.ai/v1" # default +export ORCAROUTER_MODEL="orcarouter/auto" # default +``` + +## Running + +Run the example from the `examples/orcarouter-agent` directory: + +```bash +python application.py "What is Apache Burr?" +``` + +This will build the state machine (`statemachine.png`), send your prompt to +OrcaRouter, and print the reply. The agent loops between a `human_input` action and +an `ai_response` action, accumulating a `chat_history` in state. + +You can also open `notebook.ipynb` and run the cells step by step. + +## How it works + +- `human_input` reads the prompt and appends it to the chat history in state. +- `ai_response` sends the full chat history to + `https://api.orcarouter.ai/v1/chat/completions` using the `orcarouter/auto` model + and stores the reply back in state. + +The state machine is small on purpose — swap in more actions (tool calls, human +approval, sub-agents) to build a full agent on top of OrcaRouter. diff --git a/examples/orcarouter-agent/__init__.py b/examples/orcarouter-agent/__init__.py new file mode 100644 index 000000000..13a83393a --- /dev/null +++ b/examples/orcarouter-agent/__init__.py @@ -0,0 +1,16 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. diff --git a/examples/orcarouter-agent/application.py b/examples/orcarouter-agent/application.py new file mode 100644 index 000000000..24317f445 --- /dev/null +++ b/examples/orcarouter-agent/application.py @@ -0,0 +1,98 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""A minimal Burr agent that talks to [OrcaRouter](https://www.orcarouter.ai). + +[OrcaRouter](https://www.orcarouter.ai) exposes an OpenAI-compatible API at +``https://api.orcarouter.ai/v1``, so we can point the ``openai`` client at it +and use the ``orcarouter/auto`` model alias. Set ``ORCAROUTER_API_KEY`` to +your OrcaRouter key before running. +""" + +import os +from typing import Tuple + +import openai + +from burr.core import ApplicationBuilder, State, action + +ORCAROUTER_BASE_URL = os.getenv("ORCAROUTER_BASE_URL", "https://api.orcarouter.ai/v1") +ORCAROUTER_MODEL = os.getenv("ORCAROUTER_MODEL", "orcarouter/auto") + + +def _orcarouter_client() -> openai.OpenAI: + """Create an OpenAI-compatible client pointed at the OrcaRouter gateway.""" + return openai.OpenAI( + base_url=ORCAROUTER_BASE_URL, + api_key=os.environ["ORCAROUTER_API_KEY"], + ) + + +@action(reads=[], writes=["prompt", "chat_history"]) +def human_input(state: State, prompt: str) -> Tuple[dict, State]: + """Pull human input from the outside world and add it to the chat history.""" + chat_item = {"content": prompt, "role": "user"} + return {"prompt": prompt}, state.update(prompt=prompt).append(chat_history=chat_item) + + +@action(reads=["chat_history"], writes=["response", "chat_history"]) +def ai_response(state: State) -> Tuple[dict, State]: + """Query OrcaRouter with the full chat history.""" + client = _orcarouter_client() + content = ( + client.chat.completions.create( + model=ORCAROUTER_MODEL, + messages=state["chat_history"], + ) + .choices[0] + .message.content + ) + chat_item = {"content": content, "role": "assistant"} + return {"response": content}, state.update(response=content).append(chat_history=chat_item) + + +def application(): + return ( + ApplicationBuilder() + .with_actions( + human_input=human_input, + ai_response=ai_response, + ) + .with_transitions( + ("human_input", "ai_response"), + ("ai_response", "human_input"), + ) + .with_state(chat_history=[]) + .with_entrypoint("human_input") + .build() + ) + + +if __name__ == "__main__": + import sys + + app = application() + app.visualize( + output_file_path="statemachine", + include_conditions=False, + view=False, + format="png", + ) + + prompt = sys.argv[1] if len(sys.argv) > 1 else "Tell me a one-sentence fact about the ocean." + _, result, state = app.run(halt_after=["ai_response"], inputs={"prompt": prompt}) + print(result["response"]) diff --git a/examples/orcarouter-agent/notebook.ipynb b/examples/orcarouter-agent/notebook.ipynb new file mode 100644 index 000000000..d8225efbf --- /dev/null +++ b/examples/orcarouter-agent/notebook.ipynb @@ -0,0 +1,169 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Licensed to the Apache Software Foundation (ASF) under one\n# or more contributor license agreements. See the NOTICE file\n# distributed with this work for additional information\n# regarding copyright ownership. The ASF licenses this file\n# to you under the Apache License, Version 2.0 (the\n# \"License\"); you may not use this file except in compliance\n# with the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing,\n# software distributed under the License is distributed on an\n# \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n# KIND, either express or implied. See the License for the\n# specific language governing permissions and limitations\n# under the License.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "[OrcaRouter](https://www.orcarouter.ai) exposes an OpenAI-compatible API at `https://api.orcarouter.ai/v1`.\n", + "\n", + "Install the dependencies and set your `ORCAROUTER_API_KEY`:\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install \"apache-burr[start]\" openai" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from typing import Tuple\n", + "\n", + "import openai # OrcaRouter is OpenAI-compatible\n", + "\n", + "from burr.core import action, State, Application\n", + "\n", + "ORCAROUTER_BASE_URL = os.getenv(\"ORCAROUTER_BASE_URL\", \"https://api.orcarouter.ai/v1\")\n", + "ORCAROUTER_MODEL = os.getenv(\"ORCAROUTER_MODEL\", \"orcarouter/auto\")\n", + "\n", + "assert \"ORCAROUTER_API_KEY\" in os.environ, \"set ORCAROUTER_API_KEY first\"\n", + "\n", + "def _orcarouter_client() -> openai.OpenAI:\n", + " return openai.OpenAI(\n", + " base_url=ORCAROUTER_BASE_URL,\n", + " api_key=os.environ[\"ORCAROUTER_API_KEY\"],\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Define Actions\n", + "\n", + "We define two actions:\n", + "1. `human_input` -- this is the first one, it accepts a prompt from the outside and adds it to the state\n", + "2. `ai_response` -- this sends the full chat history to OrcaRouter's `chat/completions` endpoint and stores the reply\n", + "\n", + "Note we're only ever touching the `openai` client and pointing its `base_url` at OrcaRouter." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "@action(reads=[], writes=[\"prompt\", \"chat_history\"])\n", + "def human_input(state: State, prompt: str) -> Tuple[dict, State]:\n", + " \"\"\"Pulls human input from the outside world and adds it to the chat history.\"\"\"\n", + " chat_item = {\"content\": prompt, \"role\": \"user\"}\n", + " return {\"prompt\": prompt}, state.update(prompt=prompt).append(chat_history=chat_item)\n", + "\n", + "\n", + "@action(reads=[\"chat_history\"], writes=[\"response\", \"chat_history\"])\n", + "def ai_response(state: State) -> Tuple[dict, State]:\n", + " \"\"\"Queries OrcaRouter with the chat history.\"\"\"\n", + " client = _orcarouter_client()\n", + " content = (\n", + " client.chat.completions.create(\n", + " model=ORCAROUTER_MODEL,\n", + " messages=state[\"chat_history\"],\n", + " )\n", + " .choices[0]\n", + " .message.content\n", + " )\n", + " chat_item = {\"content\": content, \"role\": \"assistant\"}\n", + " return {\"response\": content}, state.update(response=content).append(chat_history=chat_item)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Create the app\n", + "\n", + "We create our app by adding our actions, then adding transitions. The agent loops forever between `human_input` and `ai_response`, accumulating a `chat_history` in state." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "app = (\n", + " ApplicationBuilder().with_actions(\n", + " human_input=human_input,\n", + " ai_response=ai_response\n", + " ).with_transitions(\n", + " (\"human_input\", \"ai_response\"),\n", + " (\"ai_response\", \"human_input\"),\n", + " ).with_state(chat_history=[]).with_entrypoint(\"human_input\").build()\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Run the app\n", + "\n", + "To run the app, we call the `.run` function, passing in a stopping condition. In this case, we want it to halt after `ai_response`. It returns the result, and the resulting state." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "final_action, result, state = app.run(\n", + " halt_after=[\"ai_response\"],\n", + " inputs={\"prompt\": \"What is Apache Burr?\"},\n", + ")\n", + "print(state[\"response\"])" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.4" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/examples/orcarouter-agent/requirements.txt b/examples/orcarouter-agent/requirements.txt new file mode 100644 index 000000000..5e8f6fc0e --- /dev/null +++ b/examples/orcarouter-agent/requirements.txt @@ -0,0 +1,19 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +burr[start] +openai diff --git a/examples/orcarouter-agent/statemachine.png b/examples/orcarouter-agent/statemachine.png new file mode 100644 index 000000000..15cc44d92 Binary files /dev/null and b/examples/orcarouter-agent/statemachine.png differ