Migrate from TensorZero

Migrating from TensorZero to Routeplane

TensorZero is an open-source LLMOps platform, a Rust gateway bundled with observability, evaluations, optimization (fine-tuning, automated prompt engineering), and experimentation, all backed by a ClickHouse data warehouse and driven by a GitOps tensorzero.toml. Routeplane solves a narrower problem on purpose: an agent-native proxy with a single OpenAI-compatible surface, runnable as a local binary (BYOK, no infra).

This guide is for teams who adopted TensorZero mainly as a gateway (unified provider access, routing, retries, fallback) and want a leaner, agent-first surface without operating a database. If you lean on TensorZero’s optimization and experimentation loop, read What Routeplane intentionally doesn’t ship before you commit.

Why migrate?

	TensorZero	Routeplane (self-hosted)
Runtime	Rust gateway	Rust (single static binary)
Production deps	ClickHouse (observability store) + Docker/Compose	None
Configuration	tensorzero.toml — functions, variants, models, providers	Provider keys + routing presets; no schema to maintain
Deployment	Self-hosted only	Local binary (Apache 2.0)
Design focus	Full LLMOps loop: gateway + observability + evals + optimization + experimentation	Agent-first proxy: MCP / ACP / Skills, server tools, presets
Agent protocol surface	None (gateway is provider-facing)	MCP, ACP, Skills, CLI — the product, not an add-on
License	Apache 2.0	Apache 2.0

Two things worth highlighting

1. A proxy, not a platform

TensorZero is built around a feedback loop: route inference, store every call and its metrics in ClickHouse, then evaluate, fine-tune, and A/B test against that history. That’s powerful if you run that loop. And it’s why TensorZero needs a database and a typed config of functions and variants.

Routeplane deliberately stops at the gateway. There’s no tensorzero.toml to maintain, no ClickHouse to operate. You set provider keys and route by provider/model id. OpenTelemetry gives you spans and metrics for every request (the same OTLP export TensorZero offers), but they go to a backend you already run, not a warehouse the proxy manages.

2. Agent-native, single surface

TensorZero’s gateway is provider-facing. It unifies the upstreams. Routeplane adds the agent-facing half: an MCP gateway for tools, an ACP gateway for agent identity and dispatch, a server-tool loop, and structured outputs across providers. The local binary exposes an OpenAI-compatible endpoint for the clients and agents you run.

Migration paths

From an OpenAI-compatible client

If your app already talks to TensorZero’s OpenAI-compatible endpoint, migration is a base-URL swap plus dropping the tensorzero::… model prefix for a plain provider/model id:

<Tabs items={[‘Before (TensorZero)’, ‘After (Routeplane)’]}>

import openai

client = openai.OpenAI(
    base_url="http://localhost:3000/openai/v1",
    api_key="not-used",
)

response = client.chat.completions.create(
    # TensorZero model / function reference
    model="tensorzero::model_name::openai::gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)

```python import openai

Local: routeplane (BYOK via env vars) — see /get-started/quickstart

client = openai.OpenAI( base_url=“http://127.0.0.1:4356/v1”, api_key=“not-used-in-local-byok”, # loopback proxy accepts placeholder; real key is on the daemon )

response = client.chat.completions.create( model=“openai/gpt-4o-mini”, messages=[{“role”: “user”, “content”: “Hello”}], )

</Tab>
</Tabs>

A TensorZero **function** (a named prompt template + schema with one or more variants) has no single equivalent. The routing half maps to Routeplane, the templating/variant half stays in your app or moves to a [preset](/features/presets). See the [feature mapping](#feature-mapping) below.

### From the gateway + ClickHouse stack

The infra migration replaces the Compose stack (gateway container plus ClickHouse) with one binary and no database:

<Tabs items={['Before (TensorZero Compose)', 'After (Routeplane local)']}>
<Tab value="Before (TensorZero Compose)">
```bash
# docker-compose.yml runs the gateway + a ClickHouse warehouse
export OPENAI_API_KEY=sk-...
export TENSORZERO_CLICKHOUSE_URL=http://chuser:chpass@clickhouse:8123/tensorzero
docker compose up   # gateway on :3000, ClickHouse on :8123
# config mounted from ./config/tensorzero.toml

```bash # Routeplane is in private preview — request the binary at routeplane.app/get-access # (the steps below assume it's installed)

Set provider keys; Routeplane auto-detects on start

export OPENAI_API_KEY=sk-… export ANTHROPIC_API_KEY=sk-ant-…

Start the daemon

routeplane init # writes ./routeplane.yaml routeplane start # listens on http://127.0.0.1:4356

</Tab>
</Tabs>

## Feature mapping

| TensorZero concept | Routeplane equivalent | Docs |
|---|---|---|
| `[models.*]` + `[models.*.providers.*]` in `tensorzero.toml` | Provider keys (auto-detected) + the model registry | [BYOK](/features/byok), [Models](/concepts/models) |
| `[functions.*]` (named prompt + schema) | App-side, or a routing [preset](/features/presets) | [Presets](/features/presets) |
| `[functions.*.variants.*]` (per-variant model) | Routing preset variants / model ids | [Presets](/features/presets) |
| `routing` / `retries` / `fallbacks` | Model fallback rules | [Model fallback](/features/model-fallback) |
| `load_balancing` across providers | Provider selection | [Provider selection](/features/provider-selection) |
| OpenAI-compatible `/openai/v1` endpoint | OpenAI-compatible `/v1` endpoint | `POST /v1/chat/completions` on the local daemon |
| Native `POST /inference` endpoint | OpenAI-, Anthropic-, and Google-compatible protocols | see the protocol docs in this site |
| ClickHouse observability + UI | OTLP traces & metrics to your own backend | [OpenTelemetry](/features/opentelemetry) |
| OpenTelemetry (OTLP) export | OpenTelemetry (OTLP) export | [OpenTelemetry](/features/opentelemetry) |
| Embedded structured outputs (JSON functions) | Structured outputs across all providers | [Structured outputs](/features/structured-outputs) |
| — (no equivalent) | MCP / ACP / Skills agent gateways | [Tools](/concepts/tools), [Agents](/concepts/agents) |

## What Routeplane intentionally doesn't ship

To set expectations honestly: Routeplane is a gateway, not an LLMOps platform. It does **not** ship TensorZero's optimization and experimentation loop: no built-in supervised fine-tuning, RLHF, automated prompt engineering, dynamic in-context learning, inference-level or workflow evaluations, or adaptive A/B testing, and no ClickHouse-backed UI for replaying historical inferences against alternative prompts.

If your workflow depends on that closed feedback loop (collect inferences and feedback, evaluate, optimize, experiment) TensorZero is the better fit and you should keep it for those workloads. Routeplane is the right move when the value you need from TensorZero is the *gateway* and the missing half is *agent infrastructure*: tool/agent gateways, server tools, and presets.

## Migration checklist

<Callout type="info">
**Before migration**
- [ ] List the providers and models you actually route through TensorZero (skip the rest)
- [ ] Separate gateway use from platform use — are you using evals / optimization / experimentation, or just routing?
- [ ] Note where `[functions.*]` templates and schemas live; plan to keep them app-side or move them to a [preset](/features/presets)
- [ ] Decide which providers you'll supply via env vars (BYOK)
</Callout>

<Callout type="success">
**Migration**
- [ ] Install the Routeplane daemon ([Installation](/get-started/installation))
- [ ] Export provider keys — Routeplane auto-detects on start
- [ ] Update client `base_url` to `http://127.0.0.1:4356/v1`
- [ ] Replace `tensorzero::…` model strings with `provider/model` ids
- [ ] Re-point OTLP export at your backend ([OpenTelemetry](/features/opentelemetry))
- [ ] Verify with a sample request
- [ ] Decommission the gateway container and ClickHouse if you no longer need the platform loop
</Callout>

## Next steps

<Cards>
  <Card title="Quick Start" href="/get-started/quickstart" description="Run Routeplane locally in under a minute" />
  <Card title="Comparison" href="/get-started/comparison" description="Side-by-side with OpenRouter, LiteLLM, and generic gateways" />
  <Card title="Agent features" href="/concepts/tools" description="MCP, ACP, skills, server tools" />
</Cards>

## Corrections

Email `contact@routeplane.app` if this migration guide is wrong or stale.