Two independent axes — pick each separately
A deployment is defined by two orthogonal choices. Conflating them is the most common sizing mistake:Availability tier (T0 → T4)
The failure domain you survive — node → zone → region. This is what the
tiers below set, and it drives your SLA. It changes topology, not capacity.
Concurrency (sizing)
How many users/requests you serve. This sizes CPU, memory, disk, replicas,
and shards — within any tier. The platform and inference layers scale
independently (see Hardware Requirements).
The per-tier user counts below are typical pairings, not caps. 10,000 users
doesn’t force T4 — you go to T4 to survive a region outage, not to hold more
users. Need 10,000 users but single-region resilience is acceptable? Run a T2/T3
topology sized for 10,000. Pick the tier for failure domains; size for concurrency
separately.
The tier ladder
| Tier | Survives | Target SLA | Typical concurrency | Status |
|---|---|---|---|---|
| T0 Single-Node | pod restart | ~99% | ~100 | dev / PoC only — no backup |
| T1 Essential | disk loss | ~99.5% | ~100 | available |
| T2 Standard HA | any node | 99.9% | ~1,000 | available (chart default) |
| T3 Multi-AZ | zone loss | 99.95–99.99% | few thousand | aspirational (multi-AZ is an IaC change) |
| T4 Multi-Region | region loss | 99.99%+ | 10,000+ | aspirational (second region is operator-built) |
Per-tier reference architectures
- T0 · Single-Node
- T1 · Essential
- T2 · Standard HA
- T3 · Multi-AZ
- T4 · Multi-Region
AI-in-a-Box — dev / PoC only. Every component on one node. No HA and no
backup by design — data is disposable (RPO unbounded). Use only where outage
and potential data loss are acceptable.
| Resource | Bare minimum |
|---|---|
| Nodes | 1 |
| CPU / memory | 16–32 cores / 96–128 GiB |
| Node NVMe | 1–2 TiB (holds every DB + the object store) |
| Backup | none |
| Install | -f values.minimal.yaml on every dependency (1× each) |
Sizing the storage — the bare-minimum volume math
The dominant, variable storage driver is observability traces, and it scales with throughput × retention, not the tier. Total storage is the sum of four components — don’t size on traces alone:| Component | Scales with | Rough size |
|---|---|---|
| Platform baseline (queue, object-store metadata, working set) | ~fixed | ~200–500 GiB |
| Database working set (PG · ClickHouse hot · Kafka · Mongo · Valkey) | throughput | ~1.5–3 TiB (node-local) |
| Analytics / observability traces — the dominant term | req/s × spans × retention × 3 | see below |
| Model registry | model catalog, not the tier | 50 GiB → 2–5 TiB+ |
From concurrency to terabytes
Concurrency → req/s
req/s = concurrency ÷ avg request seconds. An agent request runs ~15 s
(multi-step), so 1,000 concurrent users ≈ 67 req/s. Use your own measured
request duration if it differs.| Concurrency | ≈ req/s | Trace volume (×3) |
|---|---|---|
| ~100 | ~7 | ~1 TB |
| ~1,000 | ~67 | ~12 TB |
| ~3,000 | ~200 | ~36 TB |
| ~10,000 | ~667 | ~120 TB |
Sized for the agent workload (~15 s/request, ~15 spans). The two levers that move
these numbers most are the retention window (linear) and spans per request
(fewer spans → proportionally less) — see the knobs below. Plug your own measured
request duration and span count into the formula for an exact figure.
What the “external object store” actually is
Bud reaches object storage through a plain S3 API (externalServices.s3:
endpoint + keys + bucket). By default it’s SeaweedFS, in-cluster — running on the
same nodes’ NVMe, so there is no external store until you add one. The “external”
store is simply a separate S3-compatible object store you stand up and operate
yourself, then point that config at:
- Self-hosted MinIO (most common on-prem) — a small cluster of cheap-disk nodes (HDD / standard-SSD, erasure-coded ≈ 1.5× raw). Bud ships an optional in-cluster MinIO toggle, or you run it separately.
- Ceph / RADOS-GW — if you already run Ceph, its object gateway works too.
- Cloud bucket (AWS S3 / Azure Blob) — only where the deployment isn’t air-gapped.
Scaling up or down — the knobs
| Lever | Effect | How to turn it |
|---|---|---|
| Retention window | Linear — 30-day is ⅓ of 90-day | The biggest lever. Shorten trace retention (default 30-day traces / 90-day metadata). |
| Request duration | Inverse — shorter requests = more req/s = more traces | Set by your workload; measure it. The baseline assumes ~15 s agent requests. |
| Spans / request | Linear — fewer spans, proportionally less storage | Enable span sampling; keep only the orchestration spans you need. |
| Per-node NVMe | nodes = volume ÷ per-node disk | Stack more NVMe into vg1 (8 / 16 / 24 TB) before adding nodes. |
| Model catalog | Sets registry size independently | Prune stale variants; size the registry volume to your model list. |
| Content logging | +~2–3 KB per content-bearing span | Off by default. If on, bound it with a short retention window. |
| CPU / concurrency | CPU is the binding resource for agent/event load | Size vCPU to peak req/s; add nodes/HPA for throughput (memory rarely the limit). |
Through T3 you stay at the 3-node HA floor — you scale by fitting bigger NVMe
into each node’s
vg1 pool, not by adding nodes. Only at T4 does trace volume
start to drive node count. And even at T4 you can stay all-NVMe (~5–6 big-NVMe
nodes) — you tier cold traces to external object storage to save cost, not because
it won’t fit.Next steps
Hardware Requirements
The underlying compute, memory, storage, and inference capacity model
Helm Configuration
Configure resources, retention, replicas, and the S3 endpoint