Ultrafast inference
by agentic compilers.

RSI agent swarms that discover high-performance just-in-time engines. Verified model quality.

Try the platform Talk to us

01 / PERFORMANCE

Best-in-class speed.
Quality preserved.

Measured end to end on AgentX, a benchmark of production-style agentic coding workloads.

AMDMI350X

363266%tok/s/requestp50 AgentX352 tok/s/request at p90

GLM 5.3 · FP8 · 8× MI350X

NVIDIAB200

47835%tok/s/requestp50 AgentX387 tok/s/request at p90

GLM 5.3 · NVFP4 · 8× B200

Benchmark notes & comparisons

Both results: GLM 5.3 on AgentX, which runs agentic coding workloads end to end, at concurrency 1 on 8 GPUs (TP8). Prompts are synthetic, and multi-token prediction (MTP) uses a fixed synthetic acceptance rate, so results on real traffic may differ. Speeds are per-request interactivity in output tokens/sec/request, shown at p50 with p90 below. Gains are p90 against stock vLLM v0.31 on the same hardware.

AMD: FP8 on 8× MI350X. 363 tok/s/request at p50 and 352 at p90 (3.66× vLLM); p90 time to first token 2,095 ms (14% lower); total throughput 2,860 tok/s per chip (1.43×).

NVIDIA: NVFP4 on 8× B200. 478 tok/s/request at p50 and 387.1 at p90 (1.35× vLLM); p90 time to first token 631 ms (48% lower); total throughput 2,962 tok/s per chip (1.06×).

FP8 and NVFP4 use different precisions and are not a like-for-like comparison.

Explore AgentX

02 / THE AI STACK