MLX Inference Engine Benchmarks

Throughput — Tokens per Second

Higher is better. Sustained generation speed, 5-trial mean ± SD.

rapid-mlx

58.8 ±2.2

omlx

50.5 ±0.4

mlx-lm

49.6 ±2.3

ollama

45.4 ±3.2

0102030405060

rapid-mlx leads by ~19% over the next two engines. omlx and mlx-lm are close in mean speed, but their standard deviations tell different stories: omlx is the most consistent engine tested (±0.4 tok/s), while mlx-lm (±2.3) and ollama (±3.2) vary considerably more run-to-run. For server workloads where predictable throughput matters more than peak throughput, omlx's consistency is a meaningful advantage.

Time to First Token — Cold Start

Lower is better. Latency to first output token with no warm cache.

rapid-mlx

0.246s ±0.012

mlx-lm

0.318s ±0.009

omlx

0.436s ±0.030

ollama

0.519s ±0.030

0s0.1s0.2s0.3s0.4s0.5s0.6s

rapid-mlx responds in 246 ms. Ollama takes more than twice as long. For interactive use that difference registers on every exchange. For agentic pipelines making dozens of sequential calls, it accumulates.

Time to First Token — Cached

Lower is better. After two warmup calls priming each engine's prefix cache.

mlx-lm

0.191s ±0.005

rapid-mlx

0.192s ±0.023

omlx

0.422s ±0.014

ollama

0.433s ±0.010

0s0.1s0.2s0.3s0.4s0.5s

rapid-mlx and mlx-lm both drop to ~0.19s cached — a genuine improvement from cold, indicating effective prefix caching. omlx and ollama show almost no improvement (0.436→0.422s and 0.519→0.433s respectively).

The omlx result warrants context. omlx is designed as a multi-model production server with LRU memory management and paged SSD caching — its caching strategy targets concurrent long-running sessions, not the single-request repeated-prompt pattern mlx-chronos uses to measure this metric. These numbers reflect a mismatch between benchmark design and engine design, not a caching defect.

Memory — Two Measurements That Mean Different Things

mlx-chronos records Process RSS and System RAM peak separately. On Apple Silicon they diverge significantly.

Process RSS — GB (lower is better)

Memory claimed by the engine process itself.

ollama

44.6 GB

mlx-lm

3.70 GB

omlx

3.55 GB

rapid-mlx

3.48 GB

01020304050 GB

The three MLX-native engines each use ~3.5 GB of process memory. Ollama uses 44.6 GB — roughly 12× more — for the same size model. Ollama loads model weights into CPU-addressable unified memory, which appears in full in process RSS.

System RAM Peak — GB (lower is better)

Total system memory pressure including Metal/GPU wired memory.

ollama

57.0 GB 88.9%

omlx

56.9 GB 88.9%

rapid-mlx

37.7 GB 59.0%

mlx-lm

37.5 GB 58.6%

0102030405060 GB

omlx and ollama show identical system RAM pressure (88.9%) but arrive there differently. Ollama's 57 GB comes from its process RSS — it sits in CPU memory. omlx's 57 GB comes from Metal wired memory pre-allocated at startup: the server raised the Metal wired memory limit to 56 GB on launch, reserving GPU headroom independent of which model is loaded. Its process RSS stays at 3.55 GB throughout.

rapid-mlx and mlx-lm peak at 37.5–37.7 GB (58–59%), leaving roughly 40% of system memory available. Running alongside other applications is realistic on a 64 GB machine.

What This Means in Practice

rapid-mlx

Leads on every metric relevant to single-user interactive use: throughput, cold TTFT, cached TTFT, and system memory headroom. Best choice for chat, coding assistance, and agentic pipelines.

mlx-lm

Close to rapid-mlx on throughput and nearly identical on cached TTFT (0.191s vs 0.192s). Apple's reference MLX implementation — tends to support new model architectures before other engines. Reasonable default when broad model compatibility is the priority.

omlx

Poorly characterized by this benchmark's single-request design. Its standout qualities — consistent throughput, paged SSD caching, multi-model LRU management, concurrent request handling — don't show up in a 5-trial sequential test. Evaluate on a workload that reflects actual multi-client use.

ollama

Most accessible entry point with the largest model ecosystem. The process RSS numbers are the significant finding: 44.6 GB for a 4B 4-bit model. On 16 GB or 32 GB Macs, loading a model of this size leaves almost no headroom for the rest of the system.

Recommendations

Use Case	Engine
Interactive chat, coding, agentic workflows	rapid-mlx
Multi-client local API server, batch workloads	omlx
Broadest model support, new architectures	mlx-lm
Simplest setup, largest model library	ollama
16 GB or 32 GB Mac · models above ~3B	ollama — avoid