mlx-vision

DSpark-style faster OCR on Apple Silicon.

Speculative decoding for OCR vision-language models via MLX — the vision/OCR counterpart to mlx-dspark for text.

For AI agents: read AGENTS.md and llms.txt first.

Lossless when a neural drafter verifies through the target (same output as baseline at temperature 0). Includes a training-free semi-AR lookup path (DSpark-shaped: parallel draft + sequential refinement + confidence schedule) until OCR-domain DSpark checkpoints exist.

Supported models

Preset	Model	Notes
deepseek-ocr-2	mlx-community/DeepSeek-OCR-2-8bit	default
deepseek-ocr-2-bf16	mlx-community/DeepSeek-OCR-2-bf16	highest quality
glm-ocr	mlx-community/GLM-OCR-bf16
unlimited-ocr	baidu/Unlimited-OCR	R-SWA long-horizon; draft block clamped to window

pip install git+https://github.com/will702/mlx-vision.git
# PDF pages:
pip install "mlx-vision[pdf] @ git+https://github.com/will702/mlx-vision.git"

mlx-vision models
mlx-vision --model deepseek-ocr-2 --image doc.png --mode auto -v
mlx-vision --model deepseek-ocr-2 --image doc.pdf --page 0 --dpi 200

Modes

Mode	What it does
baseline	Plain mlx-vlm generate
lookup / dspark (no drafter)	Semi-AR n-gram draft + confidence schedule (training-free)
dflash / eagle3 / mtp	Neural speculation via mlx-vlm (--drafter required)
dspark + OCR DSpark weights	DeepSpec drafter via optional mlx-dspark, else semi-AR
auto	Prefer registered neural drafter; else lookup

# Baseline
mlx-vision -m deepseek-ocr-2 -i page.png --mode baseline

# DSpark-style (training-free until OCR drafters ship)
mlx-vision -m deepseek-ocr-2 -i page.png --mode dspark --max-draft auto

# Neural DFlash/Eagle3 when you have a matched OCR drafter
mlx-vision -m glm-ocr -i page.png --mode dflash --drafter /path/to/ocr-dflash

# Unlimited-OCR (R-SWA-safe)
mlx-vision -m unlimited-ocr -i book_page.png --mode auto --max-tokens 8192

Python API

from mlx_vision import ocr

result = ocr("document.png", model="deepseek-ocr-2", mode="auto")
print(result.text)
print(result.tokens_per_sec, result.mean_accept_len)

Server

pip install 'mlx-vision[server]'
mlx-vision serve --model deepseek-ocr-2 --port 8080

OpenAI-compatible POST /v1/chat/completions with image_url (data URL or path).

Benchmark

mlx-vision bench --model deepseek-ocr-2 --image page.png --modes baseline,lookup,dspark
# PDF:
mlx-vision bench --image doc.pdf --page 2 --modes baseline,dspark

Metric	Meaning
tok/s	Decode throughput
accept	Mean accepted draft length (telemetry / neural)
lossless	Temp-0 text match vs baseline

Vision cache

Repeated OCR of the same page hits a content-hash LRU (VisionFeatureCache) to cut repeat TTFT bookkeeping.

Train OCR DSpark drafters

No public OCR DSpark weights yet. See scripts/train_drafter/README.md for a DeepSpec-based recipe targeting DeepSeek-OCR-2 and GLM-OCR.

Architecture

image → vision encode (cached) → text decode
                                      ↑
                    draft (DSpark / DFlash / lookup)
                                      ↓
                         target verify (lossless)

Confidence scheduling shortens draft length when acceptance/confidence drops (DSpark paper idea, single-user adapted).

Citation

@software{mlx_vision_2026,
  title  = {mlx-vision: DSpark-style faster OCR on Apple Silicon},
  author = {Willson, Gregorius},
  year   = {2026},
  url    = {https://github.com/will702/mlx-vision}
}

License

MIT