Fast local models,
on your Mac.

Three crossings, three tricks.

1. Reuse

   Read a shared prompt once

   Coding agents send the same system prompt and tool definitions with every session. Arf keeps the model's state at the end of that shared part, so the next session starts after it.

2. Guess

   Propose seven, check them in one pass

   A small draft model guesses the next seven tokens. The big model checks all of them in a single pass and keeps the ones it agrees with, so one pass can produce several tokens.

3. Share

   Many conversations, one pass over the weights

   Requests that arrive together decode together: each step reads the weights once for every active conversation, and a new request joins without waiting for the others to finish.

One command to a local coding agent.

arf launch claude starts the model in the background and opens Claude Code on it. The agent's instructions are read once and kept — on disk too, so a restarted server picks them up in seconds.

Will it run on my Mac?

Same rules as arf pull: it refuses weights over 90% of memory and warns past 75%.

Same Mac, same prompts, side by side.

Qwen3.8-27B at ~4 bits, one M4 Max (36 GB), SPEED-Bench coding prompts. Each engine on its default build, rounds interleaved, 2026-10-05.

	Arf	Splash 1.1.0	MLX 0.31.3
Decode, one stream (tok/s)	90.5	88.2	18.5
4 requests at once (tok/s)	99.3	99.6	61.2
Prefill, a 35K-token prompt (tok/s)	174	182	180
The same prompt again, first token	0.18 s	0.33 s	0.68 s

Splash is ~4% ahead on prefill and level at four at once. llama.cpp ran too, but every one of its rounds grew swap, so none is shown. Method: docs/PERFORMANCE.md.

The models it runs.

Text, pictures, video and sound in; text and pictures out — on the Mac's GPU, through the same OpenAI and Anthropic APIs.

• Images

  Reads a picture

  Qwen3.8's vision encoder runs on the GPU: a 512×384 image in ~0.3 s. Gemma 3 sees too.

  image_url · Anthropic image
• Video

  Watches a clip

  Frames are sampled with ffmpeg and read by the same encoder, at the rate you ask for.

  video_url · fps · max_frames
• Audio

  Listens

  Qwen3-Omni takes speech and sound as input, wav or mp3, and answers in text.

  input_audio
• Text to image

  Draws

  FLUX schnell turns a prompt into a picture, on the GPU, from the same server.

  /v1/images

Image and video answers decode without the draft. Speech output is not built yet.