apple-fm-bootstrap

Talk to Apple's on-device LLM — Apple Foundation Models — in ONE file. Zero tokens, zero download, $0.

The same model behind Apple Intelligence, running locally on the Neural Engine. No Ollama, no pip, no Hugging Face, no download, no network, no account. macOS 26+ (Tahoe), with Apple Intelligence enabled.

./apple-fm-bootstrap.sh "Explain a transformer in one sentence."
cat notes.txt | ./apple-fm-bootstrap.sh "Summarize this in 3 bullets:"   # instruction + piped body
echo "some text" | ./apple-fm-bootstrap.sh                 # stdin only
./apple-fm-bootstrap.sh --system "You are a pirate." "Hi"  # system instructions / persona
./apple-fm-bootstrap.sh --check                            # is the model available?
./apple-fm-bootstrap.sh --doctor                           # check the machine, build nothing
./apple-fm-bootstrap.sh --bootstrap                        # print the whole story

An arg is the instruction and piped stdin is the body — so you can pipe a whole document and tell it what to do, or use either alone. It's a Unix filter for the on-device model: drop it into any script.

What it lets you do

A local, free, offline text brain you can pipe anything into — verified on real macOS 27 hardware:

• Q&A / explain — apple-fm "Explain X in one sentence."
• Summarize — cat doc.txt | apple-fm "Summarize in 3 bullets:"
• Translate — apple-fm "Translate to Finnish, output only the translation: Good morning." → Hyvää huomenta.
• Rewrite / tone — apple-fm --system "Terse. Under 10 words." "Is the server healthy?" → Yes.
• Classify / extract / draft — any language task, in a shell pipe, with --system to steer it.

All on-device, zero tokens, offline, $0 — the Apple Intelligence model as a composable CLI.

Limits (honest)

The built-in model is small (~3B). It's strong at language tasks (summarize, translate, rewrite, classify, Q&A over text you give it) and weak at raw mental arithmetic and precise factual recall — e.g. it may miscompute a word problem. For grounded/agentic use (calculator, web, file, search tools), Apple's FoundationModels supports tool-calling; this bootstrap stays deliberately toolless and minimal so it builds with Command Line Tools and reads in one screen.

One self-contained shell script. It embeds a tiny Swift program, compiles it once with the system Swift toolchain, caches the binary, and runs it. Toolless on purpose (no @Generable macros) so it builds with plain Command Line Tools — full Xcode is NOT required.

Why this exists — what you actually GET

Your Mac already contains a capable LLM — the model behind Apple Intelligence, sitting on the Neural Engine. But Apple only lets you touch it through their apps (Siri, Writing Tools, Mail). apple-fm-bootstrap hands you that same model as a plain command you can pipe, script, and build on. Install one file and you get:

• A brain you own, not rent. No API key, no account, no subscription, no per-token bill, no rate limit. The compute is your Mac's; the cost is $0 — now and forever.
• Privacy by construction. Nothing leaves the machine — not a prompt, not a document, not a byte. It works on a plane, in a SCIF, offline. Your data is never someone else's training set.
• A composable primitive, not a chat app. It's a Unix filter: cat file | apple-fm "do X". It drops into shell scripts, cron jobs, Automator, other tools — AI as plumbing, next to grep and sed. You can put a language model in a pipe.
• Zero setup. One file. No Homebrew, no pip, no Docker, no model download — the model is already in macOS. First run compiles a tiny binary and you're done.
• Independence. For a huge class of work — summarize, translate, rewrite, classify, extract, draft — you stop depending on anyone's cloud. No vendor can meter it, read it, deprecate it, or take it away. The intelligence you already paid for (your Mac) does the work.

And because it's one self-explaining file (--bootstrap prints its own story), you can hand it to a friend, a collaborator, or another AI agent, and they instantly have the same capability — no infrastructure, no onboarding. You're not shipping a codebase, you're shipping the seed of a capability that grows wherever it lands. That's the whole idea: give someone a private, free, on-device brain in a single file.

Verified

Tested on real hardware (a macOS 27 machine): --check → available; a live prompt returned an on-device answer, compiled with Command Line Tools (/Library/Developer/CommandLineTools/usr/bin/swiftc). On older macOS it degrades gracefully — a clear "needs macOS 26" message and a non-zero exit, never a crash.

What it does, step by step (each checked)

1. Verify macOS 26+ by the presence of /System/Library/Frameworks/FoundationModels.framework. On older macOS it says so and stops (run it on a macOS 26 machine).
2. Verify a Swift toolchain (xcrun swiftc). If missing: xcode-select --install.
3. Compile the embedded Swift once (cached at ~/.cache/apple-fm-bootstrap/); rebuild only when the script changes.
4. Check SystemLanguageModel.default.availability and, if available, run LanguageModelSession().respond(to: prompt) — entirely on-device. Zero tokens.

Apple Foundation Models ≠ MLX

These are two different things, and it's a common mix-up:

	Apple Foundation Models (this repo)	MLX / mlx-lm
What	the built-in Apple Intelligence model	runs downloaded open models (Qwen, Llama…)
Download	none — it's part of macOS	~GBs per model from Hugging Face
Runtime	Neural Engine, Apple-native Swift framework	Apple GPU via the MLX array framework
Needs	macOS 26+, Apple Intelligence on	any Apple Silicon Mac + Python + mlx-lm

Want to run any open model locally via MLX instead? See the sibling: esaruoho/mlx-bootstrap.

Requirements

• A Mac with Apple Silicon on macOS 26 (Tahoe) or newer, Apple Intelligence enabled.
• Command Line Tools (xcode-select --install) — full Xcode not required.

Run ./apple-fm-bootstrap.sh --doctor to check your machine without building anything.

The pattern: a "bootstrap"

A single, self-contained, self-explaining file that carries a whole idea in a form another human or bot can run and understand from the file alone — idea transfer, not just code transfer. --bootstrap prints the whole story; the rest of the file is the story, executable.

License

MIT — share and improve freely. No warranty, be nice.