Why we decided not to implement Apple Foundation models in FluentDB

Start with the part that should have been a warning. The Foundation Models documentation keeps a list of capabilities to avoid, and creating code and performing logical reasoning are both on it. Writing correct SQL against your schema is both at once.

So this was never a supported use case for the on-device model. We tested it anyway, because it is far and away the most common thing people ask us for, and because "not recommended" is not the same as "impossible". We wanted a real answer rather than a shrug.

An on-device session gets 4,096 tokens in total. That budget is shared by the instructions, the prompt, the response, and the schema of every tool we expose. It goes fast.

Fitting real work into it means splitting the job across several context windows running in parallel and stitching the results back together. That looks nothing like how every other provider works. Anthropic, OpenAI, and Ollama all take roughly the same shape in our code. Apple would need its own path, with its own bugs.

That is a lot of model-specific machinery to build and maintain for one model. We would do it for a model that earned it, which brings us to the part that decided this.

We gave it the best conditions we could. Small, carefully scoped context windows. Prompts that follow Apple's recommendations to the letter. This was never going to be a fair fight against a frontier model, and it was not meant to be. We only wanted to know whether it could hold a simple line.

It could not, often enough to matter. Double quotes dropped from PostgreSQL identifiers. Queries aimed at the wrong table entirely. Output that ignored the format our own components need in order to render it.

That last one is fixable with enough scaffolding. The middle one is not. A query that errors is a small annoyance you catch immediately. A query that runs against the wrong table and hands back a plausible number is the failure we build against, and it is the one you will not catch.

The obvious reply is to skip the on-device model entirely and use Private Cloud Compute, which macOS 27 opens up to third-party apps with a much larger window. We looked at it.

It means shipping through the App Store and enrolling in a specific program: the Small Business Program, under two million lifetime downloads across all your apps, metered per user per day, with no paid tier to move up to and six months to migrate off if you pass the cap.

It is also still in beta, and Apple gives very little visibility on where it goes from here. Putting a feature people pay for on top of a door that might close is not a trade we are willing to make.