Foundation Models was announced at WWDC in June 2025 and opened to developers the same day. Public release came with iOS 26 in September. Three lines of Swift, zero configuration, zero cost per call. Sounds like marketing, so I spent a few weeks checking what that actually means in practice.
Short answer: it works. Long answer below.
What it actually is
Foundation Models is a framework that gives developers access to the same language model powering Apple Intelligence. It runs locally, on the device. No data sent to the cloud, no API key, no charge per call.
The model has around 3 billion parameters. GPT-5.5, Claude Opus 4.7, Kimi K2.6 - the current frontier - run in the cloud on compute you cannot fit in a phone. Different class entirely - and that’s fine, because that’s not the point. This model isn’t meant to replace ChatGPT. It’s meant to do specific things fast, privately, and without a bill at the end of the month.
Available on iOS 26, iPadOS 26 and macOS 26, on any device with Apple Intelligence enabled. In practice: iPhone 15 Pro or newer, any Mac with M1 or newer.
Where it shines
The biggest surprise is guided generation. Instead of asking the model for JSON (a structured data format apps use to talk to each other) and hoping it parses correctly, you declare a Swift type with the @Generable macro and get back a ready object. No regex. No validation. No “the model returned something that looks like JSON but isn’t quite.”
@Generable
struct Insight {
let title: String
let body: String
}
let result = try await session.respond(
to: prompt,
generating: Insight.self
)This is a fundamental shift from how most developers use LLMs today. Prompt engineering stops being a fight over output format - you can focus on what the model should say, not how.
The model is also surprisingly good at short, structured tasks: classification, extraction, summarization, generating concise text (don’t remind me about the Apple Intelligence notifications summaries though 😅). The first word appears in half a millisecond on iPhone 15 Pro - but only after calling session.prewarm(), which preloads the model into memory before the user asks their first question. Without it, the first call is noticeably slower. The kind of detail Apple buries halfway down a documentation page.
In practice: the user opens a screen, the app calls prewarm() in the background, and by the time they tap “generate” the model is already ready. Half a millisecond instead of a few seconds of waiting.
Where the real work begins
LanguageModelSession (a session - the object that manages your conversation with the model) is stateful, meaning it remembers history. Every respond(to:) call gets recorded in the transcript - a log of the entire conversation. The transcript grows, the model has to process it before every response, and once it hits the limit the framework throws GenerationError.exceededContextWindowSize. Not a warning - a hard error the app must handle. In production, where a session lives for a while, this is something you need to anticipate.
Second point: model.availability. The framework gives you a boolean isAvailable as a shortcut (yes/no), but behind it sits an enum (a list of cases) with three variants - Apple Intelligence disabled, device not supported, model still downloading. Each case means different app behavior and a different message for the user. If you only check yes/no, you lose that information - and instead of writing “model is still downloading, try in a moment,” you show the user a blank space with no explanation.
The framework won’t complain if you forget to check availability. It’ll silently do nothing. To the user, that looks identical to “AI is slow.” That’s why availability is the first line of every call, not optional.
Timeout is your responsibility
Foundation Models has no built-in time-based timeout. There’s a maximumResponseTokens option in GenerationOptions that caps response length - which indirectly limits generation time - but no way to say “stop after N seconds.” The model generates for as long as it generates. On a fast device with a simple prompt - half a second. On a slower one, with a long context, under system load - longer, with no guarantee.
If you don’t want the UI to freeze indefinitely, you write the timeout yourself. withThrowingTaskGroup with Task.sleep as a race - two tasks running in parallel: one generates the response, the other counts down. First one to finish wins. If the clock wins, the user gets a fallback instead of an endless spinner.
Apple doesn’t document this directly. I found out when I had to.
Fallback is half the architecture
Before I wrote my first model call, I wrote the fallbacks (alternative responses for when the model fails). The only sensible order of operations.
The model can be unavailable, can be slow, can return something unexpected - guided generation guarantees structure, not content. For each of those cases, the app needs to know what to show. A static string, a locally computed value, a previously saved result. Anything, just not a blank screen.
The best pattern I found: template-first. Show a static, sensible response immediately. Run the model in the background. If it comes back in time - swap it in smoothly. If it doesn’t - the user already saw something useful.
Concretely: the user opens a screen and sees their weekly summary with yesterday’s data. The model generates a fresh version in the background. If it finishes within four seconds, the content refreshes. If not, the user still saw a real answer, not a spinner. That’s the inversion of the typical “wait for AI, then show” approach.
What’s missing - and what I’d like to see
Foundation Models are good. A few things are missing enough that I’ll name them.
First: streaming to @Generable. Today you can either stream text token by token, or generate a Swift structure - but not both at once. I’d want to see an object fill in field by field in real time. Technically this isn’t a new problem - guided generation and streaming both work independently, combining them is an API decision, not an architectural one. Without it, you’re forced to compromise: either the user waits for a complete response, or they get raw text you have to parse yourself.
Second: inference metrics. You have no access to the token count in a session. Apple doesn’t publish a tokenizer. The only way to know how close you are to the context limit is your own heuristic - counting characters, guessing. In practice, exceededContextWindowSize is always a surprise, not something you can prepare for in advance.
Third: cross-session personalization. The model doesn’t learn the user. To give it context - habits, preferences, history - you manually manage that context and inject it into every new session. RAG (searching local data before generating a response) would be a natural next step, especially since Apple already has access to notes, mail, calendar, and health data. The infrastructure is there. Developers don’t have access to it.
Fourth: on-device fine-tuning. Apple opened up fine-tuning through LoRA adapters (small modules you “layer” onto the base model to adjust its behavior), but training one requires a Mac with at least 32-64 GB RAM or a GPU server - and you have to retrain it every time Apple updates the base model. That last part is the real friction: not a one-time investment, but ongoing maintenance tied to Apple’s release cycle. If Apple simplified this into a proper API with automatic adapter migration, it would actually be usable at scale.
My wishes and best guesses for 2027
iPhone 18 Pro is expected to get the A20 Pro chip in September 2026, built on a 2nm process with a new memory packaging approach (WMCM - RAM physically integrated closer to the CPU, GPU and Neural Engine instead of as a separate chip). Result: lower inference latency and less energy per AI task. Add to this 12 GB RAM standardized across the Pro line - more headroom for larger models. At WWDC 2026, Apple is expected to announce Core AI as a replacement for Core ML, Foundation Models expanded with third-party model support via MCP, and the Visual Intelligence API opened to developers.
A20 Pro with 12 GB RAM and the new memory architecture should let Apple run a model significantly larger than the current 3 billion parameters on-device - realistically 7-13 billion - without degrading response time. A model that reasons rather than just classifies. Combine that with the Visual Intelligence API opening, and Foundation Models stops being a text framework and becomes a multimodal one - text, image, camera context, all local.
RAG built on user data is my biggest wish and also the most realistic next step. Apple has the data, has the silicon, has the private infrastructure. The only missing piece is a developer API. If WWDC 2026 opens a Personal Context API (currently in rumors as “medium confidence”), by 2027 we could be writing apps that genuinely know the user - without sending anything to the cloud.
On-device fine-tuning on a Mac with M5 is a wish. Technically feasible, commercially risky for Apple - a model tuned to a specific user is harder to moderate. But if Apple wants to beat OpenAI on personalization, this is the only path that doesn’t require giving up privacy.
WWDC 2026 is June 8. In a month, we’ll know how much of this was right.
What this means for developers
Foundation Models changes the economics of AI in apps. If you’ve been holding off on adding AI features because you didn’t want to pay for API calls or send user data to external servers - that barrier is gone.
“Free and local” doesn’t mean “no cost.” The cost moved somewhere else - onto you. In the cloud model, you pay per token and get infrastructure included. Here the infrastructure lives in the user’s device, which they already paid for. But fallbacks, timeouts, transcript management - that’s your job, not the framework’s.
Foundation Models is production-ready. As long as you are too.
It’s also worth watching where this is heading. The A18 Pro in iPhone 16 Pro added dedicated ML accelerators built directly into the CPU - AI tasks can be handled without engaging the Neural Engine, which means lower battery drain for frequent calls. A19 Pro goes further, adding Neural Accelerators to every GPU core. In practice: a model that today requires an iPhone 15 Pro will run faster and more efficiently on a standard iPhone next year.
If WWDC 2026 opens the Visual Intelligence API as rumored, Foundation Models stops being purely a text framework. Let’s wait and see if this happens.