Every morning my weather app writes me a short note. Today's said it was a mild evening ahead, zero percent chance of rain, a good night for a run. Nothing about that sentence sounds futuristic until you learn where it was written: on the phone, by Apple's Foundation Models, with no request leaving the device. I built the feature, I have lived with it for months, and I want to write down what that is actually like, because most on-device AI takes are written by people who have never shipped one.
The part that feels like the future
The briefing is fast, free, and private in a way that is not a promise but a physical fact. There is no API bill ticking, so the app can regenerate your briefing every refresh without me sweating a server invoice. There is no network round trip, so it works in a dead zone. And your location history, the most revealing data a weather app touches, stays in your pocket. When people ask what on-device AI is for, this is the answer I give: features you could not afford to run in the cloud, attached to data you should not send there.
The part where the small model needs a chaperone
Now the honest column. Small local models are gullible about numbers. Early versions of my briefing would cheerfully call 57 degrees warm, or forget the rain jacket on a sixty percent day. The fix was humbling and very unglamorous: stop trusting the model with arithmetic. The app now computes the facts in regular code, the coldest stop on your trip, the rainy days, the UV peak, and hands the model a list of ground truths it is not allowed to contradict. The AI is the writer, never the analyst. Every on-device feature I ship now follows that rule, and it was a production bug this very week, a packing list echoing template markup, that reminded me the chaperone work is never quite finished.
Why I still choose the small model
A frontier cloud model would write prettier sentences than the one in your iPhone. It would also mean your mornings route through a data center, that the feature dies without signal, and that I would have to either charge for it or harvest something to pay for it. For a daily ritual built on your location, that trade is upside down. The local model writes a B-plus paragraph with an A-plus privacy story, and for a note you glance at for eight seconds while the coffee brews, that is the right grade to optimize.
The future of AI on phones, at least the part I am betting a studio on, is not a chatbot that knows everything. It is a dozen small, supervised writers living inside ordinary features, each handed clean facts and a narrow job, none of them ever phoning home. My mornings are already there, and it is quietly great.
— JC Mobile App Studio