The health AI gold rush: the one question worth asking first.

Five products in three months

The pace has been remarkable. Between January and March, OpenAI launched ChatGPT Health, Anthropic launched Claude for Healthcare, Amazon's Health AI opened to One Medical members, Microsoft shipped Copilot Health, and Perplexity unveiled Perplexity Health. Most promise not to train on your health data and to keep health chats separate from ordinary ones, and most connect to your records and your wearables, Apple Health, Fitbit, Oura, and the like. (IAPP)

That is genuinely useful. An assistant that can make sense of a lab result in plain English, or notice a pattern across months of data, can help you show up to an appointment better prepared. The tools are not the problem. The question is what happens to the most sensitive data you own once you plug it in.

The one question

Ask where your data lives. When the AI reads your labs or your heart rate, does the work happen on your device, or does your medical information travel to a company's servers to be processed? "We do not train on it" is a good promise, but it is not the same as "it never leaves your phone." Storing something securely on a server is still storing it on a server, which is a bigger surface for a breach, a subpoena, or a future change of terms than a file that only ever existed on your device.

This is exactly why researchers are pushing so hard on on-device and zero-egress approaches for sensitive contexts, running the model locally so patient data never has to leave the machine. When that is possible, it removes whole categories of risk instead of promising to manage them. (IEEE)

What the rules are starting to require

Regulators are catching up. Newer laws, like the Texas AI rules and a growing set of state measures, are moving toward explicit consent for health data use and clear, nutrition-label style disclosures about how a model reaches its conclusions. That is a good direction, but it puts the burden on you to actually read the label. The label only helps if you look at it.

A short checklist

Before you connect a health AI to your records, three quick checks help. Does it say plainly whether processing is on-device or in the cloud, and if cloud, is the data encrypted and kept out of training? Can you delete your data and confirm it is gone, not just hidden? And do you actually need the integration, or would a manual, occasional check give you most of the benefit with none of the exposure? None of this is a reason to avoid these tools. It is a reason to choose the version that treats your health data like it is yours, because it is.

We build on-device because health, money, and privacy are the places where "nothing leaves your phone" stops being a slogan and starts being the whole point.

For the basics, see What on-device AI means and Encryption explained. This is general information, not medical advice, always talk to a clinician about your own care. You can see what this studio builds at jcmobileappstudio.com.