AI companies train on your chats by default: what the Stanford audit found

Most people assume that a chat with an AI assistant is a private exchange between them and a machine. In 2025 a team at Stanford's Institute for Human-Centered AI (HAI) read the privacy policies of six large US developers to check whether that assumption holds. The short version: for consumer accounts, the default in the policies they reviewed was that your conversations can be used to train future models. This post summarizes what the audit reported, why it matters more for uploaded documents than for casual questions, and a five-step routine that keeps you in control regardless of what a provider changes next.

What the Stanford audit looked at

The study, led by Jennifer King at Stanford HAI, compared the publicly available privacy policies of Amazon, Anthropic, Google, Meta, Microsoft and OpenAI, focusing on how they describe the use of consumer chat data. The team published its conclusions in an explainer, Be careful what you tell your AI chatbot, and Stanford News covered it in a companion piece on chatbot privacy risks.

The findings, as reported by the authors at the time of writing:

• Training by default. All six developers' policies allowed user chat data to be used to improve or train models, and for consumer products this was the default rather than something you switch on.
• Opt-outs vary and are not always easy to find. Some products offered a toggle, some required a form, and the researchers noted that the controls were often buried in settings or in separate documents rather than presented up front.
• Long or unspecified retention. Several policies described keeping data for extended periods, or did not commit to a clear retention window, which means content may sit in provider systems long after the chat is over.
• Children's data. The authors flagged that some policies did not clearly rule out using data from minors, or relied on age declarations that are hard to enforce.
• Cross-product data flows. Where a developer runs many services (search, shopping, social media, email), the policies often allowed chat data to be combined with data from those services.

The authors were careful about scope: this was a review of what the documents say, not a technical audit of what each company's systems do. Policies change, and several providers have adjusted their settings since. Treat the specifics as a snapshot and check the provider's current controls for the plan you use.

Why documents raise the stakes

A one-line question ("how do I format a date in Excel?") carries little personal data. A pasted document is different. Contracts, CVs, medical letters, support exports and board decks are dense with names, addresses, identifiers and, often, other people's data. Once such a file is inside a chat that falls under a training-by-default policy, a few things follow:

The last row deserves emphasis. Files uploaded to a chat are frequently stored as separate objects (a "files" area, a project, a custom assistant's knowledge base). Deleting the conversation does not always delete the file. We cover that in Deleted the chat, but the file is still there.

If a document contains personal data of clients, employees or patients, you are usually the party responsible for that data under the GDPR or KVKK, whatever the provider's policy says. See GDPR, KVKK and LLM prompts for the basics.

Training toggles are necessary, not sufficient

Turning off training where a toggle exists is a good habit. It is not a complete answer, because:

• The document is still transmitted to and processed by a third party.
• Retention and human review policies apply independently of training.
• Toggles are per account and per product; a colleague on a different plan may have different defaults.
• Policies change, and the toggle you set today may not cover a feature launched next quarter.

The robust position is the one the Stanford authors also point toward: do not put identifying data into the chat in the first place unless you need to.

A five-step routine before you paste or upload

This is the routine we recommend for anyone who uses hosted assistants with real documents. It takes a few minutes and does not depend on which provider you use.

1. Anonymize before pasting

Replace names, companies, emails, phone numbers, addresses and identifiers with consistent labels ("Person 1", "Company A", email1@example.com) before the text leaves your device. Consistent labels keep the document readable for the model; black bars do not. Unformation does this in your browser for DOCX, PPTX, XLSX, PDF and text: the file never leaves your device, which you can verify with the browser's Network tab or with Wi‑Fi off. The flow is four steps: upload, choose a rule (labels, redaction, fake values, masks or hashes), review the list of findings, download.

2. Prefer business or API tiers where training is excluded

Business, team, enterprise and API plans typically come with contractual commitments not to train on your content and with defined retention. If your organization has such a plan, use it for work documents rather than a personal account. Check the provider's current terms for the exact commitments; do not assume.

3. Check the toggles on every account you use

Find the data controls for training, chat history and memory features, and set them the way you want. Repeat for each product and each account, and re-check after major product updates. If you cannot find a control, assume the default described in the audit.

4. Delete uploads, not just chats

After a task is done, look for a files or library area separate from the conversation list and delete the file there. Also delete it from projects, custom assistants and shared workspaces where it may have been attached.

5. Keep the mapping table locally

If you anonymized with consistent labels, the mapping table (which label stands for which real value) is what lets you restore the names in the answer later. Keep it on your device and never paste it into the same chat, or you have undone step 1. Unformation exports it as CSV or JSON, and the de-anonymizer applies it to an AI answer offline.

What this does not mean

None of this is a reason to stop using AI assistants. The Stanford authors' point is that consent for training should be informed and easy to withhold, and that consumer defaults do not currently deliver that. Until they do, the practical response is to control what you send. A document without identifiers can be used for training, retained, reviewed and combined with other data without exposing anyone.

Do it now

Open Unformation, drop the document you were about to upload, choose the label rule, review the findings and download the anonymized copy. Then upload that copy instead. For the format-specific pages, start with the DOCX anonymizer or paste text into the text anonymizer.