AI Chat Assistant Privacy Risks: Training, Review, Retention

Consumer AI chat assistants carry three privacy risks by default: your chats can train future models (the default at ChatGPT, Gemini, Copilot, Meta AI, Grok, Perplexity and Mistral), people may read a sample, and retention can run for years: up to five at Claude if training is on, three for human-reviewed Gemini chats.

Three more exposures sit outside any training toggle: shared links that search engines indexed, memory that outlives the chat, and court orders that override deletion. Business, enterprise and API tiers change most of these defaults by contract. The tables below are compiled mainly from providers’ own help and privacy pages as of September 2026; those pages change often, so re-check any you cite in a policy.

Training defaults and opt-out paths

Seven of the eight assistants below train on consumer chats by default. Anthropic asks Claude users to choose, but TechCrunch reported that its consent pop-up paired a prominent Accept button with a smaller training toggle “automatically set to ‘On’”.

• ChatGPT: OpenAI’s data controls page also accepts a Privacy Portal request, “Do not train on my content”.
• Claude: switching off stops new chats and coding sessions being used, but Anthropic says data stays in training already started and in models already trained. Its August 2025 terms update gave existing users until 8 October 2025 to choose.
• Gemini: Google’s activity page confirms the default; switching off still leaves 72 hours of retention.
• Copilot: Microsoft’s controls page puts the same toggles under Settings > Privacy on Windows and macOS, and Account > Privacy on mobile. Its privacy FAQ exempts organisational Entra ID accounts, Microsoft 365 Personal and Family subscribers, signed-out users, under-18s, and users in Brazil, China (excluding Hong Kong), Israel, Nigeria, South Korea and Vietnam.
• Meta AI: Meta’s April 2025 EU announcement says interactions with Meta AI “will also be used to train and improve our models”. Its privacy explainer says typing “/reset-ai” in a conversation deletes your AI messages.
• Grok: xAI’s consumer FAQ, now published under the SpaceXAI name, says logged-out users outside the EU and UK cannot opt out.
• Vibe: Mistral says Le Chat has become Vibe. Its opt-out article warns that the Vibe toggle and the API toggle (“Anonymous improvement data”) are separate, and that Team admins can switch training off organisation-wide.

Retention by mode and tier

“Not stated” means the provider pages cited give no figure. The long tails are Anthropic’s five years for de-identified training data and Google’s three years for reviewed Gemini chats; the latter survives deleting your activity. Temporary modes shorten retention rather than end it: ChatGPT keeps a copy for up to 30 days, Claude for 30 days, and Perplexity’s incognito threads last 24 hours. ZDR is a contract term that can move: since 9 June 2026 Anthropic keeps prompts and outputs on its “covered models” for 30 days even for organisations with ZDR agreements. Mistral’s Temporary Chat logs only that a chat happened, and its ZDR option excludes Vibe. Microsoft treats work-account Copilot use as processing under its data protection addendum.

What switching training off leaves exposed

• Feedback. OpenAI says rating a response makes “the entire conversation associated with that feedback” eligible for training. Anthropic keeps rated conversations for up to five years, and Team and Enterprise owners can block ratings with the Rate chats setting. xAI says feedback may be used for training after an opt-out, and Mistral says a rating with a comment authorises use of your input and output on any plan.
• Safety-flagged chats. Anthropic says flagged conversations may still improve its trust and safety models, and Microsoft reviews chats flagged as Code of Conduct violations. OpenAI routes conversations in which users plan to harm others to “a small team trained on our usage policies” and may refer “an imminent threat of serious physical harm to others” to law enforcement; it does not refer self-harm cases.
• Training that already happened. Anthropic leaves opted-out data in training runs already under way, Perplexity says opt-outs apply only to data collected afterwards, and Microsoft’s opt-out covers future conversations only.
• Advertising. From 16 December 2025 Meta uses Meta AI chats to personalise content and ads, though it says topics such as health and sexual orientation are not used to show ads; TechCrunch reports there is no opt-out. Microsoft uses Copilot history for ads when both ad and Copilot personalisation are on.

Three risks the settings pages leave out

Research has shown that models can reproduce fragments of their training data, the tail risk of training covered in our explainer on membership inference attacks. The three risks below are newer and better documented.

A court order kept deleted ChatGPT chats

On 13 May 2025 Magistrate Judge Ona T. Wang ordered OpenAI, in the copyright litigation brought by The New York Times and other publishers, to “preserve and segregate all output log data that would otherwise be deleted on a going forward basis”, including chats users had asked to delete (order). OpenAI said this reached ChatGPT Free, Plus, Pro and Team and API customers without ZDR, but not Enterprise, Edu or ZDR traffic, and that only “a small, audited OpenAI legal and security team” could access the data. An order of 9 October 2025 ended the duty as of 26 September 2025, but OpenAI keeps what it had segregated (minus requests from the EEA, Switzerland and the UK) and still preserves logs for accounts tied to domains the news plaintiffs identified. In January 2026 District Judge Sidney Stein affirmed orders to produce 20 million de-identified ChatGPT logs.

A deletion schedule is a policy, not a guarantee: OpenAI’s retention page already excepts chats it “must retain longer for security or legal obligations”. Nor was a business tier enough by itself: Team was covered, Enterprise was not.

Shared links became search results

ChatGPT’s share dialog once offered “Make this chat discoverable”. OpenAI’s CISO, Dane Stuckey, said it “introduced too many opportunities for folks to accidentally share things they didn’t intend to” and removed it, yet The Register still found chatgpt.com/share results on Bing, DuckDuckGo and Brave after Google dropped them. OpenAI’s shared-links page now says these pages “are not intended for search-engine indexing, but this does not make a link private.”

Grok’s share links were indexed by Google, Bing and DuckDuckGo (TechCrunch); Vice, citing Forbes, counted more than 370,000 conversations published “without warning or a disclaimer to the user”. In June 2025 TechCrunch found Meta AI app users posting chats to a public Discover feed, and noted that logging in with a public Instagram account made a user’s searches public too.

Memory outlives the chat

Memory stores details across conversations, apart from the chat list. In ChatGPT (Settings > Personalization > Memory), deleting a chat does not delete a saved memory drawn from it, logs of deleted memories stay for up to 30 days, and with training on OpenAI may use memories to improve its models (OpenAI). Claude’s memory is on by default for Free, Pro and Max, and memory entries generated from a chat survive the chat’s deletion (Anthropic). Gemini personalises “by referencing your chats in Gemini Apps Activity” (Google), and Copilot’s “Personalization and memory” switch sits apart from its training toggles (Microsoft). Deleting a chat about a client can leave the client in memory.

Worked example: one client prompt through three tiers

A payroll consultant pastes: “Draft a letter to the tax office for [client name], born [date], tax reference [number], explaining that her return was late because she was in hospital for surgery.” That is direct identifiers plus health data, a special category under GDPR Article 9. OpenAI documents all three tiers (enterprise privacy, platform data controls), so it makes the trace, with default settings:

The same prompt elsewhere:

• Claude Pro, training on: the chat can be retained in de-identified form for up to five years. Via the API, Anthropic deletes inputs and outputs within 30 days unless a ZDR agreement or a Usage Policy flag applies.
• Gemini, personal account: auto-deleted after 18 months by default, but a reviewer-sampled copy stays up to three years. On a qualifying Workspace account it is not human-reviewed or used for training outside your domain without permission.
• Free Gemini API, used for a quick test: an “Unpaid Service” where human reviewers may read inputs and outputs, and Google’s terms say: “Do not submit sensitive, confidential, or personal information to the Unpaid Services.”

Only the enterprise and API columns give you processor terms, no training by default and deletion you control. Swapping the name, birth date and tax reference for an internal reference shrinks the exposure in every column: a pseudonymised prompt is still personal data, but far less of it.

Regulators: named actions and where they stand in 2026

• Italy v. OpenAI, a €15 million fine annulled. On 20 December 2024 the Garante fined OpenAI €15 million and ordered a six-month information campaign, citing an unreported March 2023 breach, training without first identifying an adequate legal basis, transparency failures and no age verification. On 18 March 2026 the Rome Tribunal annulled the decision: OpenAI had set up its EU establishment in Ireland while the case was pending, so the one-stop shop made the Irish DPC competent. The court did not examine the alleged violations (Iusletter), and the Garante’s page now notes the decision’s temporary removal after judgment 4153/2026.
• DeepSeek, processing limits and app-store notices. On 30 January 2025 the Garante ordered DeepSeek’s operators to limit processing of Italian users’ data after they argued European law did not apply to them. Berlin’s commissioner asked DeepSeek on 6 May 2025 to comply or withdraw its apps; on 27 June 2025 she notified Apple and Google under Article 16 of the Digital Services Act that the app is illegal content, because its transfers to China breach Article 46(1) GDPR. The EDPB extended its ChatGPT task force to AI enforcement in February 2025, and the IAPP counted investigations in 13 European jurisdictions.
• Ireland and Meta AI, supervised rather than decided. Meta paused EU training in June 2024 after the DPC raised concerns. After the EDPB’s December 2024 opinion, it began training on EU adults’ public content and Meta AI interactions from 27 May 2025, with new notices, an easier objection form and revised impact assessments. The DPC required a report on those safeguards in October 2025 and said it would keep assessing compliance.
• Ireland and Grok, an open inquiry. On 11 April 2025 the DPC opened an inquiry into whether EU and EEA users’ public posts on X were lawfully used to train Grok.

Where things stand in 2026. The headline fine fell on jurisdiction, not merits, so ChatGPT’s EU processing now answers to Ireland, which already supervises Meta and X. On 25 September 2026 the DPC published an AI Insights Report covering about 180 AI products it engaged with from 2021 to 2025, naming Meta, OpenAI, DeepSeek and X and flagging legitimate interests for AI training and transparency as focus areas. No regulator has settled the training-default question for you; your contracts and settings are what you control today.

The legal frame in brief

Under GDPR, training on chats is a separate purpose from answering them and needs its own Article 6 basis. An opt-out default cannot be consent, so providers lean on legitimate interest, which the EDPB’s December 2024 opinion accepts in principle, citing “a conversational agent to assist users” as an example, subject to a balancing test that weighs users’ reasonable expectations. A Stanford HAI review of six US developers found all six train on chat data by default and called for opt-in. Erasure once data reaches a training set is covered in training-data privacy and data-subject rights. In the US, chat content is personal information under the CCPA and similar laws, and California’s ADMT rules attach where output drives a significant decision: see CCPA, CPRA and the ADMT rules for LLM products and, for EU automated decisions, GDPR Article 22 and LLM automated decision-making.

Decision table: which data may go into which tier

Article 28 requires processing by a processor to be “governed by a contract or other legal act”, which a consumer account does not give you, so that column stops at personal data. Article 35 requires an assessment before processing “likely to result in a high risk” and names large-scale processing of special categories as one case. The DPIA template for LLM deployment gives the structure; cross-border LLM data transfers covers the transfer mechanism.

Audit procedure for your team’s accounts

Record one row per person per assistant:

1. Account type: personal, or work (Entra ID, Workspace, enterprise workspace, API key).
2. Training control: open the path in the first table; record on or off and the date.
3. Memory: ChatGPT Settings > Personalization > Memory; Claude Settings > Memory; Copilot Memory > “Personalization and memory”. Record the state; delete stored memories that hold client or health details.
4. Shared links: ChatGPT Settings > Data controls > Shared links > Manage; Grok at grok.com/share-links. Record the count; delete links carrying personal data.
5. Search exposure: search site:chatgpt.com/share with your organisation’s and main clients’ names in Google, Bing and DuckDuckGo; record each hit’s URL and date.
6. API projects: record whether each key runs under paid or unpaid terms and whether ZDR applies.

Work use of a personal account, a training toggle left on where client data was handled, or an indexed link holding personal data counts as a finding; the last may need a breach assessment under your incident process.

FAQ

Is temporary chat or incognito mode actually private?

Only partly. Temporary modes keep chats out of your history, and most exclude them from training, but providers still hold copies: ChatGPT up to 30 days, Claude 30 days, Gemini 72 hours and Grok’s Private Chat up to 30 days, while Perplexity’s incognito threads expire after 24 hours. Court orders and safety review can reach them meanwhile.

Does deleting a chat stop it being used for AI training?

It stops future use but does not undo past use. Anthropic says a deleted conversation will not be used for future training, and Perplexity says training data it has already collected cannot be removed. Memory entries can outlive the deleted chat, and Google keeps human-reviewed Gemini chats for up to three years after you delete your activity.

Can ChatGPT conversations be subpoenaed or used in court?

Yes, they can be preserved and handed over in litigation. In the New York Times copyright case, a court ordered OpenAI to keep consumer chat logs it would otherwise have deleted, and in January 2026 a district judge upheld an order to produce 20 million de-identified logs. Enterprise, Edu and zero-data-retention API traffic fell outside the preservation order.

Can I paste client personal data into an AI chatbot under GDPR?

Not into a consumer account. It gives you no Article 28 processor contract, the provider may train on the data, and you do not control retention. A business, enterprise or API tier with a data processing agreement, no-training terms, a transfer mechanism and, for high-risk processing, a DPIA can make it defensible; pseudonymise first where you can.

Which AI chatbot is the most private by default?

None of the consumer tiers is private by default: seven of the eight covered here train on chats by default, and Claude’s consent toggle was reported pre-set to On. The protective defaults sit in business tiers: OpenAI, Anthropic, Google Workspace, Microsoft, Perplexity Enterprise, Mistral Enterprise and xAI’s business plans exclude customer content from training.