The vendors are doing a careful balancing act here. On one hand, they want to inspire trust in their own service. But at the same time, they need new human-created content to improve their model. Human content is super valuable. If you use an existing model to synthesize new training content, the new content only reflects the model, and does not really add to it. This is the reason why the model makers try to carve out user groups and services where they can collect new human creations, with as little attention or reputational damage as possible.
Here a quick summary of the promises, tricks and treats hidden into the terms of the most widely used GenAI services:
|
GenAI Service
|
Privacy and data protection promises
|
Comments
|
|
ChatGPT
|
The data protection is based on account type. User prompts are excluded from training by default when the user is on a Team or Enterprise plan.
|
Users that do not log in, or log in with an email address associated with a Free, Plus or Pro plan risk having their prompt data used for training. There is a process to opt out from allowing training by sending a request via their privacy portal.
|
|
Microsoft Copilot and M365 Copilot
|
“Enterprise-grade security, privacy, and compliance”. The DPA is the same offered for Office services like OneDrive.
|
Some Copilot features involve web searches, performed by Bing, where the exact search terms are said to be somehow redacted.
There have been issues with internal ‘oversharing’ related to Microsoft Connected Experiences. Enabling them should be a focused project, where oversharing is tested and mitigated.
|
|
Microsoft Copilot for Bing
|
No promise of privacy or data protection in the Terms. Microsoft reserves rights to prompts and responses, even to reproduce and display publicly.
|
There is an end user opt out for sharing data for personalized ads.
The user also needs to warrant that he/she has rights to all content in his/her prompt.
There is a real risk that an end user is not aware of the actual Copilot being used. They all have Copilot in the name and a similar icon.
|
|
Google Gemini
|
Google’s recently updated Gemini Apps Privacy Hub discloses that there are human reviewers that read and annotate GenAI interactions for service development purposes. However, when used with a work or school account, Google promises not to use data for training or subject it to a human review, although there’s a sentence in the Privacy Hub that caveats this: “For work users who have access to Gemini Apps as an additional service, while Gemini Apps Activity is on, Google collects your Gemini Apps conversations and other information you provide to improve and develop Google products and services and machine-learning technologies.”
|
It is particularly difficult with Gemini to know what terms apply in each situation. Gemini comes in many flavors depending on the service bundle, account type or promotion. A business user checking personal email, or the quoted term on the left column are examples of that. Google advises to look for a shield icon in the Gemini prompt. Absence of the icon may mean a human review and use of data for training purposes. Against expectations, there was no shield in NROC Security’s Gemini for Workspace used to edit this blog.
|
|
DeepSeek
|
DeepSeek reserves the right to use data for service development and does not rule our use of prompts for model training. It’s promised to be done in a de-identified and encrypted way. However it does not specify if any PII in the prompt is obfuscated, or only the identity of the prompter.
|
The Terms are written under the Chinese law. Unlike most of the similar services, DeepSeek does not have any anonymous mode, but all usage requires a registration of an account.
|
|
Grok
|
Data protection varies between a consumer user and enterprise user. Enterprise data is not used for product development or model training. Consumer data is used for both, although a consumer has a toggle to opt out from allowing data to be used for model training.
Enterprise service terms, Consumer service terms
|
What plans come with the enterprise data protection and at what cost has not been publicly disclosed as of mid March 2025. Business users with personal IDs can be assumed to now use the service under the consumer terms.
|
The above puts enterprise infosec teams into a new position. Until now, one could vet a SaaS product and contractually hold the vendor accountable for data protection. Use case, and used data was rather predictable, related directly to the purpose of the SaaS product.
With this new GenAI breed of SaaS, every business user invents new use cases daily and uses a selection of GenAI services based often on personal familiarity and preference. At the same time, the data protection depends, like the above table shows, on flavor of the service, type of login, level of subscription, geography or end user settings. This is likely more complex than what one can expect a business user to master.
Knowing your GenAI portfolio that is being used and limiting the amount of used services is a good first step. Purchasing some enterprise grade GenAI services makes sense. Actually enforcing the end users to use only corporate subscriptions, or have their individual opt outs correctly configured is already a lot harder. Therefore, instructing that no PII or corporate secrets are used in prompts (into any services), is often done to limit the diligence required by end users.
That may still require too much from the end user, especially as the amount of GenAI apps is increasing rather than shrinking, if not supported by technical controls. We suggest authenticating every usage on corporate id, monitoring usage, redacting PII and blocking use of classified content in the prompts.
***
NROC Security helps organizations get visibility and set guardrails for GenAI apps, like ChatGPTs, copilots and the likes. We enable organizations to realize the productivity benefits from those apps while mitigating risks and compliance issues. We inspect both prompts and responses, which is unique, and deliver from the cloud with no agents or browser plugins. We are expanding from personal productivity AI to business process AI by governing the flow of proprietary data -- 'the right docs to the right AI'.
.png)
