‘A company should be able to use a model without giving up the knowledge that makes it unique’: Microsoft CEO Satya Nadella says enterprises shouldn’t be sharing so much data with AI providers

Companies using AI should be wary about losing control of their data – and the risk of AI developers making use of that knowledge.

That’s according to Microsoft CEO Satya Nadella, who in a recent blog post warned many AI users are “paying twice” thanks to what he calls the “reverse information paradox”.

This is a process through which AI providers, particularly frontier labs, get access to proprietary data about their own customers. Nadella said the trend represents a huge risk for enterprises, and called for protections akin to patents.

The critique is intriguing as Microsoft itself is an AI company, offering plenty of AI-driven products to customers and cramming AI features into its software. Beyond that, Microsoft was an early funder of OpenAI, though that relationship has weakened of late.

The ‘AI exhaust’

This concern isn’t just about uploading corporate data to systems, or accidentally including proprietary data in a prompt, though those are both concerns. Instead, Nadella said every interaction with AI inherently includes information that could be useful.

“Models learn from ‘exhaust,’ the prompts people write, the tools agents use, and especially the corrections people make when the model is wrong,” he wrote.

“Every correction is distilled into institutional know-how. It’s the kind of knowledge a competitor could never buy, and the kind that leaks almost imperceptibly: trace by trace, correction by correction, eval by eval.”

Ironically, Nadella said it was necessary for the “great innovation” of allowing model developers to use public data – some of them face lawsuits for hoovering up copyrighted data to train their systems.

When it comes to business data, providers should share anything learned with customers. Very few actually do this, however.

“If learning flows in only one direction, economic value converges toward the owners of the learning infrastructure rather than the creators of the knowledge itself,” he said. “Therefore, it’s imperative that we distribute the learning infrastructure to every firm so that they can control their own learning loop.”

Nadella’s tips on how to protect your company from AI

Nadella said the existing setup requires a protection system on par with patents, but also advised companies to set a “trust boundary” by considering a few key aspects – some of which may be easier by simply switching to open source AI.

He suggested companies should control their own “evals”, or evaluations of the AI system, because that reveals what looks “good” to the organization, and keep ownership of decisions, feedbacks, and other outputs for their own use.

Beyond that, companies should build capability with AI via their own learning environments where models can learn using real workflows without exposing corporate data.

Improving choice can be achieved by decoupling the orchestration layer from any single model, enabling the ability to switch to another model if one is withdrawn. This can also help organizations cut costs by choosing the right model based on their individual needs.