Does using a public AI tool break client confidentiality?

It can, depending on the provider's terms, your settings and your professional rules. Avoid pasting confidential material into any tool without a reviewed policy. Business and API plans usually give stronger contractual protection than free consumer tools.

Will a provider train its models on my data?

Terms differ by provider and plan. Business and API plans commonly exclude training on your data, but consumer plans may not. Read the contract and the settings, and keep a record of what you chose.

Can private AI match frontier models?

For many defined tasks, such as summarizing documents or answering questions from your own files, it can. Frontier models are still stronger at open-ended, difficult work. Test both on your actual tasks before deciding.

What should an AI usage policy contain?

Approved tools, the data classes allowed in each, human review requirements and logging. It should also say who may approve a new tool and what to do after a mistake. Keep it to a page or two so staff will read it.

How can Clayton help?

Through our Super Intelligence service: staff training, model selection and private deployment. We help you decide which workloads can use a frontier API and which should stay inside your own infrastructure. We also write the usage policy.

Is a local model always more private?

Not automatically. Running a model on your own hardware keeps prompts from leaving your network, but you still need access controls, encrypted storage, logging and a policy on what staff may paste in. A poorly secured local server can expose more than a well-governed business plan with a reputable provider. Privacy comes from the controls around the model, not only its location.

Private AI & Local LLMs for Sensitive Data

Key takeaways

Not every task needs a frontier model; routine work runs well and cheaply on smaller models.
Information that must stay in-house can be processed by local models inside your firewall.
A router can decide, per request, whether a query goes to a local, private-cloud or frontier model.
Training your team matters as much as the technology.

Why this question has come up

Most firms we talk to are not asking whether to use AI. Their staff already do, often with a personal account on a consumer tool. The risk is not that AI is dangerous in the abstract. It is that a paralegal pastes a privileged memo into a chat window, an analyst uploads a client spreadsheet, or a clinician's assistant drops a note with a patient name into a public tool, and no one has decided whether that is allowed. A good policy starts by sorting the work into classes of data and then deciding where each class may run.

A simple data classification

Public or non-sensitive: marketing drafts, public research, general coding questions. Almost any tier is acceptable.
Internal: business plans, internal email, non-public financials. A contractually protected tier is appropriate: a private cloud tenancy or an enterprise API with data-use terms you have read.
Regulated or privileged: attorney-client material, protected health information, customer financial data, personal data under privacy laws. Keep it local, or use a provider only under a written agreement that fits the rule that applies to you, such as a business associate agreement for health data.
Secrets: credentials, keys, unreleased source code and trade secrets. Never send these to any external model, and do not paste them into a local one that logs prompts to a shared location.

The classes map to the tiers on this page. The point of a router is that staff do not need to remember the table. The system looks at the request and its attached files, applies the rule, and either sends it to the permitted model or blocks it with a clear message.

Three tiers

Local: open-weight models on your own hardware, for privileged or regulated information that must never leave your network.
Private cloud: models in a dedicated environment under contract, for internal work that may leave the building but not the company.
Frontier: the most capable models, such as Anthropic's Claude, for complex reasoning on information cleared for it.

Between a local model and a public chatbot sits a private-cloud tier: hosted models reached through your own cloud account. Examples are Amazon Bedrock on Amazon Web Services (AWS), Google Cloud's Gemini Enterprise Agent Platform, Microsoft Foundry (formerly Azure AI Foundry) on Microsoft Azure, and OCI Generative AI on Oracle Cloud Infrastructure (OCI). Each is governed by its own contract and data terms, so read them against the data classes above before any sensitive material is sent.

Let a router decide

A router classifies each request — by data sensitivity, task difficulty and cost — and sends it to the right tier. Staff use one interface; the policy is enforced in software, not by memory.

What a local tier actually involves

A local model is an open-weight model running on hardware you own or control, such as a workstation with a capable GPU or a small server in a locked room or a colocation rack. The prompt and the documents never leave your network, which makes the privilege and data-residency conversation simpler. The trade-offs are real. Open-weight models are usually less capable than the best frontier models on the hardest reasoning tasks. You are responsible for patching the serving software, controlling who can reach it, and keeping logs under access control. The hardware has a cost, and it needs a person to operate it. For many routine tasks, such as summarizing a deposition transcript, classifying documents, drafting a first version of a routine letter or extracting fields from invoices, a well-chosen smaller model is enough.

Retrieval is the usual way to make a local model useful on your own documents. The system indexes your files into a vector store and passes relevant passages to the model along with the question. Two cautions follow. The index must respect the same access controls as the source documents, or a junior employee will retrieve a partner's files by asking nicely. And embeddings are derived from your data, so the vector store is as sensitive as the originals.

Risks that apply to every tier

The OWASP Top 10 for LLM Applications (2025 edition) is a useful list of what goes wrong. It includes prompt injection, where text in a document or web page changes the model's behavior; sensitive information disclosure; supply chain risk from third-party models and components; data and model poisoning; improper output handling, where model output is passed to other systems without checks; excessive agency, where a model is given more permissions than it needs; system prompt leakage; weaknesses in vector and embedding components; misinformation; and unbounded consumption. Few of these depend on whether the model is local or remote. A local model that is connected to your mailbox and file shares, and that follows instructions found in an email, is a bigger risk than a frontier model that only answers questions.

The NIST Generative AI Profile (NIST AI 600-1) is a companion to the AI Risk Management Framework. It helps organizations identify the risks specific to generative AI and choose actions to manage them, and it is a reasonable structure for an internal policy. We list it as a reference, and your counsel or compliance officer should decide which obligations apply to your sector.

Controls that matter in practice

An approved-tools list, with a plain statement of what each tool may be used for.
Single sign-on and multi-factor authentication on every AI tool, so that departing staff lose access at once.
Logging of prompts and responses for the local and private tiers, stored with the same protection as the data they contain.
A rule that a person reviews any AI output before it reaches a client, a court, a regulator or a patient.
Least privilege for connectors: an assistant that drafts email does not need permission to delete files.
Vendor review: where the data is processed, whether it is used for training, how long it is retained, and who the sub-processors are.
Training, with real examples from your own work, repeated at least once a year.

Measure before you scale

Pilot one use case, compare quality and cost against how the work is done today, and only then expand. The right model for each task level delivers the best result for the best cost.

Train the people

The fastest returns come from teams who know how to use AI well. Training — how to write prompts, check outputs and build simple tools — is the foundation of every adoption we lead.

Running a pilot

Choose one use case with a clear baseline: for example, first-pass review of incoming documents. Time how long the work takes now, and what mistakes occur. Run the pilot with a small group, compare quality and time, record what the model got wrong, and calculate the cost per task for each tier. The result tells you which tasks belong on the local tier, which justify a frontier model, and which should not use AI at all. Repeat for the next use case. A pilot that fails cheaply is a good outcome.

Cost belongs in the pilot too. Frontier models are billed by usage, so a task that runs thousands of times a month can become expensive, while a local model has a fixed hardware cost and falls to near zero per additional task once it is running. Compare the full cost of each tier, including the staff time to operate it, for the volume you really expect. Many firms find that a mixed design costs less than either extreme, and keeps the most sensitive work in-house at the same time.

Where we fit

Our team designs and builds local, private-cloud and frontier AI setups, including routing, retrieval over your own documents, access controls and staff training. We do not assume that the most powerful model is the right one. For some work it is. For privileged or regulated material, a model running inside your network is often the better answer. This article is general information and does not replace advice from your counsel on professional-conduct, privacy or health-information rules that apply to your work.

Sources

This article is general information, not legal advice. Confirm your obligations with your counsel. Last reviewed November 8, 2026.

Keep reading

Next · RegulationHIPAA Security Rule Checklist for Medical Practices and Business AssociatesThe HIPAA Security Rule applies to every practice and vendor that touches electronic patient data. Here is what it requires today, the deadlines that apply after a breach, and where HHS's proposed overhaul stands.9 min read

Need help with this?

American‑made Super Intelligence: AI your team can learn, trust and deploy.

Super Intelligence

Local, Private or Frontier? How to Use AI Without Exposing Sensitive Data