How to connect AI to company data without giving it access to everything

The request is almost always the same sentence. “We want an assistant that can answer questions about our business.” The next sentence is where projects succeed or stall: what is it allowed to read?

There are two easy answers and both are wrong. Give the assistant broad access and you have created a data exposure problem with a chat interface. Give it nothing and you have created a more elaborate way to paste text into a box, which is the manual work the project was meant to remove.

The useful answer is narrower and more concrete. Connect the assistant to a defined set of sources, carry the requesting user’s existing permissions through to retrieval, and keep consequential actions behind a person.

Start from an approved source list, not a data lake

Every connection should be a decision someone made on purpose. Write down which systems the assistant may read, what each one contributes, and who owns it. A workable first list is short: an internal knowledge base, a document library, a ticketing or case system, a customer relationship management system, or a single line-of-business application.

Then write down what is excluded and why. Payroll records, protected health information, personnel files, and legal matter files are common exclusions, and the reason is usually a retention or disclosure rule rather than a technical limitation.

A source list does two things. It bounds what a review has to cover, and it gives you a place to record additions later instead of quietly widening access.

Carry the user’s permissions through to retrieval

The most important design decision is who the assistant acts as when it reads. There are three patterns, and they behave very differently.

Pattern	How it reads	Trade-off
Permission-aware retrieval	As the requesting user, with their existing access	Correct by construction; more work to build
Service identity with scoped access	As one application identity, reading a defined source set	Simpler; every user of the assistant can reach everything that identity can
Unscoped application access	As a broad identity, filtered afterward	Fast to prototype; the highest-risk option

The first pattern is the one to aim for. When the assistant searches on behalf of the person asking, and the underlying system enforces that person’s permissions, then a user cannot receive an answer built from documents they are not allowed to open. The security boundary stays in the system that already owns it, and access reviews follow the access reviews you already run.

The second pattern is legitimate when the source set is genuinely shared, such as published internal documentation or a product catalog. Be explicit about it. If the assistant uses one service identity, then anyone who can ask the assistant can reach whatever that identity can reach, and that needs to be an intentional decision rather than a side effect.

The third pattern is where most incidents come from. An over-privileged service account, an index built from everything, and filtering applied somewhere after retrieval. It reads well in a demo and badly in a review.

Retrieval scope is not the only boundary

Permissions answer “what may this person see.” They do not answer what the assistant should carry into a prompt, what it may write back, or what gets retained afterward.

• Minimize context. Retrieve what the question needs rather than whole documents when the answer needs a paragraph. Less context is easier to reason about and cheaper to run.
• Separate read from write. Reading approved sources and changing records of record are different risk levels. Start with reads and drafts.
• Keep secrets out of prompts. Credentials, connection strings, and API keys belong in the application’s configuration or a managed identity, never inside a prompt or a shared template.
• Retain deliberately. Decide what the assistant logs, how long it is kept, and who can read the logs. Logs frequently contain the sensitive material the access rules were protecting.
• Test the boundary. Make a user who lacks access try to retrieve a protected document. If the attempt succeeds, the design has a gap regardless of what the diagram says.

Keep people in the loop where mistakes are expensive

A permission-aware assistant can still be wrong. It can summarize inaccurately, cite an outdated revision, or act on an instruction that was never authorized for it. The control for that is not a better prompt. It is deciding in advance which actions need a person.

A practical split:

• Read and summarize: low consequence, no approval needed beyond existing access rules.
• Draft: a person reviews and sends it. The assistant never sends on its own.
• Change a record, spend money, or contact someone outside the organization: a named person authorizes it, and the authorization is recorded.

This is also where an assistant earns trust. People accept a tool that drafts well and hands control back. They do not accept one that quietly commits them to something.

Logging, testing, and rollback

Every connected workflow needs an answer to three operational questions. What is recorded about each interaction, so a problem can be reconstructed later? How is the workflow tested before it runs on real work? And how do you turn it off or reverse what it did?

For a first implementation, the honest answers can be short. A structured log of request, sources retrieved, and outcome. A small set of evaluation cases that must keep passing. A documented switch that disables the workflow, and a way to correct or withdraw an action it already took. What matters is that the answers exist before the workflow touches anything that matters.

What this costs you

Permission-aware retrieval is more work than connecting an over-privileged account. There is configuration in the identity provider, sometimes an application registration per source, and testing that mirrors real access differences. Expect the first workflow to take longer than a demo did.

In return, the security model stays where the organization already manages it, and the assistant does not become a new place where data accumulates. That is a much smaller surface to review, and a much easier one to explain to whoever asks how it works.

Where to start

Pick one workflow where the friction is obvious and the sources are known. Write down the approved source list, the identity the assistant will use, the actions that require approval, and the way the workflow is turned off. Then build the smallest version that respects all four.

If you want that boundary designed before anything is connected, the AI Opportunity Assessment is where to define it, and a controlled example workflow shows the shape of the result. If you would rather start with a conversation about your own systems and constraints, get in touch.