How to connect a secure chat to your business's own data

A shared chat can give employees a quick entry point to the organization's knowledge. The question can be simple – “which procedure applies to this case?” – but the solution behind it needs to handle data sources, identity, authorization, model selection, sources, and follow-up.

The most important security requirement is that the chat does not create a new, broader access than the user already has. Two people can use the same interface and still get different supporting materials depending on their role and document authorization.

What is meant by a chat on your own data?

A chat on your own data allows the user to ask questions in natural language to information that the organization has approved. This can be policies, manuals, reports, cases, or structured databases.

Typically, RAG is used for document-based knowledge. The solution searches for relevant text passages and sends them as context to a language model, which formulates an answer. For structured tasks, the chat can instead call an API or a database query under controlled forms.

”Own data” does not necessarily mean that everything is in the same storage system or that the model runs locally. It means that the organization controls which sources are connected, which users get access, and how the information is processed.

A shared entry point to the organization's knowledge should not mean shared access to everything.

From data source to searchable content

The first step is a source inventory. Where is the information, who owns it, how is it updated, and what permissions apply? SharePoint, file shares, intranet, and business systems can have different technical and organizational rules.

Documents then need to be read, split up, and indexed. Metadata such as title, date, document type, department, and access group should be included. When the source changes, the index needs to be updated without old versions continuing to affect the answers.

Data quality is a product issue. If the same procedure exists in four versions, the organization needs to decide which one is valid. The chat should not be forced to guess which information management applies.

Identity and access follow the user

Login, for example with Microsoft Entra ID, identifies the user. Thereafter, the solution must determine which functions and sources that identity is allowed to use.

Access may need to be controlled at multiple levels:

• access to the platform itself

• access to a specific assistant or function

• access to a data source, folder, or document type

• access to an action, such as export or system update

Permission filters should be applied before retrieving information, not just after the model has seen the material. Otherwise, there is a risk that sensitive information affects the answer even if the final text is filtered.

When an employee changes roles or leaves, access needs to be changed through the organization's regular identity lifecycle. Avoid parallel manual user lists that quickly become outdated.

Permission-controlled AI with Microsoft Entra ID – how access works

Read more about security in the Violet AI Platform

How RAG is used

When the user asks a question, it is converted into a search representation. The system searches only in the sources the user is authorized to access and selects relevant sections. These are sent to the language model along with instructions to stick to the supporting material and display sources.

RAG research shows the value of combining the model's generative capability with an external searchable memory.[1] In an enterprise environment, the method needs to be complemented with authorization, source management, and testing for the actual document volume.

Search and response should be measured separately. If the correct documents are not retrieved, the issue is likely index, filters, or search strategy. If the correct documents are retrieved but the response is wrong, the instructions, model, or post-processing need to be reviewed.

What is RAG? How companies use their own data in a language model

Which model should answer?

The choice of model depends on language, task, information class, response time, cost, and operational requirements. An external model via API can provide high capacity and quick access to new features. A local model may be necessary when information must not leave a controlled environment.

A model-neutral architecture makes it possible to choose a model per use case and switch when requirements or performance change. It does not eliminate all vendor lock-in – integrations and testing still need to be updated – but it makes the model a replaceable component instead of the entire product.

Sensitive questions can also be routed to a different model than general questions. Such routing requires reliable classification and testing so that data does not end up in the wrong place.

Local language models or cloud models – how to choose?

Logging, sources, and follow-up

Source references allow the user to check the answer and proceed to the original. Ideally, show the document title, relevant section, and date. A link to a large document without highlighted support provides limited transparency.

Logs are needed for troubleshooting, security, cost tracking, and quality work. But logs can themselves contain sensitive information. Therefore, define which fields are saved, who is allowed to read them, retention, and how personal data is handled.

Follow up on common questions, missing answers, incorrect sources, and feedback. Also evaluate behavior after every change to the model, indexing, or instructions. NIST recommends ongoing measurement and management of generative system risks over the lifecycle.[2]

What happens to company data when you use AI?

Security that must be tested

Instructions from users or documents can attempt to influence the model to act contrary to its rules. This is called prompt injection. Content in source material should always be treated as data, never as trusted system instructions.

Also test questions that attempt to compile more information than the user is allowed to see, hidden instructions in files, unusual file formats, and outputs that can be dangerous when sent to another system. Technical controls, least privilege, and scoped tools are important security settings.

What is required before starting?

A good pilot starts with a bounded knowledge area and a clear user group. Inventory:

• systems and data sources to be connected

• information owners and update processes

• information classification and personal data

• user groups, roles, and document permissions

• languages, file formats, and tables that must work

• requirements for local, Swedish, or European data processing

• test questions, forbidden responses, and acceptance criteria

• responsibility for support, incidents, and content after the pilot

Do not choose the entire intranet as the first test. A smaller area makes it possible to understand both the benefits and quality issues before more sources and users are added.

From idea to pilot: what is a PoC, prototype, and pilot?

Examples of use cases

Internal knowledge

Employees find policies and procedures with direct links to the source.

Technical documentation

Engineers search in manuals, reports, and historical analyses without sensitive data leaving the controlled environment.

Customer service

Agents get suggestions based on current terms and product information, but approve the response before dispatch.

AI assistant or AI agent – what is the difference?

Regulated document analysis

The user extracts information, compares versions, and creates draft reports with traceability to the document level.

Secure document analysis with AI in regulated industries

Talk to your Data

The chat combines documents with structured data sources and can answer questions based on user permissions.

Read more about the Violet AI Platform

From shared entry point to controlled use

The technical architecture needs to be based on how responsibilities and permissions are distributed in the organization. Data sources have clear owners, users have different roles, and every response should be used in a specific context. When these parts work together, the chat can make knowledge more accessible without bypassing existing boundaries.

Sources

1. Lewis, P. et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks:
   https://arxiv.org/abs/2005.11401

2. NIST, Generative AI Profile:
   https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence

3. IMY, Guidance on GDPR and AI:
   https://www.imy.se/verksamhet/dataskydd/innovationsportalen/vagledning-om-gdpr-och-ai/

4. Microsoft, Azure OpenAI on your data:
   https://learn.microsoft.com/en-us/azure/foundry-classic/openai/concepts/use-your-data