How AI Is Changing
Your Organization’s Security

01 / The changing landscape

AI changes the pace of attack and the shape of your exposure

In its 3 November 2026 assessment, the UK’s National Cyber Security Centre describes how AI could increase the speed and scale of cyber intrusion and help attackers exploit weaknesses. Its scenarios extend to 2028 and are explicitly plausible possibilities, rather than predictions. The practical implication for a business is to improve its ability to find, prioritize, and fix exposure quickly. [1]

There are two connected workstreams: protecting existing systems against AI-assisted attacks, and securing the AI tools you adopt. A patched application can still leak information through an over-permissioned assistant. A carefully configured assistant can still sit behind a compromised employee account. Your review needs to cover both.

02 / Risks that matter

Six risks to put on your security agenda

1. More convincing impersonation and fraud

The FBI has documented criminals using generated text, cloned voices, and synthetic video to make fraud more believable. Fluent language or a familiar-sounding caller is weak evidence of identity. [2]

Practical response: Require independent verification for payment-detail changes, account recovery, and sensitive-data requests. Use contact details already held in your records, separate approval for important changes, and a clear way for staff to pause suspicious requests without being penalized for the delay.

2. Sensitive data reaching the wrong place

NIST’s generative AI risk profile identifies privacy risks from leakage and unauthorized disclosure or use of sensitive information. Exposure can arise before a model produces an answer: staff may upload confidential files, or a connector may retrieve more information than a task needs. [3]

Practical response: Maintain an approved-tool list and define which data each tool may handle. Check the provider’s actual retention, training-use, deletion, and access settings for your service and account. Classify documents before connecting them, limit retrieval permissions, and make it easy for staff to report accidental uploads. An “enterprise” label alone does not establish that your configuration meets your needs.

3. Prompt injection through documents and other inputs

Prompt injection happens when input steers an AI system away from its intended behavior. OWASP distinguishes direct instructions from indirect attacks carried through material such as web pages or files. Retrieval-augmented generation, which adds retrieved documents to a model’s context, does not eliminate this risk. [4]

Illustrative example: A support assistant reads a customer attachment containing instructions to disclose internal notes. Treat the attachment as untrusted data. Enforce document access in the application, validate output and tool requests, and test whether hostile content can cross those boundaries. A policy written into the system prompt should be one layer of defense.

4. Agents with excessive permissions or autonomy

A system that can send messages, change records, or run commands introduces consequences beyond an incorrect answer. OWASP calls out excessive functionality, permissions, and autonomy as causes of excessive agency. [5]

Practical response: Start with read-only access where possible. Give each integration only the actions and resources it needs. Validate authorization on the server for every action, require meaningful approval for consequential changes, record what happened, and provide a way to revoke access or stop the workflow quickly. Approval should show the actual recipient, data, or change being authorized.

5. New dependencies in the AI supply chain

AI systems depend on more than a model provider. Packages, model files, datasets, retrieval sources, and third-party integrations all influence the result. OWASP identifies risks involving compromised components, weak model provenance, and outdated dependencies. [6]

Practical response: Record which components and suppliers a workflow depends on, obtain them from trusted sources, and review changes before enabling them. Ask who can update a connector or knowledge base, how credentials are stored, and how an affected integration would be disabled. Reassess risk when the model, tools, permissions, or data sources change.

6. Insecure generated code and misplaced confidence

GitHub’s responsible-use guidance warns that AI-generated code can contain security vulnerabilities and should be reviewed and tested. NIST also identifies confidently incorrect output and over-reliance on AI as risk areas. [7] [3]

Practical response: Keep code review, dependency and secret scanning, authorization tests, and release approvals in the development process. Ask a reviewer to check failure cases and access boundaries. For a security report, verify important claims against logs, configuration, or a reproducible test before acting on them.

03 / Defensive value

Use AI to strengthen work your team can verify

AI can also support maintenance, test development, and code review; GitHub lists these among its agentic tools’ intended uses. The value should be evaluated against the quality of the resulting work. [7]

Choose a bounded pilot: draft a regression test, explain a proposed patch, or summarize an incident timeline from approved records. Compare the result with a human-reviewed baseline. Track missed issues, false alarms, correction time, and any sensitive data introduced into the workflow. Expand access only when those results justify it.

04 / Make it practical

A 90-day plan for your organization

The following is our suggested starting plan. Adapt its sequence and timing to your systems and highest-impact risks. NIST provides a voluntary framework for AI risk management, while the NCSC’s secure AI guidance covers design, development, deployment, and operation. These resources can inform an ongoing program. [9] [8]

05 / Evidence over assumptions

What should an AI security assessment test?

Begin with a concrete workflow and ask what could go wrong if its output or instructions were manipulated. Agree on authorized test boundaries and use synthetic data where possible. A useful assessment should answer:

• Can one user retrieve another user’s restricted information?
• Can content in a document, message, or tool result trigger an unauthorized action?
• Are important changes independently checked and approved?
• Do logs provide enough evidence to investigate a failure without unnecessarily exposing sensitive data?
• Can the organization stop the integration and recover its work?

Record the conditions, evidence, business impact, and remediation for each finding. Recheck the affected workflow after a fix and after material changes. A model’s behavior is only part of the assessment; the application’s permissions, connected systems, and human decisions determine much of the impact.