AI Act in Practice: The New AI Risks Organisations Need to Address

On 4 June, we hosted a webinar at NorthGRC on the AI Act, AI governance and the new risks organisations face as artificial intelligence becomes an increasingly integrated part of everyday business operations.

You can watch the webinar recording here.

And perhaps the best place to start is with a story.

A few months ago, Lone Forland, Product Expert at NorthGRC, was preparing documentation for the company's recertification process. To save time, she asked an AI assistant to summarise the findings from an internal audit.

Within seconds, the AI produced a clear and well-structured summary. However, one recommendation stood out:

"Upgrade the access control system for the basement server room."

There was just one problem.

 

"We haven't had a server room for 15 years. We're a 100% cloud-based company. We do have a basement, but we use it as a gym," says Lone Forland.

Fortunately, the mistake was easy to spot.

But the example illustrates a challenge that many organisations are already encountering in practice: AI systems can generate responses that sound convincing and credible without necessarily being correct. This is known as a hallucination.

And it is just one of the reasons why the EU has introduced the AI Act.

The AI Act Is About More Than Regulation

Many organisations still view AI as just another technology tool alongside cloud solutions, automation platforms and business software.

However, AI introduces challenges that differ significantly from the technologies we have traditionally worked with.

"One of the areas where AI really differs from traditional systems is what we call the black box phenomenon," explains Magnus, Head of Consulting Services at NorthGRC.

"It can be extremely difficult to understand what lies behind the conclusions or recommendations generated by an AI system."

Traditional systems generally follow predefined rules and programmed decision paths. AI systems, on the other hand, rely on vast datasets and highly complex models, making it difficult to explain exactly how a particular decision was reached.

That is why the AI Act is not just about compliance. It is also about governance, accountability and ensuring that organisations can use AI in a transparent, secure and responsible manner.

Who Falls Within the Scope of the AI Act?

One of the questions we hear most frequently is whether the AI Act only applies to organisations developing AI systems.

The short answer is no.

The AI Act introduces several different roles, including providers, importers, distributors and deployers – organisations that use AI systems within their own operations and business processes.

For most organisations, the deployer role will be the most relevant. This means that even if you are not developing AI yourself, you still have responsibilities regarding how the technology is used.

At the same time, we are seeing more organisations building their own AI agents or adding AI functionality on top of existing solutions. This can quickly blur the line between deployer and provider.

"If you build your own logic on top of an AI system, you should be aware that in some cases you may move from being a deployer to also becoming a provider," says Magnus.

This distinction is important because the regulatory requirements change significantly depending on the role your organisation assumes.

Four AI Risks Every Organisation Should Understand

The AI Act is built around a risk-based approach.

Before discussing compliance, it is essential to understand the risks that AI introduces.

1. Hallucinations

Hallucinations occur when AI generates information that appears correct but is actually false.

The fictional server room is a relatively harmless example.

Other cases can be much more serious.

 

"We have already seen examples of professionals referring to information or legal cases that never actually existed," says Magnus.

When AI is used in legal, financial or compliance-related contexts, the consequences can be significant.

AI should therefore always be viewed as an assistant, not an authority.

2. Bias

AI models are only as good as the data they are trained on. If the training data contains bias, the outputs may do the same. This can be particularly relevant in recruitment processes, where historical data may unintentionally favour certain groups of candidates.

"There is always a risk that bias in the training data influences the recommendations or assessments produced by the system," explains Magnus.

For organisations, this means AI-generated outcomes should not be accepted uncritically, particularly when they affect people or high-impact decisions.

3. The Black Box Challenge

One of the most discussed issues in AI is explainability. If an AI system recommends rejecting a candidate, lowering a credit score or prioritising one customer over another, the organisation must be able to explain why.

And that is not always straightforward.

"When using AI-based systems, you don't necessarily know what is happening inside the model when decisions are being made," says Magnus.

This challenge becomes particularly important for high-risk AI systems, where the AI Act introduces stricter requirements for documentation and traceability.

4. Shadow AI

Most organisations are already familiar with the concept of shadow IT. Shadow AI is the next generation of the same challenge. Employees begin using AI tools outside established governance processes, while AI capabilities are simultaneously being added to existing systems at a rapid pace.

• A CRM platform gains an AI assistant.

• A recruitment system introduces AI-based candidate screening.

• Microsoft 365 gets Copilot.

Suddenly, AI has become part of systems the organisation already uses every day.

"You should not focus only on new AI systems. You also need to be aware that existing systems are continuously being enhanced with AI functionality," says Magnus.

For this reason, organisations should establish processes to continuously identify and assess new AI capabilities across their existing technology landscape.

The AI Act Has Already Taken Effect

Although some of the AI Act's key provisions will not apply fully until the coming years, the regulation is already having an impact.

Since 2 February 2025, certain AI practices have been prohibited, and the requirement for AI literacy has also come into force. See the table below for key requirements and implementation dates.

This means organisations must ensure that employees working with AI have an appropriate understanding of both the technology and the associated risks.

"It's not enough to know how to use AI. You also need to understand the risks that come with using it," says Magnus.

This includes understanding issues such as hallucinations, bias, lack of explainability, data protection and information security.

Turning the AI Act into Practical Action

Understanding the AI Act is one thing. Turning its requirements into practical processes, controls, risk assessments and documentation is something else entirely.

At NorthGRC, we have incorporated the AI Act requirements into a dedicated framework designed to help organisations work systematically with AI governance, AI literacy, risk management and compliance.

See how NorthGRC helps organisations manage AI Act compliance in practice.

What We Learned from Our Own Certification Process

At NorthGRC, we recently completed our ISO 27001 and ISO 27002 recertification process.

One thing became very clear: auditors are paying close attention to how organisations use AI.

This includes dedicated AI tools such as Gemini and Copilot, as well as existing systems that have gained AI functionality over time.

"We actually received very positive feedback on the way we approached AI governance," says Lone Forland.

Our work included:

• Establishing a responsible AI use policy
• Training employees on how and when AI can be used
• Mapping AI systems and AI-enabled functionality
• Conducting risk assessments
• Implementing relevant controls
• Documenting responsibilities and decisions

One important lesson was that AI compliance does not start from scratch. Organisations already working systematically with information security, GDPR, ISO 27001, NIS2 or DORA often have much of the foundation in place.

 

"A lot of the work is not about creating entirely new processes. It's about extending the processes you already have so they also cover AI," says Magnus.

How to Get Started

For most organisations, the first steps are not about advanced technology. They are about gaining visibility and control.

Start by asking four questions:

1. Which AI systems are we currently using?
2. Which AI capabilities already exist within our existing systems?
3. What risks do they introduce?
4. Which regulatory requirements apply to our use cases?

From there, organisations can build on:

• AI policies
• Governance structures and accountability
• Awareness and AI literacy
• Risk assessments
• Controls and documentation
• Ongoing monitoring

The earlier this work begins, the easier it becomes to integrate AI compliance into existing governance processes.

AI Requires Governance – Not Just Enthusiasm

AI offers enormous opportunities to improve efficiency, enhance decision-making and free up time for more valuable work.

At the same time, it introduces new risks that organisations must be able to identify, assess and manage.

AI does not necessarily make mistakes in ways that are easy to detect.

That is why the AI Act is ultimately about ensuring organisations can use AI with the necessary levels of control, transparency and accountability.

Need Help Navigating the AI Act?

The AI Act introduces new requirements around governance, risk management, documentation and workforce competence. For many organisations, the biggest challenge is not understanding the regulation itself – it is turning the requirements into practical processes that work in day-to-day operations.

At NorthGRC, we help organisations establish AI governance frameworks, conduct risk assessments, implement controls and create the documentation needed to manage AI compliance effectively.

Whether you are mapping your AI systems, assessing risks or building a governance framework from the ground up, we would be happy to discuss your situation and help identify the next steps.

Get in touch for an informal conversation about the AI Act and AI governance.