Agentic AI is now on the board agenda, raising urgent strategic and governance questions for leaders.

It is our observation that within a relatively short horizon, AI agents could constitute a significant part of an organisation’s workforce. This is the planning assumption several major Australian enterprises are adopting.

Now is the time for leaders to ensure appropriate governance is in place to manage increasingly adaptive, proactive and autonomous AI agents, and maximise strategic value.

Without this focus, the governance gap will be very stark; no board would allow a human workforce to operate without clear reporting lines, defined authority limits, performance oversight and the ability to manage and terminate. Sound governance is especially vital for agentic AI as the speed to scale is a real risk. Whereas human workforces grow incrementally, an AI agent workforce can scale quickly and exponentially. It is vital boards appreciate this asymmetry between deployment speed and governance readiness as they govern this technology.

What is agentic AI?

Understanding what AI agents are, and how they differ from other AI systems, is vital for effective board oversight. In this guide, we use the definitions aligned with the Department of Industry, Science and Resources’ Terms and Definitions, used in the National AI Centre’s Guidance for AI Adoption (AI6).

An AI system is a machine-based system that generates outputs (such as predictions, content, recommendations or decisions) in response to the input they receive. Most AI systems organisations currently use are in this category: they respond when prompted and stop when the task is complete.

An AI agent is an automated entity (often comprised of multiple orchestrated AI and technological systems) that senses and responds to its environment and takes actions to achieve goals. AI agents go beyond answering questions or generating content, they act and adapt. Boards/ leaders could think of AI agents as a new category of ‘worker’, one that scales instantly, never sleeps and can create other ‘workers’ (often automatically).

Unlike other AI systems, AI agents can:

• interpret objectives;
• plan and execute multistep actions;
• use tools such as databases, email, APIs, code execution environments and communication platforms;
• coordinate with other AI agents, and even create new ones, resulting in a multi-agent architecture; and
• take real-world action, often with minimal or no human involvement.

The critical difference between AI agents, AI systems, and traditional technology is autonomy. Agents have a degree of autonomy and can adapt their behaviour over time, develop strategies not anticipated by developers or deployers, and operate at speed and scale. Agentic AI amplifies value — but also amplifies risk.

The board’s role: managing agentic risk and capturing strategic value

AI literacy is an increasingly important board competency. To effectively govern agentic AI systems, board members need to understand:

• what makes AI agents different from traditional automation and other types of AI systems, including generative AI;
• the risks those differences create; and
• whether and how the organisation is governing and deploying AI agents responsibly.

Under Australian law, organisations will generally be liable for the acts of their AI systems — including where those systems act without explicit human instruction. Directors’ duties of care, diligence and acting in the best interests of the company apply squarely to the governance of artificial intelligence, including agentic AI. This means Australian boards need to be proactive in governing AI use and management – and this governance needs to be ongoing and agile. Boards are increasingly including AI governance as a standing agenda item, in a similar way to privacy or cyber security.

With this responsibility in mind, we now turn to the five key focus areas that can help board members to govern agentic AI to minimise risk and maximise value.

Five key focus areas when governing agentic AI

1. Perform an AI governance health check

Before approving or scaling agentic AI, boards should satisfy themselves that foundational AI governance is in place and operational. The following questions provide a practical starting point.

If these foundations are missing, AI agents will expose — and amplify — existing gaps.

2. Understand the unique features of the AI agent

Once the foundations are in place, one of the most useful questions a board member can ask is:

What new risks introduced by agentic AI are outside the scope of our existing policies, processes and controls?

To answer that question at a board level, the organisation first needs a shared language for describing and comparing AI agents – a strong classification scheme is crucial. If AI agents are going to be prevalent in our organisations, we need equivalent role descriptions and authority matrices for these AI workers.

Boards should ask management: Do we have a classification scheme that can be applied consistently to any AI agent we deploy or procure?

For example, a classification scheme might capture the following 5 dimensions about an AI agent:

• Function – what is the AI agent designed to do? This can range from a single, well-defined task, to a complex, multi-step process involving discretionary judgement.
• Predictability – how consistent and explainable is the AI agent’s behaviour?
• Operating environment – how controlled and complex is the operating environment in which the AI agent operates? (An agent in an internal, closed system carries a different risk to one that is customer facing).
• Scope – what systems and data can the AI agent access?
• Autonomy – how independently does the AI agent act? This ranges from recommending actions for human decision to executing within defined boundaries under human monitoring, to deciding and acting independently.

Together, these dimensions allow the organisation to assess and compare AI agents consistently – and to identify where existing policies and controls may be insufficient.

Critically, classification should not be static. Boards should expect that management has implemented processes to enable identification and reassessment when any of these dimensions change materially — particularly autonomy, scope or environment, which can carry significant governance and risk implications.

3. Understand the new risks AI agents create: map new and amplified risks

With classification in place, boards can interrogate agentic AI risk and opportunity more effectively and satisfy themselves appropriate controls are in place. The fact that AI agents can act autonomously and adapt means that errors can propagate rapidly before they are detected. This makes traditional, intent-based governance insufficient, and makes governance frameworks designed around fixed rules and predictable behaviour inadequate. Outcomes may diverge from expectations, even where systems are deployed in good faith.

Boards should ask management:

Have unique AI agentic risks been identified, assessed and mitigated? Do we understand our key legal and technical obligations in relation to agentic AI use? How are we managing identified risks?

Amplified risks, which are existing risks that agentic AI increases in scale, speed or complexity, include:

• Existing data governance may be insufficient for agentic AI, as an agent reasoning across various datasets (e.g. HR, finance, operational data) may surface or act on sensitive inferences that no single dataset would have revealed alone.
• Agentic AI can blur traditional permissions and accountability boundaries. An agent authorised to book travel, for example, may also need the technical access to approve expenses or reassign budgets if constraints are not explicitly defined.
• Many agentic systems depend on model providers that may update underlying models without notice. This can increase supply chain and operational risks, as behavioural changes flow through automatically to operations.
• Agentic AI can create significant audit and explainability challenges, as AI agents make chains of decisions faster than humans can review.

Novel risks, being risks that agentic AI introduces that are unlikely to be managed by traditional governance frameworks, include:

• Goal misalignment will occur where AI agents interpret an objective too literally, too broadly, or otherwise incorrectly. An AI agent instructed to reduce customer wait times might begin cancelling un-resolved tickets rather than escalating them.
• The behaviour of AI agents can change overtime, sometimes without notice, and for reasons that are difficult to explain – leading to the risk of behavioural drift. The ongoing ability to independently assess or predict the correct outcome becomes increasingly important.
• Where multiple agents interact, multi-agent failures can emerge. This may include, for example, taking conflicting and duplicating actions that cascade in ways that are difficult to trace and manage.
• Tool misuse occurs when AI agents invoke external tools and systems in unintended ways — for example, querying data they were not intended to access, or calling a tool repeatedly in ways that cause downstream disruption.

4. Set bright lines — and enforce them

Once the risks are understood, the board can then guide leadership to translate these risks into guardrails, and ensure those guardrails are operationally enforced.

Drawing the lines

Start by agreeing 'bright lines': non-negotiable limits on what AI agents must never do independently. The key question is: What things won't we let AI agents do?

Examples may include:

• decisions with significant legal or regulatory consequences;
• actions affecting vulnerable individuals; and
• irreversible or high-value transactions above defined thresholds.

Bright lines define the outer boundaries of acceptable agentic AI systems. They can be incorporated into an AI agent classification scheme to form part of the organisation's shared language (e.g. hard limits on scope, autonomy level, or operating environment) and enforced through operational controls.

Build the guardrails

Once the lines are drawn, the organisation needs controls to enforce them. These controls might include, for example:

• unique identifiers so agents can be distinguished and outputs are attributable;
• least-privilege access to systems and datasets, with appropriate and time-limited permissions;
• sandbox testing before deployment, with defined testing and acceptance criteria; and
• a documented oversight model specifying which actions require human approval versus monitoring only.

To accommodate these guardrails, existing frameworks, particularly risk management, cyber, procurement and data governance, will likely require AI agent-specific extensions. The AI Governance Framework should be reviewed to confirm it addresses AI agent-specific risks and that reporting to the board is appropriately focussed to support meaningful oversight. Where AI agents collect or act on personal information, the organisation's Privacy Policy should also be reviewed - both to reflect how AI systems are used and to address any obligation to disclose automated decision-making.

The appropriate level of governance will of course vary with the maturity of the organisation and the complexity of its AI agent deployments. And as such, boards will need to calibrate their oversight accordingly. That said, a key a critical over-arching control is whether the system can be shut down, and whether the organisation has a back-up plan to ensure uninterrupted service delivery to customers.

Make sure AI agents can be terminated quickly

If something goes wrong, can we immediately shut down the AI agent?

Rapid containment capability is essential. AI agents should be able to be immediately stopped without specialised technical knowledge, and this should be tested like any other business continuity measure.

5. Align AI agent deployment to strategy and value creation

AI agents are not only a risk to be managed — they give rise to critical strategic opportunities and considerations.

Boards should be asking:

• Where might AI agents create the most value, and is investment aligned to strategic priorities?
• What can we do now because of agentic AI that we could not do before? This is where transformational value lies.
• Is the organisation investing in the foundations on which AI agents rely — data quality, system integration and clear processes?
• Are we moving at the right pace?

Moving too slowly creates competitive risk. Moving too fast, without governance, creates financial, operational regulatory and reputational risk. The board’s role is to ensure deliberate, informed choices about ambition, readiness and control.

A board's role in AI governance needs to be agile as the maturity and complexity of the organisation's AI strategy and governance evolve.

AI agents will reshape how organisations operate. Boards that govern early, clearly and deliberately will be best placed to capture its value — without losing control.

Let's take your next AI step together. Contact our national AI Advisory team today, to discuss your current boardroom AI themes.