Universities are facing a familiar challenge with unfamiliar tools. The advent of powerful AI offers a promise of more efficient and consistent outcomes in academic (and indeed non-academic) misconduct investigations. But with that promise comes a critical question: how do universities harness AI's value without compromising their regulatory obligations, ethics and reputation?

The universities that get this balance right will not only streamline investigations, but they’ll build trust in the integrity of their assurance and disciplinary systems and establish themselves as technological leaders in the higher education sector. Yet, those that rush in without adequate safeguards risk undermining the very legitimacy they seek to protect.

This article explores the investigative pressures universities face, the AI tools they’re turning to, and the governance principles that must guide their use.

Why universities are turning to AI

Universities are under pressure to investigate a growing volume of student academic misconduct cases with fewer resources. Timely responses are critical because delays in resolving matters can harm student wellbeing, impact academic timelines and expose universities to regulatory and reputational risk.

However, investigations into student use of AI are increasingly complexity as evidence typically exists across multiple digital platforms. Furthermore, the AI tools at students' disposal are becoming more sophisticated meaning that detection has become exponentially more difficult.

The higher education sector is grappling with the temptation to revert to high-stakes invigilated exams, against all the evidence that this form of assessment impacts alignment, equity, wellbeing, and risks compromising assessment in ways that disadvantage many students. Assuming that doesn't happen, and the volume and complexity challenge remains, it seems obvious that AI must be part of the solution.

There are also other benefits of using AI in the investigative process, including greater consistency of outcomes. Despite concerns, including the potential for AI to import a level of bias, it would be naive to suggest that human-led investigations aren't themselves vulnerable to similar follies, like unconscious bias.

The value of AI: what it can and can’t do

In a context of overburdened university support functions, AI offers a compelling path forward.

Natural language processing (NLP) tools can detect patterns in incidents and so can identify inconsistencies in statements for a particular investigation, and across the investigation portfolio. Tools like Turnitin and exam proctoring software are now standard to detect plagiarism and academic misconduct. Generative AI can create draft reports and findings in an instant. AI can even scan social media for evidence of misconduct and predictively model which students may be at risk of future misconduct.

As such, the value of AI lies in its ability to enhance investigative and document production capacity without increasing headcount, reduce human error, and accelerate resolution timelines. Universities that effectively integrate AI responsibly into their operations will gain a competitive edge, distinguishing themselves from others through innovation, fairness, and enhanced educational outcomes.

But the implementation of these tools raises some fundamental questions that need to be carefully thought through. These include:

• What are the safeguards to ensure the adequate protection of and access to input and output data of AI tools?
• What is the role that AI would play compared with the investigating officer to ensure appropriate human intervention?
• What other guardrails need to be in place to ensure the AI is functioning as desired and students get a fair hearing (for example, periodic testing against bias)?
• How will students be meaningfully informed when disciplinary action relies (at least in part) on an algorithm they may not understand and is not readily explainable?

The hard line: what universities must get right with AI

Even without whole-of-economy legislation to govern AI in Australia, its use in universities and specifically for investigations is already subject to a myriad of existing regulations.

For example, universities must comply with their relevant State legislation (such as the Privacy and Personal Protection Act 1998 in NSW) ensuring that personal data is handled lawfully and AI is leveraged with appropriate protection around that personal data. A recent study from the Journal of Academic Ethics indicated that more than 50% of faculty members and students in higher education are already concerned with data privacy and transparency of the systems as they stand. This concern is likely to be exacerbated without carefully considered expansion of AI use in student academic misconduct investigations.

Anti-discrimination laws also apply. AI tools must not produce outcomes that disproportionately affect students based on race, language, disability, or other protected attributes. These concerns are not merely hypothetical. History has shown that AI systems replicate the bias embedded in their training data, such examples include Amazon’s abandoned AI hiring tool which discriminated against women.. Special consideration must be given to preventing bias against cultural and socio-economic factors. If unchecked, AI may unfairly perpetuate bias both in detection and investigations of non-native English speakers or those from lower socio-economic backgrounds.

Fundamental principles of administrative law require students have a right to procedural fairness, including knowing the case against them and having a genuine opportunity to respond. In the absence of due process, students may lose trust in higher education institutions and turn to alternatives.

In the context of investigations potentially leading to disciplinary action, due process considerations arise throughout the process:

• Disclosure: Tell students the case against them and how AI tools were used.

   

• Access: Enable students to challenge AI generated evidence and understand the methodology, noting the general challenge around explainability of AI.

   

• Impartiality: Ensure AI tools do not introduce bias or undermine the investigators’ neutrality. A risk when the tools are poorly trained and maintained. 
• Response: Rapid AI processing must not come at the cost of students’ opportunity to prepare a proper response.

Without appropriate safeguards, AI risks undermining at least the perception of due process, fairness and therefore trust in the disciplinary process of higher education institutions. Recently, it was found that Australian Catholic University used AI detection tools to implicate nearly 6,000 students of academic misconduct, with one-quarter being dismissed as wrongly identified after further investigation.

The path forward: a general AI governance framework and key considerations

To mitigate the risks of AI, universities can implement an AI governance framework which houses 'constitutional documents' like statements of AI ambition, risk appetite and ethical principles. This could also include applicable policies and procedures such as acceptable AI use and AI vendor risk management, accountable bodies and officers. The AI governance framework should take account of the 10 guardrails set out in the Voluntary AI Safety Standards released by the Department of Industry, Science and Resources. It should also set out a risk management framework to apply to proposed uses of AI, with special treatment for those defined to be 'high risk', arguably including academic misconduct investigations.

For use cases such as investigations, some other practice steps universities should take include:

• Mapping the end-to-end investigation process to understand where AI is likely to deliver benefits and also where it imports risk that should be closely monitored by a "human in the loop".

   

• Ensure that each step of the process, and the use of AI, is readily understandable with clear statements on how risks have been mitigated and confirming a student's right to challenge.

   

• Interrogate the data sets on which any AI is trained to ensure it is diverse and representative to minimise bias.

   

• Routinely examine outputs to critically assess potentially biased outcomes, with a feedback loop to the AI system owner.

   

• Ensure investigators have strong understanding of the workings of the AI tools through training and are fully equipped to interpret and check AI outputs.

From efficiency to integrity: why governance matters in AI

AI has the potential to deliver significant benefits to the process of investigating student academic misconduct, but only if it’s used judiciously. Implementing AI is so much more than a technology decision.

The universities that succeed won’t be those that adopt the tools fastest. They will be the ones that embed AI fairness, transparency and integrity.