Practical uses for AI in IA from real-life experiences

What can internal audit use AI for? Shehryar Humayun, Managing Director – Models, Data & AI Audit at Lloyds Banking Group, explains some real-life use cases that his team has tried and tested.

How to grow with AI

1. Training. Upskill auditors to be AI-ready, for example, by offering prompt engineering courses. Auditors need to be using AI as well as auditing AI. They also need to understand the limitations and risks.
2. Make AI accessible for all auditors. You can start by sharing useful prompts and try creating agents and chatbots that offer quick wins with low risk.
3. Hire for a learning mindset. AI is developing fast. Curiosity and a learning mindset are more important than technological expertise.
4. Give auditors a safe environment in which to experiment as learning opportunities. If you don’t get things right the first time, you won’t innovate.
5. Look for innovative ways to audit. Re-imagine what you’re doing.
6. Develop an innovation process. Spot an opportunity in an audit, turn it into an experiment, and then productionise it into an app for all auditors to use.

“Traditional” AI

The focus is now on Gen AI and Agentic AI; however, the older “more traditional” forms of AI, such as supervised and unsupervised machine learning, still provide great opportunities for auditors.

Supervised machine learning takes historical data and uses it to make predictions, for example, using your past viewing history to recommend films on Netflix.

Humayun’s team has been using supervised machine learning for several years to measure the reliability and consistency of human decision making.  Use cases have included assessing:

• customer complaints classifications;
• categorisation of risk events;
• allocation of securities trades between banking and
• trading books;
• behavioural risk reviews;
• categorisation of data quality issues; and
• the classifications in financial crime monitoring.

Unsupervised machine learning looks for patterns in data, and Humayun’s team used this to build an app that auditors use to spot outliers in large datasets. “We’ve used this in testing FX trading processes, mortgage interest rates, funds benchmarking, and for anti-bribery and corruption controls,” Humayun says.

Apps for accessibility

One major cause of resistance to using AI is unfamiliarity and fear. “To break the fear barrier, we democratised AI, creating an app portal and delivering AI in a familiar format they could use immediately,” Humayun explains. “Everyone knows how to use apps.”

Once internal auditors open the app, they can analyse data they need to test or explore. There are different apps for different uses – new ideas for AI applications are encouraged, assessed, and controlled appropriately, and auditors and engineers must work together to design and build solutions.

Gen AI

Lots of audit functions are using AI, particularly Gen AI, to improve operational processes and efficiency, but not many are using it for internal audit fieldwork and testing. Humayun says they’ve started using this for audit testing. “It helps us to find the needle in the haystack,” he says. “It’s helped augment teams with specialist insight outside their core domain.”

One example is a review of transaction documentation against detailed technical requirements that would have previously required huge numbers of experts with non-banking specialist knowledge. This would have been expensive and time-consuming.

“Using AI, we assessed 13,000 agreements against a 400-page technical requirements document that would have required specialist knowledge, and so it helped auditors focus on those that required human judgment,” Humayun says.

AI is also being used to support business monitoring, creating manageable briefing packs by analysing the information coming out of committees against detailed risk-management criteria. Other uses include looking at how controls are documented and categorised.

 “We are using Gen AI to improve the quality of our audit documentation,” Humayun adds. “We’ve embedded AI into our audit management system so we can use it alongside our own judgments to give auditors feedback on their work papers, avoiding later review points and QA findings.”

Another innovation is a chatbot called Aria, which has access to a secure knowledge base of past audits, controls, findings, and more. Internal auditors can ask specific questions knowing the information source is reliable, helping them to plan, draft findings, and look for repeat issues and read across between audits.

Fighting fire with fire

Humayun’s team is also using AI to audit AI. They’ve created tools to check whether business AI models are adequately controlled and assess their performance. “For me, that’s a brilliant way of taking the audit profession forward,” he says. “Auditors can look at the controls and at what management does, but these are deeper tests, looking at what the AI in the business is really doing.”

He believes this gives internal audit a much more powerful conversation to have with the business. “It’s so much more valuable than just asking what controls they have in place,” he adds.

IA in the frontline of innovation

Internal audit is the third line of defence, but Humayun says it can be in the frontline for innovation.  “We are a team of 350 people in a company of 65,000 people,” he points out. “Our reach across the organisation and ability to be nimble puts us at the leading edge. We want to be the pathfinders for the business.”

He has the advantage of leading teams that roll out AI in audit and teams that audit AI, so he sees both sides. “That gives you so much knowledge when you set out to audit AI, ” he says.  “It’s important that auditors have experience on both sides, particularly in smaller teams.”

Supercharged humans

The pace of change is mind-boggling, Humayun says, but he believes AI is currently being used less than is often implied.

“Many businesses are not yet using autonomous agentic AI,” he says. “You have to think about accountability if you allow AI to make decisions using judgment and learning, which is the only reason you would use autonomous agents. This is incredibly challenging.”

But things are changing fast. “Most important is the ability to learn, curiosity, and a growth mindset,” Humayun says. “The real benefits come from re-imagining what you are doing. Thinking differently is what truly adds value.”

However, Humayun stresses that you still need a human in the loop. “You need to train the audit team to understand what GenAI does. You must understand its risks and limitations,” he warns. “And once you have this data, humans must still think about how they will use it. So human judgment becomes even more important”.

“AI answers questions, but humans still need to ask the right questions – this is what auditors do for a living. It’s not about replacing internal audit, but about super-charging our skills and abilities.”