Tools for the job: How to audit innovation
A 2019 study by Accenture showed that 71 per cent of 10,000 companies in 18 industry sectors were “either in the throes of, or on the brink of, significant disruption”. No industry is exempt and the Covid-19 crisis is changing the landscape still further. New winners will emerge, but innovation will be the main driver of successful responses that shape the recovery.
While innovation is positively correlated with the financial performance of the organisation, it is risky. The internal audit core principles state that internal audit should be insightful, proactive and future-focused, and should promote organisational improvement – which means that it should pay close attention to innovation. This could include conducting an audit of innovation, depending on the organisation’s risk appetite and risk maturity.
How do you audit innovation?
Internal auditors are traditionally conditioned to avoid or mitigate risks, rather than to embrace and seize opportunities. This may explain the lack of tools, knowledge and literature around auditing innovation. It would be a complex audit mixing performance with research and development, organisational culture, project management and strategy, among others. The auditors would need broad expertise and insight, including knowledge of their sector, geographical location and wider socioeconomic trends.
Preparation
1 Ensure the organisation has defined its customers. Internal auditors should also check whether the organisation can answer the following questions:
• What is the most important problem or the biggest pain point for customers that the organisation is seeking to solve?
• How will customers measure the gain or benefit from any change or innovation?
2 Check the tone from the top. This is crucial in the innovation process. Many organisations aim merely to push innovation as a product or as the latest "flavour of the month" into organisational culture. This will lead to limited entrepreneurial management for a few departments (normally engineering or marketing), but will not generate broad awareness of the processes and goals associated with actual innovation.
The result could be a lack of understanding and appreciation for goals and methods if the innovation and its output are misinterpreted.
3 Check the organisation’s strategy on innovation including:
• Is the time horizon of the innovation strategy defined as three to five years (preferably five or more)?
• Is there a strong innovation alignment among different groups of employees?
• Has management decided which types of innovation are most important to the organisation?
• Has this decision been communicated clearly across the organisation?
• Is the focus wider than just disruptive innovation? While this is a common goal, it is not a strategy in itself.
• Is there a framework setting priorities and clarifying objectives across multiple innovation projects over time, and has it been communicated across the organisation?
There should be:
• clear objectives and priorities regarding the innovation;
• a clear and coherent value proposition for innovation;
• a defined proportion of resources allocated to different innovations;
• a balance between all types of innovation (in line with the global strategy of the organisation).
Internal auditors could use portfolio management techniques to audit the risks, criteria and efficiencies of each part of the innovation portfolio.
4 Check who is involved in the innovation process. This depends on people, not technology. Organisations employ specialists and generalists. Specialists have vast, but narrow, knowledge and are rarely beneficial in the innovation process. Generalists, who can search for ideas across countries, industries, companies, technologies and functions, are crucial here.
Increasing the number of people trying to solve a problem also increases the chances of success.
Internal audit should ask whether the right people are involved in innovation in their organisation.
5 Check the actual behaviour and culture around innovative projects.
If the risk appetite for innovation is low, mistakes will not be accepted by managers, who will support innovation only on the surface and in public. Be aware that trade-offs are an inherent part of innovation and consider this when exploring cases where innovation failed.
The innovation process
1 Is there clear insight into the issue that innovation seeks to solve? Check how the organisation is acquiring customer insights. Focus on the following risks:
• Exclusive reliance on focus groups or surveys. Customers do not always know their preferences and often change their minds when a new solution is presented.
• The team is too homogenous and lacks diversity of culture, background and expertise. Increasing the diversity of a team boosts the chances of successful innovation.
• Management is focusing only on the organisation's current technological capabilities. Companies tend to direct innovation questions to the R&D department and to go silent when asked about business model innovation.
2 Has management established the job that needs to be done? This process should focus on exploring the problem, not on finding the solution. Internal auditors should focus on the following risks:
• The “job to be done” has been defined as a solution. This leads to a narrow focus on solutions and not on solving the actual problem.
• Insights focus on details with limited connectivity between aspects.
• The organisation is not using all three modes of logic: abductive, inductive and deductive.
3 Has the organisation built a prototype of the product/innovation?
At this point, there should be theoretical, virtual and minimally viable prototypes; the goal is to test hypotheses and answer crucial questions. Prototypes should be designed to answer specific questions and managers should use clear metrics to assess them.
Internal auditors should focus on the following risks:
• Limited use of options A or B experiments in prototype development.
• Limited number of, or no, prototypes developed.
• Limited prototype testing, or testing on narrow customer segments.
• Inefficient process for developing prototypes.
4 How is the business validating the business model or the go-to-market strategy? Validating the business model involves testing not only the product (step 3), but also other components of the business model such as pricing strategy, customer acquisition strategy and the cost structure strategy.
Internal auditors should use three metrics:
• Love metrics (including measuring time, enthusiasm, recommendations, net promoter score and payment).
• Growth metrics (including, for example, customer acquisition economies of scale; activation, advocacy, retention and referral of customers; new revenues; efficiency of processes for defects and on-time deliveries, etc).
• Market power metrics (including market share, return on investment and financial accounting metrics).
Be aware of biases in each stage of the innovation process.
Measure uncertainty in terms of probability and define ambiguity in the process as “unknown unknowns”.
Biases include:
• Confirmation biases – biases that arise when people cling to, and search for, data that supports the initial or personal views of the decision-maker, prompting them to affirm or reject the hypotheses, despite evidence of opposing data or information.
• Anchoring – when the initial impressions of the innovation persist, so if the innovation has been labelled as a winner at the start, this view then persists despite contrary evidence.
• Attribution bias – when past success is attributed to the management’s or business’s capabilities, while failures are blamed on external forces. This leads to over-optimism.
• Recency effect – recent experiences are more powerful than older ones, so recent positive outcomes can lead to over-confidence about future success (and vice versa).
Look for a mix of analytical and quantitative methods behind management decisions about innovation. The major risk of analytical tools is that their precision creates an illusion of objectivity and accuracy. The innovation process requires a mix of traditional financial analysis together with qualitative methods, which include learning. When it comes to placing the bets on an innovation, there should be clear working hypotheses defining at least technology, markets, customers, the business model and strategy choices.
Consider how these hypotheses have been developed, and check there are defined criteria for acceptance or rejection, as well as a clear path for testing with potentially limited resources.
There is no magic bullet for the innovation process, but internal audit must explore ways to audit innovation now, when the world is changing at a phenomenal rate, if it is to be a valuable player in the boardroom and see the positive, as well as the negative, side of risks.
Matej Drascek is chief audit executive of Hranilnica LON d.d. and president of IIA in Slovenia. He will be presenting a session on auditing strategy at the Internal Audit Conference on 30 September-1 October.
Miha Skerlavaj PhD is professor of management at the Faculty of Economics, University of Ljubljana, Slovenia.
This article was first published in July 2020.