Good to chat: we asked ChatGPT what it can do
OpenAI’s new ChatGPT technology has been hailed as a gamechanger since its launch a couple of months ago. Its answers to a varied range of questions are impressive (if sometimes bland) and early users have been excited by its potential – while anxious about whether such technology could one day do their job better than they can.
While internal auditors may well find its ability to synthesise data, identify questions and create concise summaries of multiple texts a boon, there are serious pitfalls that internal audit teams should be considering if they or their colleagues across the business start to rely heavily on this powerful tool.
In the short-term, it is likely that organisations will need to take legal advice if, for example, HR teams use it to whittle down candidate lists. New rules may well come into force to prevent unscrupulous use. There are also ethical questions, such as informing customers or employees if your organisation is using this kind of technology to make decisions that concern them.
Additionally, internal auditors could run some random questions about their organisation through the system – what could investors, customers and staff learn from this technology? Are the answers in line with what the organisation would want them to see? Are they accurate and should there be new controls over information that is made publicly available now that it’s so easy to find? The information may have already been in the public domain, but if it was buried deeply on a social media app or on page 55 of a long report, it would have been rarely spotted. Now it could be on the first line of a two-paragraph answer to a simple question. Reputation, security and privacy could all be issues here.
Error is another important consideration. Over-reliance on ChatGPT could lead to false assumptions or to people making poor decisions. The technology is only as good as the information it trawls – which poses interesting issues around misinformation and “fake news”.
In the longer term, there are also serious questions to ask about originality, ideas and insights – if everyone has access to the same information, will humans continue to use it differently or will we become mental sheep, following the clever chatbot wherever it leads us and missing things that would have been obvious to a human viewer?
Furthermore, will emotional intelligence become even more important, helping the human AI-handlers in internal audit and other functions to identify the gaps by understanding that humans behave irrationally and are motivated by fears and impulses rather than data?
Overall, ChatGPT is an exciting glimpse of the future through a door that is likely to open fully in the next few months and years. There are huge opportunities that are already clear and many more will emerge as people start to use the technology more.
At the moment, ChatGPT is still free to use and open for experimentation. We would love to hear how you are using it in your internal audit function. What is it good at? Where have you encountered problems? In the next issue, we want to explore how it is changing/could change the way you work. Send your stories to ruth.prickett@iia.org.uk and let us know whether you are happy for us to publish them.
In the meantime, the following pages contain some questions we posed to ChatGPT – and what ChatGPT replied.
Will ChatGPT change the way we work?
“Yes, ChatGPT and other AI-powered language models have the potential to change the way we work in several ways. They can automate repetitive and time-consuming tasks, provide real-time access to large amounts of data and information, and support data-driven decision-making.
In addition, language models like ChatGPT can improve communication and collaboration between departments and stakeholders, leading to more efficient and effective work processes.
However, it is important to consider both the benefits and potential risks associated with the use of AI in the workplace. Organisations must carefully evaluate and manage these risks to ensure that AI is used responsibly and ethically, and to maximise the positive impact it can have on work processes and outcomes.
Overall, ChatGPT and other language models have the potential to significantly impact the way we work and transform various industries, but their impact will depend on how they are adopted and implemented.”
What can ChatGPT do for internal auditors?
“ChatGPT can provide several benefits to internal auditors, including:
1 Automating repetitive tasks: ChatGPT can automate tasks such as data collection and analysis, freeing up time for auditors to focus on higher value-added activities.
2 Enhancing decision-making: By providing access to large amounts of data and information in real-time, ChatGPT can help auditors make informed decisions and detect potential risks.
3 Improving communication: ChatGPT can help to facilitate communication between auditors, stakeholders, and other departments, improving collaboration and increasing efficiency.
4 Enhancing reporting: By providing clear and concise information, ChatGPT can help improve the quality and accuracy of internal audit reports.
5 Providing insights: ChatGPT can provide insights into complex data, identify trends and patterns, and support data-driven decision-making.
Overall, ChatGPT has the potential to transform the internal audit process by providing a more efficient and effective solution to support decision-making and drive better outcomes.”
What risks could ChatGPT create for organisations?
“ChatGPT and other language models pose several potential risks for organisations, including:
1 Data privacy and security risks: ChatGPT relies on large amounts of data to operate, which could be sensitive or confidential information. The storage and handling of this data must be properly secured to avoid potential breaches.
2 Bias and discrimination: Language models like ChatGPT can perpetuate existing biases and discrimination present in the training data, leading to discriminatory decisions or actions.
3 Misinformation and errors: ChatGPT is only as accurate as the information it was trained on, and may not always provide correct or accurate answers. This could lead to decision-making based on incorrect information.
4 Dependency and over-reliance: Organisations may become overly reliant on ChatGPT, potentially reducing their ability to think critically and make decisions independently.
5 Job displacement: Automation of certain tasks through the use of ChatGPT could lead to job displacement and workforce changes.
Organisations must carefully evaluate and manage these risks when implementing language models like ChatGPT to ensure they are used responsibly and ethically.”
Can you draft a policy to enable a company to benefit from ChatGPT and control the risks it creates?
“Sample ChatGPT Policy:
Purpose: The purpose of this policy is to establish guidelines for the responsible and ethical use of ChatGPT within the company to maximise its benefits while minimising potential risks.
Scope: This policy applies to all employees, contractors and third-party partners who use or interact with ChatGPT in the course of their work with the company.
Guiding Principles:
• Respect data privacy and security: Employees must ensure that sensitive or confidential information is not shared with ChatGPT and that appropriate measures are in place to secure data related to ChatGPT.
• Avoid discrimination and bias: The company is committed to ensuring that ChatGPT does not perpetuate discrimination or bias in decision-making or output.
• Minimise misinformation and errors: Employees must be mindful of the limitations of ChatGPT and take appropriate steps to verify the accuracy of its responses and output.
• Avoid over-reliance: Employees must not rely solely on ChatGPT and must be able to think critically and make decisions independently.
• Manage workforce changes: The company recognises the potential for job displacement through the use of ChatGPT and will work to manage any workforce changes that may result in a responsible and ethical manner.
Responsibilities:
• Employees: Employees are responsible for using ChatGPT in accordance with this policy and reporting any violations or concerns.
• Management: Management is responsible for enforcing this policy and ensuring that employees receive training and support to use ChatGPT responsibly and effectively.
• IT and security: IT and security teams are responsible for ensuring that appropriate measures are in place to secure data related to ChatGPT and for monitoring the use of ChatGPT for compliance with this policy.
Enforcement: Violations of this policy may result in disciplinary action, up to and including termination of employment. The company reserves the right to take legal action as appropriate to address violations of this policy.
This sample policy provides a framework for companies to use ChatGPT in a responsible and ethical manner. It is important for companies to customise this policy to fit their specific needs and requirements, and to regularly review and update it to ensure that it remains relevant and effective.”
I'm planning to audit culture in my organisation. What questions should I ask?
“Here are some sample questions that you can ask when auditing culture in an organisation:
1 What is the company’s mission, vision and values, and how do they influence the company’s culture?
2 How are employees engaged and involved in shaping and reinforcing the company’s culture?
3 How does the company’s culture impact employee behaviour, motivation and performance?
4 Are there any cultural differences between different departments or teams within the company?
5 How does the company’s leadership and management style contribute to the company’s culture?
6 How does the company’s culture support diversity, equality and inclusion, and what initiatives are in place to address any cultural challenges?
7 How does the company’s culture support innovation and creative thinking?
8 How does the company’s culture impact customer satisfaction and loyalty?
9 Are there any cultural issues or challenges that employees have raised or that management is aware of?
10 How does the company measure the impact of its culture on employee satisfaction, productivity and business outcomes?
These questions can provide a starting point for auditing culture in your organisation and help you to identify areas of strength and areas that may need improvement. It is also important to consider the specific needs and objectives of your organisation when developing your audit plan.”
I chair an audit committee in a large organisation. What questions should I ask thechief audit executive (CAE) about ChatGPT?
“As a chair of an audit committee, you can ask the following questions to your chief audit executive (CAE) about the implementation of ChatGPT in your organisation:
1 How does the use of ChatGPT align with our organisation’s audit strategy and objectives?
2 What is the approach and process for assessing the reliability and accuracy of ChatGPT’s outputs?
3 How will ChatGPT be integrated into the overall internal audit process and what role will it play in the risk assessment and audit planning process?
4 How will ChatGPT’s outputs be validated and what measures will be put in place to ensure the quality and consistency of the information generated?
5 What are the potential risks associated with the use of ChatGPT, including the risk of errors and bias, and what measures are in place to mitigate these risks?
6 What policies and procedures will be put in place to govern the use of ChatGPT, including data privacy and security?
7 What training and development will be provided to internal auditors to help them effectively use ChatGPT in their work?
8 How will the effectiveness of the use of ChatGPT be evaluated and monitored over time?
These questions can help you better understand the benefits and risks of using ChatGPT in your organisation and assess the preparedness and capacity of the internal audit function to effectively use this technology.”
How can I improve the questions I ask ChatGPT?
“Here are some tips to help you improve the questions you ask ChatGPT:
1 Be clear and specific: Clearly state what you want to know and be as specific as possible. Avoid vague or open-ended questions.
2 Use proper grammar and spelling: Ensure your questions are grammatically correct and spelled correctly to improve the quality of ChatGPT’s response.
3 Provide context: Provide enough context for ChatGPT to accurately understand the context and intent behind your question.
4 Keep it simple: Avoid asking complex or convoluted questions. Instead, break down the question into smaller, simpler parts if necessary.
5 Be mindful of the model’s knowledge cutoff: ChatGPT has a knowledge cutoff, so keep this in mind when asking questions about more recent events or developments.
6 Be respectful and ethical: Avoid asking questions that are disrespectful, offensive or unethical. ChatGPT is designed to promote responsible and ethical use.
By following these tips, you can help ensure that you get the most accurate and helpful responses from ChatGPT.”
Is ChatGPT a good thing or a bad thing for human creativity?
“ChatGPT has the potential to both enhance and limit human creativity.
On one hand, it can provide new opportunities for people to express their creativity by offering new forms of communication and collaboration. For example, it can help people generate new ideas and content by providing information and inspiration.
On the other hand, it can also lead to a reliance on technology for problem-solving and decision-making, which could limit the development of critical thinking skills and reduce the opportunity for original thought and expression. Additionally, widespread use of language models like ChatGPT may lead to homogenisation of language and thought, reducing diversity in creative expression.
Overall, the impact of ChatGPT on human creativity will depend on how it is used and integrated into society.”
This article was published in March 2023.