is

Amsterdam, 22 November 2016

Are auditors allowed to learn from their mistakes?

FAR Masterclass on ‘Designing audit firm environments for sustainable learning from errors’

‘For me this Masterclass verifies why auditors in the field and scientists need to work together:
we need to share knowledge and test whether that knowledge is useful in the field.’

– participant from the Masterclass who works for a Big 4 company –

The topic of the first FAR Masterclass is how we as an audit profession deal with making mistakes. Are we allowed to learn from our mistakes? This interactive Masterclass was a classic example of an effective conversation between practitioners in the field and scientists, as the discussions helped both parties forward and led to new insights. Everyone left the room filled with enthusiasm and energy. The four most important take-aways of the Masterclass were:

  • A safe learning environment is a requirement for an increase of audit quality.
  • Judgment quality is related to whether or not individuals have been allowed to learn in the past and whether they have learned how to reflect on those mistakes.
  • The years of experience and the amount of confidence of an auditor do not predict the accuracy of the judgment nor the number of mistakes made by that person.
  • Your own behavior is crucial for creating that safe learning environment! Read what you can do in the following paragraphs.

The traditional image within the audit profession is that clients make mistakes and the auditor detects these misstatements. However, in this Masterclass we did not focus on the mistakes made by the clients, but the mistakes made by the auditors themselves. How do we as a profession deal with mistakes? Are we allowed to learn from mistakes? Professor Wim Gijselaers, Professor Roger Meuwissen and PhD student Therese Grohnert (Maastricht University) led the Masterclass which resulted in an effective conversation between scientists and practitioners in which both sides learned from each other by sharing knowledge and experiences.

At the beginning of the Masterclass participants were divided over four tables. Two tables with auditors employed at firms like Mazars, EY, SNS Reaal, KPMG, Audit Empire, PwC and Baker Tilly Berk. Researches of several Dutch universities sat at the third table. Policymakers and regulators (NBA and AFM) took a seat at the fourth table. Distributing the participants over four tables underlines the different perspectives present within the profession, raised awareness of these differences and fueled the discussion.

‘How many mistakes do we actually make as auditors?’ was the question Wim, Roger and Therese asked the participants. The participants seemed pessimistic as they expected that merely 20% of the auditors provide a correct judgment. Research among 225 auditors spread over different locations within one of the Big 4 firms gave the following results (see table 1, thesis Therese Grohnert). Hence, the actual percentage of auditors (31.0% – 47.7%) who used information and came to a correct judgment was larger than the participants expected.

Table 1: The distribution of 225 auditors (in %) among the four quadrants regardless of their location

Accuracy Judgment¹
No Yes
Informed Judgment² No 13.2%-43.1% 12.1%-26.3%
Yes 13.2%-13.8% 31.0%-47.4%

¹  Informed Judgment: did participants access the necessary (financial) information to make a judgment that fits the current situation?
²  Accuracy Judgment: did participants recognize that the debtors in the case had a different likelihood of settling their open positions?

The participants were shocked by the percentage of their colleagues that ‘speculate’ or made ‘intuition-based’ judgments (25.3% – 69.4%). ‘I have a pessimistic view of the world, but I was hoping for a better score’ were the words of a participating regulator. At this point the question is: Why do these differences arise? The participants provided several explanations, such as the organizational culture, ‘No-no’s are people who are lazy’, ‘When you have to deal with deadlines, you tend to listen more to the experience of others’, ‘People are more willing to gamble when they work independently, compared to when they work in a team’ and the level of experience. The speakers immediately spoke up on the last issue, since their research revealed that an experienced auditor (12-15 years experience) uses less information and does not provide a correct judgment more often. At the same time this experienced auditor is more confident than less experienced auditors (thesis Therese Grohnert). Additionally, the confidence that auditors have in their own judgment is not correlated with the amount of information they have at their disposal. People who judged using their intuition were namely the most confident of all auditors (thesis Therese Grohnert). In other words, people are not able to properly assess themselves whether their judgment is correct.

Auditors who have participated in a “professional skepticism” training or who use a checklist do not make fewer mistakes in their judgment, yet they do become more confident about their judgment.

 

Learning influences audit quality. Learning is not something you do only during a course or training. No, it is an ongoing process that involves learning from day-to-day events. To be able to adapt to the ever changing environment you must have the capacity to learn. The study of Wim, Roger and Therese mentioned three workplace learning mechanisms that have a positive impact on audit quality (see figure 1).

Figure 1: Overview of the three workplace learning mechanisms that have a positive impact on audit quality

‘Judgment quality is gerelateerd of mensen hebben kúnnen leren en hebben leren reflecteren’ (prof. Wim Gijselaers)

We can conclude that learning affects audit quality, but what is the current learning environment within the accountancy profession? Are we allowed to make mistakes or learn from them? Research shows that junior auditors with up to three years of experience who previously made a significant error do not tell their manager about a second mistake in order to prevent negative consequences. This effect disappears when this junior auditor works in an environment where learning from mistakes is openly encouraged (thesis Therese Grohnert). In other words, a safe learning environment stimulates learning and ensures higher audit quality.

Auditors who have made mistakes earlier indicate that they are more willing to hide these mistakes, compared to the auditors who have not made any errors yet. Apparently these auditors have learned that it is not smart to admit mistakes.

What can you do to create a safe learning environment? First you have to create trust. Do not put someone who admits making a mistake down. Ask ‘What do you think?’ on a regular basis. ‘Do not tell young employees too often that the things they notice and mention are not important’ (Bouwens and Bik, 2016). If you ask questions it offers junior auditors the opportunity to share their observations. Provide people with possibilities for self-development, and ‘we need to challenge people’. ‘We need to start talking again. If you want to deliver quality, you need to talk to each other and exchange information’. Lead by example and inform your colleagues when you made a mistake. Moreover as an experienced auditor, you should be aware of the effect your presence has on more junior colleagues (Nelson, Proell and Randel, 2016). The latter issue is also one of the topics that Wim, Roger and Therese will investigate further in the by FAR initiated research: ‘Moving Forward audit teams: Designing firm environments for sustainable learning from errors’. This research studies the impact of changing and hierarchical team structures in audit teams and the traditional image that auditors do not make or are not allowed to make mistakes. If you want to provide input on this issue please feel free to contact Therese Grohnert (t.grohnert@maastrichtuniversity.nl).

In the following books / articles you can read more about this subject:

  • ‘Thinking fast en slow’ by Daniël Kahneman.
  • ‘How company culture shapes employee motivation’ by L. McGregor en N. Doshi in Harvard Business Review (25/11/2015).
  • ‘Team-Oriented Leadership, Audit Risk and Auditors’ Willingness to Raise Audit Issues’ by M.W. Nelson, C.A. Proell & A. E. Randel in The Accounting Review: November 2016, Vol. 91, No. 6, pp. 1781-1805.
  • ‘How an accounting firm convinced its employees they could change the world’ by B. Pfau in Harvard Business Review (6/10/2015).
  • ‘Zwijgen is zilver, spreken is goud? De invloed van leiderschap op audit kwaliteit’ by Jan Bouwens en Olof Bik in Accountant.nl, Oktober 2016.
  • ‘Judge | Fail | Learn – Enabling Auditors to Make High-Quality Judgments by Designing Effective Learning Environments’ by Therese Grohnert (Maastricht University). This thesis will appear on the FAR website in the spring of 2017.

‘Judgment quality depends on whether or not individuals were allowed to learn and
whether they have learned how to reflect on their mistakes.’

– Professor Wim Gijselaers –

 
 
 
 

Share This