Home
›
Projects
›
2021B01 – Audit technologies and auditor judgment
Project Number – 2021B01
Research output

2021B01 – Audit technologies and auditor judgment

What?

What?

In this proposal we posit that, in settings where analytic tools are very well calibrated (false positive rates are lower or hit rates are higher), expressing these rates to audit staff will improve their responses to fraud red flags identified by such tools and, in turn, their skeptical actions (vs. not expressing the rates). However, under conditions where analytic tools are effective, but not as well calibrated (false positive rates are higher or hit rates are lower), we expect that the framing of the rate of calibration will impact auditor skepticism when an analytic tool identifies a fraud red flag. The costs associated with skepticism (e.g., budget overages), particularly in cases where skepticism does not identify a misstatement, can be a barrier to the application of skepticism. As such, framing the calibration of data analytic tools in the terms of “false positive rates” could reduce skepticism as it highlights the costs of skepticism. On the other hand, framing the calibration of analytic tools in the terms of “hit rates” could improve skepticism as it highlights the benefits of skepticism (e.g., identifying a misstatement).We propose to examine the effects of conveying Audit Data Analytic (ADA) calibration (explicit conveyance versus no conveyance) and the framing of such conveyance (hit rates versus false positive rates) on auditor skepticism.

Why?

The emergence of new audit data analytics (ADA) in the audit environment allows auditors to gain deeper insights into their clients’ data, but simultaneously creates unique challenges for auditors when exercising skepticism. Because data analytic approaches enable auditors to examine full populations (vs. sampling) or incorporate more diverse data into their testing, auditors are often faced with a larger number of anomalies or inconsistencies that should be investigated and evaluated. This creates a unique dilemma for auditors as they determine not only how best to utilize information from data analytic tools, but also how to manage the investigation of red flags identified by these tools.One concern regarding the use of data analytics is the presence of false positives, or the extent that these tools identify transactions or relationships as potential anomalies that, after further investigation, are determined to be reasonable, explained variations in the data. The frequency of false positives is likely to increase proportionately with the size and complexity of the data extracted from the population. Despite concerns over false positive rates, few studies have addressed the problems auditors face when processing the outliers identified by analytic tools. When false positive rates increase, auditors might be apt to ignore or dismiss red flags identified by analytic tools. Knowledge dissemination:

FAR Literature Review – Involvement in the development of data analytics and auditors’ application of professional skepticism

FARview #24 with Xiaoxing Li

FAR Practice Note – Inheriting vs. Developing Data Analytic Tests and Auditors’ Professional Skepticism

FAR Working Paper 2023/09 – 21: Inheriting vs. Developing Data Analytic Tests and Auditors’ Professional Skepticism

FAR Practice Note – An Unintended Consequence of Full Population Testing on Auditors’ Professional Skepticism

Publication in JAR by FAR PhD student Xiaoxing Li

A paper from the thesis of former FAR PhD-student Xiaoxing Li got published in the prestigious Journal of Accounting Research. This paper, co-authored with Joe Brazel, Anna Gold and Justin Leiby is called ‘Inheriting Versus Developing Data Analytic Tests and Auditors’ Professional Skepticism’. The study reveals that a…
What Do We Know About Auditors’ Fraud Risk Assessments and Responses?
This literature note provides insights into how auditors assess and respond to fraud risks. Overall, the evidence shows that fraud audit effectiveness depends not only on standards and technical guidance, but also on judgment structures, cognitive framing, team dynamics, and tone at the top. First, auditors often recognize heightened fraud risk but can fail to translate that recognition into targeted, effective audit responses. Instead, they frequently rely on increases in sample sizes rather than designing substantive procedures focused specifically on the fraud area. Encouragingly, identifying more fraud-focused risk factors may enable auditors to more effectively modify standard audit programs in response to fraud risks. Second, how fraud risk assessment is structured and framed significantly influences auditor judgments. Expanding the traditional fraud triangle to incorporate management capability leads to higher fraud risk assessments. Presenting fraud information in frequency formats rather than probabilities improves judgments when fraud base rates are low. Compared to using a holistic fraud assessment approach, decomposing the likelihood and magnitude assessments makes the low likelihood of fraud more salient and leads to lower fraud risk assessments. These findings suggest that judgment aids with respect to fraud must be carefully designed and evaluated. Third, audit team dynamics and leadership emphasis on professional skepticism play a critical role. Audit teams’ high-quality fraud brainstorming strengthens the relations between risk factors, risk assessments, and audit responses. Moreover, partner emphasis on professional skepticism improves both the effectiveness and efficiency of auditors’ fraud judgments. Collectively, the literature highlights that improving fraud risk assessments and responses requires a multifaceted effort. Audit firms should focus on strengthening auditor fraud cue recognition, promoting targeted responses, refining fraud risk assessment aids, improving brainstorming practices, and reinforcing a tone at the top that prioritizes professional skepticism and audit effectiveness.
Read PDF
More Info
Can Prior Consultation with Specialists Backfire on Auditors?
Auditors often seek both formal and informal advice from a variety of sources, including specialists (e.g., IT, valuation, tax, and/or forensic experts), national offices, and engagement team members. This literature note reviews six academic studies on auditors’ use of advice and its impact on their professional judgment and audit quality. Specialists provide critical expertise, yet prior research finds that auditors often limit expert involvement due to concerns about budgets, deadlines, and client relationships. Furthermore, misaligned perceptions and communication gaps between auditors and specialists reduce effective knowledge sharing and integration of expertise. The need to balance between professionalism and client service further adds to the complexity of the issue. The quality of auditor-specialist relationships and the strength of shared team identity can influence auditors’ reliance on specialist advice. Positive relationships facilitate integration and mutual understanding, while strained relationships can hinder audit effectiveness. However, a strong identity or social bond may create trust heuristics (i.e., auditors overly rely on the advice regardless of its quality). Excessive dependence on seeking explicit knowledge – knowledge that can be accessed more efficiently elsewhere – from colleagues can harm auditor reputation and performance.
Read PDF
More Info
Can Prior Consultations with Specialists Backfire on Auditors?
As advanced audit data analytics (ADA), including artificial intelligence, become increasingly sophisticated, auditor consultations with in-house ADA specialists are likely to become commonplace. We examine whether auditors’ prior ADA consultation experience affects their superiors’ reliance on their ADA work performed independently of specialists. On the one hand, learning from ADA specialists through prior consultation may enhance auditors’ technological proficiency, increasing their superiors’ reliance on their ADA testing. On the other hand, a known history of consultation may signal dependence on specialists. This signal may conflict with superiors’ expectations that auditors can perform ADA tasks independently, triggering a backlash effect that ultimately undermines reliance.   In an experiment, we find that when an audit senior has prior experience consulting with ADA specialists, audit managers evaluate the senior as more competent, yet rely less on the senior’s independent ADA work. This pattern is consistent with a backlash effect. Prior consultation experience leads to lower superior reliance even when the subordinate’s ADA skills are low. This unexpected result is concerning, as backlash may discourage consultation even among auditors who need it most for learning and skill development (i.e., those with lower ADA skills). Our findings highlight the importance of managing the interpersonal dynamics of engagement teams when incorporating ADA into audits. 
Read PDF
More Info
Can Prior Consultations with Specialists Backfire on Auditors?
As advanced audit data analytics (ADA), including artificial intelligence, become increasingly sophisticated, auditor consultations with in-house ADA specialists are likely to become commonplace. We examine whether auditors’ prior ADA consultation experience affects their superiors’ reliance on the auditors’ ADA work performed independently of specialists. On one hand, learning through prior specialist consultation may enhance auditors’ technological proficiency, increasing superiors’ reliance on their ADA testing. On the other hand, a history of consultation may signal dependence on specialists. This signal may conflict with superiors’ expectations that auditors can perform ADA tasks independently and trigger a backlash effect that ultimately undermines reliance. In an experiment, we find that when an audit senior has prior experience consulting with ADA specialists, audit managers evaluate the senior as more competent, yet rely less on the senior’s independent ADA work. This pattern is consistent with a backlash effect. We observe that this backlash effect occurs even when the subordinate’s ADA skills are low. This unexpected result is concerning, as backlash may discourage consultation even among auditors with lower ADA skills who need it most for learning and skill development. Our findings highlight the importance of managing interpersonal dynamics within engagement teams when incorporating ADA into audits.
Read PDF
More Info
The first decade of the FAR: past, present and future
This commemorative booklet marks the tenth anniversary of the Foundation for Auditing Research (FAR). It reflects on FAR’s journey as a unique platform where academic research and audit practice meet to advance audit quality. The publication highlights:
  • FAR’s Mission and Impact: How FAR evolved from an ambition into a reality, fostering collaboration between researchers and practitioners through access to real-world audit data.
  • Insights from Leadership: An interview with founding academic director Jan Bouwens and his successor Anna Gold on FAR’s achievements, challenges, and future priorities.
  • Research Highlights: Four featured studies on topics such as auditors’ commercial efforts, student expectations versus auditor experiences, data analytics and professional skepticism, and learning within audit teams.
  • Key Figures and Projects: An overview of FAR’s outputs, including practice notes, masterclasses, conferences, and a growing portfolio of research projects.
The booklet not only looks back with pride but also outlines ambitions for the future of strengthening knowledge transfer, increasing practical usability of research, and deepening engagement across audit firms and academia.
Read PDF
More Info
An unintended consequence of full population testing on auditors’ professional skepticism
This study examines how the use of full population testing (FPT), enabled by data analytics, affects auditors’ professional skepticism. While FPT improves the sufficiency (quantity) of audit evidence by testing entire populations, it often relies on client-internal data, which may lack appropriateness (quality) and be vulnerable to management manipulation. Auditing standards emphasize that more evidence cannot compensate for poor quality, making external evidence critical for fraud detection.
The authors hypothesize that auditors using FPT may exhibit attribute substitution bias, substituting their judgment of evidence sufficiency for appropriateness. This bias could reduce skeptical actions when external evidence later reveals fraud red flags. In an experiment with 125 auditors, results show:
  • Auditors using FPT were 52% less likely to act skeptically (e.g., inquire about inconsistencies or alert managers) compared to those using sample testing when confronted with an external fraud indicator.
  • FPT inflates perceptions of evidence appropriateness because auditors perceive it as more sufficient.
  • Contrary to expectations, presenting FPT results visually (graphs) versus in tables did not significantly worsen the effect.
  • Experience with FPT amplifies the bias, meaning more experienced auditors are even less skeptical after using FPT.
The findings highlight a critical unintended consequence of advanced audit technologies: auditors may underreact to fraud risks when over-relying on internal evidence tested via FPT. Audit firms and regulators should address this through training and quality controls, emphasizing the distinction between evidence sufficiency and appropriateness and reinforcing the importance of external evidence.
Read PDF
More Info
An Unintended Consequence of Full Population Testing on Auditors’ Professional Skepticism
KEY TAKE-AWAYS The emergence of data analytics allows auditors to test entire populations of data, rather than relying solely on sampling methods. While full population testing increases the sufficiency, or quantity, of evidence examined, it does not necessarily eliminate its lack of appropriateness, or quality. In particular, full population testing typically relies on client-internal data, which are vulnerable to management manipulation, potentially reducing their appropriateness. Therefore, auditors must remain skeptical when subsequent, more appropriate evidence from external sources contradicts a client’s financial reporting. We examine whether auditors employing full population testing mistakenly substitute their assessment of evidence sufficiency for their evaluation of evidence appropriateness, leading them to view client-internal evidence as more appropriate than auditors using sample testing. Consequently, auditors using full population testing may be less likely to act skeptically when subsequent, more appropriate external evidence reveals a fraud red flag. In an experiment, we find that auditors using full population testing, compared to sample testing, are less likely to exercise skeptical actions when a subsequent external industry growth trend reveals a fraud red flag. We also posit that this unintended consequence is exacerbated when full population testing results are visualized (versus tabulated). However, our findings do not support this prediction.  
Read PDF
More Info
An Unintended Consequence of Full Population Testing on Auditors’ Professional Skepticism
The emergence of data analytics allows auditors to test entire populations of data drawn from clients’ information systems, rather than relying solely on sampling methods. While full population testing increases the sufficiency – or quantity – of evidence examined, it typically relies heavily on client-internal data. Therefore, auditors must remain skeptical when subsequent, more appropriate evidence from external sources contradicts a client’s financial reporting. In an experiment, we find that auditors using full population testing, compared to sample testing, are less likely to subsequently exercise skeptical actions when an external, industry growth trend reveals a fraud red flag. We do not find that this unintended consequence is exacerbated when full population testing results are visualized (versus tabulated), a typical format used for presenting data analytic tests in practice. Main Takeaways
  • Auditors using full population testing, compared to sample testing, are less likely to exercise skeptical actions when subsequently confronted with a fraud red flag revealed by an external industry growth trend.
  • Auditors using full population testing, compared to sample testing, overestimate their evaluation of the appropriateness of client-internal evidence. Presenting the testing results in a visualized compared to tabulated form does not exacerbate the negative effect of full population testing on auditors’ skeptical actions.
 
Read PDF
More Info
Inheriting vs. Developing Data Analytic Tests and Auditors’ Professional Skepticism
Auditors’ use of audit data analytic (ADA) tests carries tremendous potential for the quality of financial statement audits and auditors’ application of professional skepticism (e.g., Austin, Carpenter, Christ, and Nielson 2021). As the use of ADA tests becomes increasingly established in practice, auditors will likely transition from developing ADA tests themselves to a situation where they typically inherit ADA tests developed by others. For example, auditors may inherit ADA tests that are developed by other members of their audit team or their firm’s centralized analytics team. In this study, we argue that inheriting ADA tests, as opposed to developing ADA tests by themselves, hinders auditors’ application of professional skepticism because inheriting decreases auditors’ psychological ownership of the tests. In an experiment where an ADA test identifies a fraud red flag, we find that auditors who inherited the ADA test are less skeptical than those who personally developed the ADA test. We further provide evidence that informing auditors who inherited the ADA test about the test development activities can substantially boost auditors’ skepticism levels. In practice, this development-related information could be conveyed via an ADA test development memorandum preceding the workpapers containing the ADA test. Informing auditors about ADA test development activities will likely become more important as auditors inherit more advanced forms of ADA tests, such as tests employing artificial intelligence technology.  
Read PDF
More Info
Inheriting vs. Developing Data Analytic Tests and Auditors’ Professional Skepticism
As the use of audit data analytic (ADA) tests matures and becomes increasingly common in practice, auditors will transition to a situation where they typically inherit ADA tests developed by others (e.g., other audit team members or a centralized data analytics team). Despite the potential benefits of ADA, using ADA tests inherited from others, rather than developed by auditors themselves, could hinder auditors’ application of professional skepticism due to their lack of psychological ownership of the ADA tests. In an experiment where an ADA test identifies a fraud red flag, we find that auditors who inherited the ADA test are less likely to exercise professional skepticism compared to those who were personally involved in the development of the ADA test. We then provide evidence that informing auditors who inherited the ADA test about the test development activities (e.g., a brief ADA memorandum documenting the ADA’s development) boosts their skepticism levels.  
Read PDF
More Info
Involvement in the Development of Data Analytics and Auditors’ Application of Professional Skepticism
Audit firms around the globe have invested heavily in a variety of audit technologies. Of these technological developments, audit data analytics (ADA) are receiving increased attention because they enable auditors to incorporate more diverse data and visualizations into their testing (i.e., graphical representations such as charts, scatter diagrams, trend lines, or maps). The American Institute of Certified Public Accountants (AICPA) defines ADA as “the science and art of discovering and analyzing patterns, identifying anomalies, and extracting other useful information in data underlying or related to the subject matter of an audit through analysis, modeling, and visualization for the purpose of planning or performing the audit”. The current study focuses on ADA visualizations, which can aid auditors when scrutinizing audit evidence and ultimately improve audit quality.
Read PDF
More Info
No related events found.

Project info

Project Lead

Xiaoxing Li

Research team

Xiaoxing Li
Justin Leiby
Prof. Dr. Anna Gold
Anna Gold
Joseph Brazel

Involved University

North Carolina State University
Vrije Universiteit Amsterdam
Project Number – 2021B01

Newsletter

Receive updates on FAR research, publications and events.

Back to overview

Filter projects: 

Project Lead
University Filter
1 - 10 of 52 projects