An unintended consequence of full population testing on auditors’ professional skepticism

Publication Summary

This study examines how the use of full population testing (FPT), enabled by data analytics, affects auditors’ professional skepticism. While FPT improves the sufficiency (quantity) of audit evidence by testing entire populations, it often relies on client-internal data, which may lack appropriateness (quality) and be vulnerable to management manipulation. Auditing standards emphasize that more evidence cannot compensate for poor quality, making external evidence critical for fraud detection.

The authors hypothesize that auditors using FPT may exhibit attribute substitution bias, substituting their judgment of evidence sufficiency for appropriateness. This bias could reduce skeptical actions when external evidence later reveals fraud red flags. In an experiment with 125 auditors, results show:

Auditors using FPT were 52% less likely to act skeptically (e.g., inquire about inconsistencies or alert managers) compared to those using sample testing when confronted with an external fraud indicator.
FPT inflates perceptions of evidence appropriateness because auditors perceive it as more sufficient.
Contrary to expectations, presenting FPT results visually (graphs) versus in tables did not significantly worsen the effect.
Experience with FPT amplifies the bias, meaning more experienced auditors are even less skeptical after using FPT.

The findings highlight a critical unintended consequence of advanced audit technologies: auditors may underreact to fraud risks when over-relying on internal evidence tested via FPT. Audit firms and regulators should address this through training and quality controls, emphasizing the distinction between evidence sufficiency and appropriateness and reinforcing the importance of external evidence.