The Auditors Cannot Audit Themselves

PwC just got caught publishing four thought leadership reports out of its Middle East practice with fabricated citations, dead footnotes, and an academic study on Riyadh air quality that appears to have never existed. GPTZero ran the analysis, the Financial Times verified it, and one report scored as effectively fully AI generated. Another promoted a PwC framework called Citizen Pulse that researchers could find almost no evidence for outside the report itself. 

The full investigation is here: https://lnkd.in/g8YuGDpu and it is worth your time. 

What should stop you is not the sloppiness, it is what those documents were for. These were marketing pieces for PwC’s AI advisory practice, written by the firm that will show up to assess whether your AI governance program is real. And this is not one bad quarter for one firm. EY pulled a study in May. KPMG got caught last month. Deloitte refunded the Australian government last October. That is three quarters of the Big Four inside a year, all found the same way, all by an outside detection startup rather than by their own quality control. Which for decades has been the accounting sectors claim, we audit ourselves, that and independence which also is largely proving to be non-existent in light of not doing their due diligence for these recent IPO’s. 

Here is what I keep coming back to. Every one of those firms has a documented AI usage policy. Every one has a quality control process for published research. Every one would write you up for exactly this finding. The control existed on paper, nobody executed it, and nobody caught that nobody executed it, because there is no independent party checking the checkers. 

The Big Four audit everyone else. Who audits them? Peer review inside the same small club and a regulator focused on financial statements, not on whether the AI governance practice can govern its own AI. I have said before that AI governance without independent assurance is empty. This is what empty looks like. A citation check is the cheapest control in the building and it was not run, on the exact documents whose purpose was to prove the firm knows how to run controls. If you are buying AI advisory work right now, ask the vendor to walk you through how their last published report was fact checked, by whom, and what the evidence of that review looks like. Ask them to show you the artifact, not the policy. If they cannot produce it, you are not buying assurance. You are buying a logo, and the logo is currently getting caught. 

So here is my gut check for the week: if the firm assessing your AI controls cannot pass its own, what exactly are you paying for?