Oct 16, 2026 6 min read guardrails

The new audit trails & attestation requirements - compliance gets harder...

S3T PodCast 101626

0:00

/911.0204081632653

AI Governance will soon require collection of evidence

And by collection of evidence, we don't mean just meeting minutes for committees reviewing AI risks. Teams designing agentic platforms will need to not only have realtime controls, but also have effective ways of logging and preserving evidence about what those realtime controls actually did.

New finance sector guidance this week along with reports of FTC subpoenas are raising the importance of documented oversight, incident records and the ability to prove that day to day operations are complying with policies and regulations. Audit requirements are about to get way more fine grained and detailed. Classic approaches will no longer work.

The new audit trails

Traditional audit trails sought to prove that we maintained control of our systems, and that we kept them in compliance and followed the rules. Companies collected and maintained what auditors call "management evidence" that could be used in an audit to prove compliance.

AI has a different set of flows that traditional controls and management evidence collection approaches don't map to cleanly. These flows track through the process of a person writing a prompt, the AI system adding additional instructions to that prompt, breaking the request down into subtasks and handing them off to different models or agents perhaps across different environments, perhaps reading and processing data, returning answers or datasets or results. All of this activity in many cases will be subject to policies, privacy commitments, regulations. Someone at some point will have to attest to whether or not this process followed the rules and expectations. Was the users data ever shared? Was protected data or systems ever exposed to entities outside of US jurisdictions? Who or what could read the prompt or the proceedings at every step of the process? If agents were involved were they complying with the identity management and security protocols as they performed their tasks?

All these questions will need to be answered. All of the workloads passing through an AI system will need to stay on their respective tracks and be handled properly. Evidence of what did and did not happen will have to be retained for attestation purposes.

Clearly a redesign of the typical patterns of compliance software will be necessary.

Already new evidence collection requirements are starting to emerge. One recent wake up call: the FTC has subpoenaed OpenAI and Anthropic for evidence of security measures, privacy policies, incident records and more.

Likewise some new entrants are already designing for this more complex AI audit trail. Near‘s Private Inference with Attestation offers a glimpse of what to expect, and how incumbents will need to redesign their approaches:

(This is not an endorsement of Near.ai, cited to provide an example of what kinds of offerings are emerging.)

Companies in healthcare and finance will likely need to move the fastest.

The Financial Stability Board just released its final report on Sound Practices for Responsible Adoption of Artificial Intelligence (AI). The report recommends the collection and retention of "evidence of governance and control".

💡

"The financial institution’s risk appetite and tolerance guide decisions on AI use, selection and as well as the controls needed to monitor, manage and mitigate associated risks. These controls may include effective oversight, evidence of governance and control - particularly for highly autonomous AI - and clear escalation protocols." - FSB

Takeaways for the IT & Business Partnership

To put this in perspective of the traditional IT & Business partnership...there seems to be a growing expectation that the Business will increasingly develop AI solutions, IT will have to make them reliable and secure....is this true? vibe coding etc?

Maybe. Such a paradigm shift could end up being one of the most defining tensions in modern enterprise technology. Rather than a clean, peaceful handover, current research and industry experience show this shift creates a high-stakes friction zone.

Either way the talent formula is going to be crucial:

Look for engineers and analysts that understand compliance but also have AI / agentic experience preferably in regulated industries.
Legal teams need to bring in outside or tech advisors who understand this emerging space - and review all agreements, as well as terms and conditions of products and partner / vendor arrangements.

We are already seeing that generative AI tools and "vibe coding" (using natural language to prompt LLMs into generating functional application code) are drastically lowering the barrier to software creation.

Decentralized Creation is Exploding: Line-of-business (LOB) builder teams in finance, ops, marketing etc are no longer willing to wait in 10-month IT queues. They are using natural language to spin up custom automations, internal tools, data pipelines, and specialized AI agents.
IT's Role is Pivoting: IT is shifting away from being the primary build factory and toward being an enabler, platform steward, and governance provider. Instead of writing the business logic, IT configures the safe sandbox (identity, API gateways, policy engines) in which business-authored logic runs.

However, "Business builds, IT fixes" is a toxic operational model if executed as a toss over the wall handoff. When the business hacks together a prototype and throws it over the wall for IT to "make enterprise-grade," IT becomes an overwhelmed bottleneck, and technical debt compounds exponentially. Not to mention security risk. We have Vibe coding. But is there such a thing as "Vibe Reliability" or "Vibe Security"?

Short answer: No.

"Vibe coding" works because probabilistic text generation excels at syntax, boilerplate, and happy-path business logic. However, security, reliability, and system resilience are deterministic, non-functional requirements. You cannot prompt-engineer your way around edge-case failure modes, race conditions, memory leaks, or dynamic attack vectors.

Security requires zero-trust architecture, threat modeling, least-privilege access control, and continuous vulnerability scanning. An LLM will gladly generate code that executes a SQL query or calls an external API without sanitization for the prompt author who doesn't explicitly know to guard against it. Gartner predicts that by 2030, more than 40% of enterprise will suffer security incidents driven by unvetted shadow AI deployments and loose security controls and permissions.

The Emerging Model: Governed Platforms

Instead of cleaning up mess after the fact, IT teams should get ready to provide a curated platform sandbox approach:

Patterns aka "Paved Roads:" Pre-approved, secure connectors to enterprise databases and SaaS tools.
Automated Guardrails: CI/CD pipelines that run static security analysis (SAST) and compliance checks automatically before any vibe-coded solution touches production data or moves into prod environments.
Observability by Default: Centralized logging and telemetry embedded into the platform so IT teams maintain full visibility into what business-built agents are doing.

🤖AI Fatigue ...People aren't just scared of it, they may be getting sick of it.

Consider John Stewart's brutal takedown of the recent pageantry between the Trump administration and its alleged funders in Silicon Valley. Or this fiery sermon about hyperscalers.

Balanced perspective: AI is definitely changing how people work. Business large and small, including individual proprietors are using ChatGPT, Claude, Copilot agents to get work done. But AI also has half the market share of GLP-1s as Azeem Azar noted:

"...as of August, just 4.5% of U.S. consumers had an active paid personal subscription to ChatGPT, Gemini, or Claude" - A16Z The Top 100 Gen AI Consumer Apps
To put this in perspective 11% of US adults currently take GLP-1s.

These point contribute to the worries about the possibility we are witnessing an overbuild of AI capacity.

🚂 A short history of overbuilds

Overbuilding has a history—even when the underlying opportunity is real.

Railroads, 1870s: New projects outpaced demand, depressing investment returns. Railroad financing failures helped ignite the Panic of 1873 and the ensuing economic contraction. Banking Panics of the Gilded Age 1863-1913
Telecommunications, late 1990s–2001: Optimistic forecasts drove excessive investment in networks and equipment. The subsequent spending reversal contributed to the 2001 recession and slow recovery—even as internet usage continued to develop. See What Investment Patterns across Equipment and Industries Tell Us about the Recent Investment Boom and Bust
U.S. housing, 2000s: Excess construction combined with risky mortgage lending and financial leverage. Falling prices and defaults turned the housing bust into problem for the entire financial system, triggering the Great Recession. See The Supply Side of the Housing Boom and Bust of the 2000s

The lesson for AI: growing demand alone does not guarantee profitable capacity. Build costs, utilization, pricing and debt obligations determine whether the investment pays off in time to avoid a financial reset.

Opinions expressed are those of the individuals and do not reflect the official positions of companies or organizations those individuals may be affiliated with. Not financial, investment or legal advice, and no offers for securities or investment opportunities are intended. Mentions should not be construed as endorsements. Authors or guests may hold assets discussed or may have interests in companies mentioned.