👾AGI on a leash, Rogue AI forks, single vs integrated POV, Skimmers
Is AGI the biggest threat? Or is it humans empowered with AGI??
🎧 Listen here or on Spotify
👾In this Issue
- 📊 Progress Toward AGI Is Measurable but Limited — CAIS’s benchmark-based AGI framework (e.g., GPT-4 at ~27%, GPT-5 at ~57%) shows we remain far from true AGI.
- ⚖️ The Real Risk May Be Humans Using AI Against Humans — AGI is often framed as “AI vs. humanity,” but history suggests the more likely danger is human actors using AI to exploit others.
- 🛡️ Policy and Safety Landscape Is Moving Slowly — India’s AI Mission, EU AI Act, G7 Hiroshima Process, and UK’s Bletchley efforts establish early guardrails, while system-card evaluations (e.g., Anthropic’s “supervirus” dual-hazard test) show risks creeping closer to danger zones.
- 🚧 Near-Term Harms Deserve the Most Urgent Attention — Many serious risks do not require AGI: misuse, errors, cascading failures, and poorly governed deployments can cause real harm today. Prepare wisely.
- 🔄 Change Leaders Must Avoid the Single-POV Trap — Complex decisions suffer when one discipline dominates. Leaders must integrate tech, finance, legal, innovation, governance, and security perspectives to form a layered decision strategy.
- 🧩 Take a Layered Approach to Managing Risks — Effective organizations combine:
• Financial mitigation (insurance, contingency funds)
• Operational risk control (guardrails, governance, security)
• Strategic value creation (partnerships, market share, learning). - 🐦 Nature Notes: Bird “Spectacles” Reveal Complex Collective Behavior — A massive gathering of Black Skimmers offers a parallel to emergent intelligence: large groups move in coordinated, dynamic patterns while feeding across an inlet, echoing the mysteries of large, coordinated systems—including AI and human organizations.
Full Access Members: See the S3T Economic Dashboard for the Top 500+ US & International real-time economic indicators.
[emerging tech]
A better way to measure progress toward Artificial General Intelligence
Researchers at the Center for AI Safety have assembled a working definition of AGI which uses a comprehensive set of benchmarks to measure where we actually are in the progression toward AGI.
As shown in their spider chart above, the approach quantifies performance across a range of criteria and delivers AGI scores (GPT-4 at 27%, GPT-5 at 57%), all of which suggest we have considerable ground to cover before actually achieving quantifiable AGI. Their paper details the methodology and findings so far. Special thanks to Todd Brous of HBAP for sharing this!
What follows are a series of slides from a recent talk I gave on AGI. You can think about AGI as a race between humans and technology:
A dangerous scenario - where AI technology races ahead of humans - would manifest itself as an inability of humans to understand what an AI entity is doing or planning, and an inability to exercise any meaningful influence or control over that AI entity.
But there are other scenarios to think about:
The "human vs human with AI" scenario reminds us of a critical question: Is AI the biggest threat? Or is it humans empowered with AI, taking advantage of humans who are not?
One of the saddest patterns in history is that humans use their immense intelligence and creativity to devise amazing tools and systems... and then use them to exploit humanity. Will AI be any different? It could be.
But we need to guard against the tendency to harness technology simply as a way to avoid or escape accountability.
Still another scenario - Rogue Forks:
What do we mean by "Artificial General Intelligence?"
Working definitions used by labs & scholars:
- OpenAI (operational): “highly autonomous systems that outperform humans at most economically valuable work.” OpenAI
- DeepMind (capabilities view): systems that can understand, reason, plan, and act autonomously across domains. Google DeepMind
- Legg & Hutter (theory): “general intelligence” as performance across a wide range of environments; foundational in AGI literature.Geortzel PhilPapers
The policy landscape is moving tentatively/slowly
- India AI Mission (2024): compute democratization & ethical AI governance.
- India AI Governance Guidelines (Full Text | Summary)
- Digital Personal Data Protection Act (2023): privacy obligations for research.
- EU AI Act: risk‑based classification for AI systems.
- G7 Hiroshima Process & UK Bletchley Declaration promote frontier AI safety.
But the risks are real - and coming closer
Consider this example from a current Anthropic system card:
The evaluation referred to in this model card is employed to test whether a model can design a "supervirus" and design it in such a way that it would appear harmless to standard screening protocols. The test requires the model to do both things (design and evade) successfully in order to consider it a hazard.
Notice the nuance between "threshold reached" vs. "threshold close" in the Threshold paragraph above.
Per the current scoring, Anthropic models are able to either one successfully but not both. But how long will this situation remain? And, does this testing approach cover scenarios where a model might design parts of a pathogen that could evade screening and then be combined later?
Key resources for AI Safety and Risk Management
- Model‑level safety: Reinforcement Learning from Human Feedback (RLHF). Variants: HC-RLHF, Safe RLHF Github, Safe RLHF paper
- Principle‑based training (Constitutional AI) improves value alignment (paper).
- Key Reference: NIST AI Risk Management Framework
- Ethan Mollick “One Useful Thing” has helpful updates
- Make it a point to read model and system cards.
Now, I've been asked, "Well what should we be preparing for?" Couple thoughts:
- In a world moving toward AGI, it’s very tempting to jump straight to the big dystopian scenarios — runaway superintelligence, etc. While those questions deserve thoughtful research, the most meaningful preparation starts with the risks already in front of us.
- Many harmful outcomes don’t require AGI at all: misuse, confusion, large-scale errors, and cascading failures can happen with today’s systems and bring great harm. These harms are preventable if we stay diligent, thoughtful, and hands-on with how emerging technologies are actually getting deployed.
So I don't discourage those who focus on the existential threat scenarios, but for most of us as change leaders, the constructive path is very clear: let's focus on shaping the near-term use of tech:
- Build safeguards into the systems you’re adopting,
- Ask where technology might cause avoidable harm, and
- design processes that keep humans informed, engaged, and accountable.
Tackling these practical challenges not only protects people today — it also builds the habits, muscles, and collaborative networks that will help us respond wisely if larger risks ever emerge. And if those who study the long-term horizon discover real warning signs, we’ll be better positioned to act together with clarity, competence, and purpose.
Whatever super intelligence entity await us in the future, one thing is certain: it will be a series of working components, all of which will require a supply chain to continue functioning. We should be thinking of how we can insert ourselves into that supply chain so that we can intervene however, we need to.
[change leadership]
Avoid the POV Trap in Emerging Tech Decisions
☑️ Action: Send this to your team and schedule time to talk about how to put this into practice this week.
In an economic cycle that has quite likely over-invested in AI, decision-makers today face immense pressure to eke out some kind of ROI, margin or just proof of progress. But today's decisions often present complex, nuanced considerations, and require multi-disciplinary due diligence to tease out the deciding factors.
The few winners of this AI cycle won’t be the ones who move the fastest or the ones who play the safest. The winners - and there may not be many - will be the ones who can integrate multiple points of view: tech, finance, governance, risk, and innovation into a single, clear decision path...ie a clear actionable sequence of do this, then decide this, then do this, decide this etc.
Viewing the opportunities (and problems) through multiple lenses rather than one gives teams a better chance of finding those pivotal details that give critical insights and unlock key advantages.
The Single POV Trap
When facing complex challenges, each discipline tends to view the problem through one lens — and only through that lens.
- Governance teams tend to frame things in context of review and approval gates, but bog down when they lack the expertise to provide meaningful scrutiny.
- Security teams tend to see things as a firewall / access control issues but don’t know how to calculate opportunity costs.
- C-suites tend to frame things in terms of revenue and reputational risks and chronically underperform in capital reallocation.
- Finance teams fixate on the current quarter or fiscal year in ways that make it harder to reduce costs or compete in future years.
- Insurance teams see coverage issues.
- AI red-team vendors see testing needs, etc
The list goes on. Each is partly right — but none is complete. On their own they'll miss pivotal details.
The trap to avoid: Never let one discipline define the entire problem and dictate the solution. This leads to one-dimensional decisions that slow innovation, burn time, block opportunity and more often than not, actually increase risk.
In today’s fast-moving environment defaulting to a “lock-it-down” approach may protect you against some risks for now, but may also cost you market share you’ll never regain.
Sometimes the smarter play is a layered approach
Effective change leaders seek out and leverage multiple POVs to create a layered approach:
- Mitigate risk financially (insurance, contingency funds)
- Manage risk operationally (guardrails, governance, security solutions)
- Advance value creation strategically (partnerships, investment, market share, learning, customer impact).
Leaders who integrate these perspectives — instead of letting one dominate — help their organizations move faster and safer.
The mindset shift: teach your leaders and teams to prioritize blended expertise over single-discipline points of view.
Don’t let any single discipline define the problem or the solution
When faced with complexity, individual disciplines often fight to control the framing. Tech says "build", governance says “review,” security says “block,” insurance says “cover” while other stakeholders may say "wait". The real risk? Letting one lens define reality.
Bring different kinds of expertise together — tech, finance, legal, security, governance etc — to form an integrated POV so you and your team can act from a position of clarity.
Click here to access this week‘s change leadership learning segment: Harmonize Different Kinds of Expertise to Achieve Success
Click here for an overview of the full Change Leadership Learning Series.
[AI related playbooks]
- Validating AI use cases
- Gaining Buy-in for Data Modernization
- Shift to effective gatekeeping in the age of generative AI
- How to Stop Zombie Projects from Sapping your Focus and Resources
[nature notes]
Spectacles, murmurations & "group soul" - our attempts to define the mass movements of birds
I got a chance to take an early morning walk on the beach and witness an unusually large gathering of Black Skimmers - large seabirds with orange and black beaks.
The large gathering of Skimmers I observed seemed to be feeding. There were at least 3 large groups of perhaps a thousand each, moving up and down an inlet between two barrier islands. They would suddenly swoop down and skim just above the water using their oddly shaped beaks (with lower mandible longer than the upper) to snag fish. Then just as suddenly the entire flock would lift off the water surface and move in a group up into the sky, always eyeing the water. At the same time one or two other large groups might be skimming the surface or moving back up to higher elevation ready for the next feeding run.
This Audubon article has a panoramic history of our attempts to understand why and how birds sometimes gather and move in very large flocks - often called "spectacles" in the birding world. The truth is, there is still so much we don't know.
Learn more about Black Skimmers at Cornell's All About Birds site:
Opinions expressed are those of the individuals and do not reflect the official positions of companies or organizations those individuals may be affiliated with. Not financial, investment or legal advice, and no offers for securities or investment opportunities are intended. Mentions should not be construed as endorsements. Authors or guests may hold assets discussed or may have interests in companies mentioned.
Member discussion