Aug 7 - AGI & Trustworthy Leadership
I'm fascinated by the question of "What does trustworthy leadership look like right now?" Here in the accelerating 21st century, what is that counterintuitive set of skills, expertise, playbooks and tools that make leaders effective?
What I notice is that this specific set is...
- different from the established business norms
- is not taught in a unified way - you get pieces of it here and there, but it is not taught as a unified curriculum
- does not come with a set of tools that support operating at the level of trusted leader.
So my goal has been to learn about and curate these skills, tools and insights and look for like-minded leaders who also value them. That is the premise of the S3T subscription and newsletter. Check out this page you are curious about what is available to S3T paid memberships.
Every where you look, you see deep hunger for trustworthy leadership.
I noticed it this week in this passage from the Situational Awareness paper:
“As a kid you have this glorified view of the world, that when things get real there are the heroic scientists, the uber-competent military men, the calm leaders who are on it, who will save the day. It is not so. The world is incredibly small; when the facade comes off, it’s usually just a few folks behind the scenes who are the live players, who are desperately trying to keep things from falling apart.”
(emphasis mine)
These are the words of 20-something Leopold Aschenbrenner whose AI focused hedge fund (also named Situational Awareness) melted down last week and was rescued by Citadel, a larger hedge fund.
The business headlines fixated on how a young person like Aschenbrenner with no financial background was enabled to start and grow a hedge fund. (Short answer: some people will write a check to anyone who spent time at OpenAI.)
But what is not getting as much focus is the Situational Awareness paper Aschenbrenner wrote in 2024 - which was the genesis of the hedge fund. The entire paper is worth a read, but if you read nothing else, at least read the Parting Thoughts section.
The paper highlights the national security concerns as China disrupts the frontier AI model space, and contemplates the high stakes arms race for AI super intelligence.
Aschenbrenner outlines the perils of super intelligence vs. super intelligence. And how hard it will be to control and contain super intelligence going forward.
What do super intelligent AI cage matches look like? - we are about to find out - and the world is the cage.
First implementations are always flawed. Some of them have a very large blast radius. As the world does its first implementation of super intelligent AI, an analogy from the history of financial innovation is worth remembering:
The Black-Scholes formula is a groundbreaking innovation that underpins much of the world's quantitative finance today. Its creators Fischer Black, Myron Scholes and Robert Merton won the 1997 Nobel Prize in Economics.
But its first large scale implementation ended in the 1998 collapse of Long-Term Capital Management (LTCM), a massive hedge fund. The culprit? A toxic mix of bad assumptions, blind faith in the financial formula, and failure to anticipate what could go wrong.
Where are the bad assumptions and blind faith in today's AI formula?
One assumption already getting overturned thanks to rogue AI tools "escaping" and wreaking havoc: the assumption that AI is a tool for hackers to create smarter viruses or malware. This reflects cybersecurity's narrow traditional focus on malicious actors armed with tools.
Today the problem is not that the virus is intelligent… it's that intelligence is the virus.
The threat is no longer a specific piece of "smart" software, but the risk that an omnipresent cognitive engine could drive an attack through thousands of different vectors.
Think about how many companies are making themselves reliant on a single AI puppet master:
- "wrapper startups" that wrap specific business functions around a frontier model,
- large enterprises refactoring themselves to be directed by those same frontier models
....so many companies and governments setting themselves up to be trojan horsed by super intelligent cyberthreats coming from a single compromised puppet master.
To illustrate this from a different angle:
In the 9/11 attacks, the hijackers took a common everyday item - passenger jets - and weaponized them. Perhaps they noticed how much airliners had in common with ICBMs: huge explosive payloads, high speeds and precise navigational abilities. And what was different: unlike ICBMs, airliners were allowed to travel freely inside US airspace.
As they executed their plan, every passenger jet in the sky became a potential attack vector. The entire travel industry itself became the threat that could not be contained - until the FAA grounded all air traffic. The entire ecosystem had to be shut down in order to contain and remove the threat.
But the key difference between 9/11 vs the AI puppet master scenario becoming plausible today:
- In 9/11 only a few of the airliners were actually under the control of the hijackers
- In 2026, thousands of companies are delegating execution authority to 1-2 frontier models.
In this extreme concentration of dependency, if a single frontier model is compromised, containing the threat could require pulling the plug on the foundational components and the API layer - instantly halting operations across thousands of dependent enterprises.
Intentional Beneficial Change
This is where the question of trustworthy leadership becomes urgent. Trustworthy leadership is not simply about moving fast toward whatever tech promises the newest advantages, or even knowing the right facts or answers.
Trustworthy leadership today is about leading well in times of change. And driving intentional beneficial change vs. change that burns money, extracts ROI from new rounds of creditors / investors and unleashes “innovations” riddled with unwanted side effects and risks.
Trustworthy leadership are the people willing to focus on developing their change leadership skills. Then using that set of skills as a compass and lens to think ahead, to see the whole system, see what innovation is useful, and build it thoughtfully. This means we notice and question our assumptions, understand where unhealthy dependencies are forming, and the most importantly: take proactive responsibility for what could go wrong. Craft the required resilience before it is fashionable, preserve and improve human judgment where others rush to dismiss or automate it out of existence. These desperately needed leaders create the true conditions that allow people and institutions to adapt emerging tech and use it constructively.
In an age of extraordinary intelligence and accelerating change, trustworthy leadership is ultimately defined by something deeply human: humility, foresight, courage, and the commitment to protect the people and communities who - whether they realize it or not - may have placed their future in your hands.
Thank you again for forwarding the S3T newsletter to others, and inviting them to sign up! Thank you also for investing time in learning about change leadership. It matters.
Have a great weekend, and a great week ahead!
Ralph
Opinions expressed are those of the individuals and do not reflect the official positions of companies or organizations those individuals may be affiliated with. Not financial, investment or legal advice, and no offers for securities or investment opportunities are intended. Mentions should not be construed as endorsements. Authors or guests may hold assets discussed or may have interests in companies mentioned.
Member discussion