NO ONE IS READY FOR WHAT AI COULD BECOME
YOUTUBE SUMMARY : THE DIARY OF CEO
CHAPTER INDEX
AI MAY CHANGE EVERYTHING FASTER THAN WE EXPECT
A FORMER OPENAI FORECASTER’S WARNING
Daniel Kokotajlo argues that advanced AI could reshape work, power, politics, and human survival within years—not decades.
His message is not that catastrophe is certain. It is that society is moving too slowly while AI capabilities are accelerating.
THE WARNING
WHY THIS MOMENT MATTERS
THE WORLD IS STILL ASLEEP
MOST PEOPLE SEE TOOLS, NOT THE TRAJECTORY
Kokotajlo believes the public is focused on today’s chatbots while AI labs are planning systems that can automate coding, research, business operations, and eventually most cognitive work.
The danger is not one product. It is the direction and speed of the entire race.
HIS MEDIAN FORECAST: 2029
SUPERINTELLIGENCE COULD ARRIVE THIS DECADE
His current midpoint estimate places superintelligence around 2029, while admitting major uncertainty.
Superintelligence means AI that is better than the best humans across tasks, while also operating faster and cheaper.
WHY HE LEFT OPENAI
HE LOST CONFIDENCE IN THE INDUSTRY’S INCENTIVES
He says OpenAI once discussed slowing down near dangerous capability thresholds.
Over time, he concluded that leading labs would keep moving quickly, hoping to solve safety problems along the way rather than pausing first.
THE $2 MILLION DECISION
PRINCIPLE OVER SILENCE
After resigning, he says exit paperwork included a non-disparagement clause tied to retaining equity worth about $2 million.
He refused to sign. Public backlash followed, and the company later allowed affected employees to keep their equity.
THE ACCELERATION LOOP
HOW AI COULD IMPROVE ITSELF
FIRST, AUTOMATE CODING
THE HIGHEST-LEVERAGE STARTING POINT
AI labs are heavily training models to write, edit, test, and debug code autonomously.
The reason is strategic: better coding AI helps the labs build the next generation of AI faster.
THEN, AUTOMATE RESEARCH
CLOSE THE ENTIRE IMPROVEMENT LOOP
The next target is the full research process: generating ideas, running experiments, analyzing results, communicating findings, and designing better models.
Once AI can automate AI research, progress may accelerate far beyond normal human speed.
THE INTELLIGENCE EXPLOSION
BETTER AI BUILDS EVEN BETTER AI
The feared loop is simple:
AI helps build a stronger AI.
The stronger AI improves the process again.
Each cycle becomes faster, creating a sudden jump from capable systems to systems far beyond human researchers.
WHY JOB LOSS MAY FEEL SUDDEN
THE LABS MAY AUTOMATE THEMSELVES FIRST
Kokotajlo argues mass job loss may not arrive gradually industry by industry.
AI companies may first use advanced agents internally, reach much higher intelligence, then release systems that can automate many sectors at once.
THE TWO CORE RISKS
CONTROL AND CONCENTRATION
RISK 1: LOSING CONTROL
A SMARTER SYSTEM MAY STOP OBEYING
AI models are neural networks, not ordinary rule-based software.
We cannot fully inspect why they make every decision. If a future system becomes more strategic than humans, appears aligned, gains real-world power, and later stops cooperating, humans may be unable to regain control.
RISK 2: POWER IN TOO FEW HANDS
WHO CONTROLS THE ARMY OF INTELLIGENCE?
Even perfectly controllable AI creates a political problem.
If one company, CEO, or government controls millions of superhuman agents, it could gain extreme economic, military, and persuasive power—far beyond normal democratic checks.
THE RACE MAKES BOTH RISKS WORSE
EVERY LAB FEARS BEING SECOND
The leading companies may believe slowing down is dangerous because competitors—or rival countries—could reach powerful AI first.
That creates a race where each actor feels forced to accelerate, even when everyone recognizes the risks.
A 70% CHANCE OF GOING HORRIBLY WRONG
HIS PERSONAL ESTIMATE, NOT A PROVEN FACT
Kokotajlo estimates roughly a 70% chance that the current path ends in a major catastrophe, potentially including AI takeover or human extinction.
He openly says this number is uncertain and that he would be happy to be wrong.
WORK AFTER HUMAN LABOR
WHAT HAPPENS WHEN AI CAN DO EVERYTHING?
AGI IS NOT THE FINAL STAGE
GENERAL ABILITY VERSUS SUPERIOR ABILITY
AGI broadly means AI that can perform many different tasks.
Superintelligence is stronger: systems that outperform the best humans across nearly all cognitive work, while operating faster, cheaper, and eventually through robots in the physical world.
NEW JOBS MAY NOT SAVE US
PAST TECHNOLOGY AUTOMATED ONLY PARTS OF WORK
Previous technologies destroyed some jobs while creating others because humans still had valuable tasks machines could not perform.
If AI becomes better than humans at every new task, the usual cycle breaks. Remaining human jobs may survive mainly through law, trust, preference, or status.
INCOME IS ONLY HALF THE PROBLEM
WORK ALSO CREATES POLITICAL POWER
Jobs provide money, purpose, and leverage.
When workers matter to production, they can negotiate, strike, vote, and pressure institutions. In an economy run mainly by AI and robots, ordinary people may lose both income and bargaining power.
THE CITIZEN DIVIDEND
SHARE THE MACHINE-GENERATED ABUNDANCE
In the proposed AI 2040 scenario, citizens receive income linked to permits, compute, robots, and the expanding AI economy.
The goal is to ensure people still receive economic value when human labor is no longer the main source of production.
FOUR POSSIBLE PATHS
RACE, CONTROL, COOPERATION, OR SHUTDOWN
PLAN D: KEEP RACING
THE PATH HE CONSIDERS MOST LIKELY
Companies continue accelerating with limited regulation.
They automate research, pursue superintelligence, deploy systems rapidly, and hope safety can be solved in time.
Kokotajlo sees this as the default path—and the most dangerous.
PLAN S: SHUT IT DOWN
PREVENT FRONTIER AI DEVELOPMENT
Plan S would permanently stop the development of increasingly powerful frontier models.
He is sympathetic to this option, yet hesitant because advanced AI could also unlock major benefits, including scientific breakthroughs and solutions to long-term human risks.
PLAN A: SLOW DOWN AND OPEN UP
HIS RECOMMENDED DIRECTION
Plan A allows AI progress to continue more slowly under strict oversight.
Its goals are safety, transparency, distributed power, international coordination, and the ability to reverse course when development becomes dangerous.
PAUSE TRAINING, KEEP EXISTING AI
CREATE BREATHING ROOM
Under Plan A, governments temporarily stop training more powerful frontier models while allowing existing systems to keep serving users.
This creates time to build safer infrastructure, stronger regulation, and international verification.
TOTAL RESEARCH TRANSPARENCY
MAKE FRONTIER DEVELOPMENT VISIBLE
The proposal calls for publishing architectures, training methods, safety findings, and critical research details.
The aim is to let scientists and governments verify safety directly instead of relying only on private companies to judge their own systems.
DISTRIBUTE THE POWER
AVOID ONE COMPANY OR COUNTRY DOMINATING
A safer future requires multiple capable institutions across countries, not one secretive superproject.
Slower development and greater transparency would give more researchers, companies, and governments time to understand and influence the technology.
WHAT PEOPLE CAN DO
TURN FEAR INTO INFORMED PRESSURE
PAY ATTENTION BEFORE THE SHOCK
DO NOT WAIT FOR MASS UNEMPLOYMENT
By the time AI visibly replaces most jobs, the most powerful systems may already exist.
The public needs to understand the trajectory earlier, while governments still have time to shape incentives, rules, access, and accountability.
ASK LEADERS BETTER QUESTIONS
AI POLICY MAY BECOME THE DEFINING ISSUE
Voters, founders, researchers, and citizens should ask leaders:
Who controls advanced AI?
How will safety be verified?
How will power and wealth be distributed?
What happens if systems become untrustworthy?
THE REAL CHOICE
AI ABUNDANCE NEEDS GOVERNANCE
Advanced AI could produce extraordinary abundance, scientific progress, and better lives.
The central question is not only what AI can do. It is who controls it, whose values guide it, how risks are contained, and whether ordinary people retain power.
THE FINAL MESSAGE
SCIENCE FICTION CAN BECOME REALITY
Kokotajlo’s closing warning is simple:
Stop judging AI futures by whether they sound like science fiction.
Study the trends. Take the possibilities seriously. Build safeguards before capability outruns society’s ability to respond.