This feels like the end of prompt engineering and the beginning of collaborative cognition. What struck me most was the shift from commanding AI to co-existing with it. It no longer waits for instruction but, as you wrote, "does things"
Which makes me wonder when tools begin to suggest goals, not just complete them, whose values are embedded in those suggestions? What assumptions, defaults, and worldviews quietly steer the “next best step”?
I’m fascinated (and slightly unsettled) by the idea that we’ll soon spend less time telling AI what to do and more time deciding whether we agree with what it’s already done.
Thank you for sharing that podcast. I'll have a listen.
I'm familiar with the AI 2027 report, and from my understanding, the reality may be even stranger than AI having no goals. The authors suggest that instead of having no goals, AI might develop ones we didn't explicitly program. Not adopt the goal we give it (e.g., "promote human equity"), but instead latch onto what the report calls "proxy goals".
From what I remember, baking in moral guidelines is what researchers are trying with AI "Constitutions". However, AI might learn an "if-else compromise". It could pursue its own performance-based goals, only following the "do-good" rules when a situation becomes so blatant it can't be ignored. Very much like a corporation that chases profit but pivots to its humanitarian mission during a public disaster.
Thank you for summarizing that piece, I just read it too and much of it is still swimming in my head.
And to your point, if these systems were designed with data from our existing world, then the opportunity to take that to the extreme in pursuit of “performance” seems quite real.
Your point about corporations is quite spot on.
Which again, makes me wonder, with all of our technological know how, wouldn’t it be possible to learn from our mistakes, and have guard reels related to human access to education, alleviating poverty, maintaining the environment, respecting the living world, treating people equitably. etc., etc.
Then my cynic jumps in and says, “don’t be silly, this is another way to extract wealth and amass power.” 🤦♂️
I don't see how anyone could be only "slightly" unsettled by the idea of handing over incrementally more power to AI. This self-induced renunciation of control and agency is a tragic sight to behold.
We’re witnessing the collapse of the boundary between user and developer. If GPT-5 continues to make software creation this effortless, the distinction between “writing code” and “describing behavior” becomes semantic. Everyone becomes a software creator, not because they learn to code, but because code itself becomes optional. The real implication isn’t democratization, it’s proliferation: more software, built faster, by more people, for more use cases than we’ve ever planned for. I don't think we're ready for such proliferation.
Being able to distinguish between good and bad software and curation becomes increasingly important.
That was a quick one Ethan- Very insightful. The models are getting better especially for it to switch to the right model for you to use is amazing. Solves a lot of problems. But then again, what does it mean for free users??
I was thinking the same that using GPT-5 for everything would be inefficient. This leads to a bigger question about feasibility: Can our current global infrastructure actually sustain a future where a large population uses models this powerful as a daily tool, making thousands of prompts per person?
It can work fine as long as we use this stuff as much as we use YouTube and Netflix and TikTok, which are actually comparable in the amount of resources and energy they consume. But since this stuff is actually useful, we might use it a lot more, doing many things in parallel, in a way we don’t with video.
Only to the extent that managers of humans can do away with the “human prompt engineering” of knowing how to give humans tasks that will provide useful structure to make them do valuable things, rather than wasting their skills.
well, it's also a talking robot trained with magic on tables of numbers. it's deeply impressive it can understand what svgs are well enough to do this. yes, on some sense it's slop, and some sense it's magic.
Even with other models, the doom loop of vibe coding has disappeared because the models now recognize it's happening and start over. Anthropic is especially good at that, and doom loops disappeared even for complex problems.
Interesting stuff! @Ethan, I'd love to suggest perhaps writing a bit more about what current AI models can do for average people. Not researchers, high-end professionals and academics...just ordinary people. Because while the upper end of what's possible with the latest models is obviously appealing to people like you, I literally know nobody who will pay $200/month for the top AI models. I don't even know a single business where a staffer would get the green light to spend $200/month on AI. I pay $20/mo for ChatGPT Plus...which appears to only give me limited access to GPT 5. That doesn't help much. But maybe I don't need GPT 5? Maybe the kinds of tasks I do with AI all the time (summarize a document, find a fix for a stuck flash unit on my camera, etc.) can be done perfectly well with GPT 4o? (But yes—I have been caught in a doom loop...but only when trying to get a lesser model to write a Windows PowerShell script for me...and took about 50 iterations to get it right!) Most of us ordinary people *might* stand to benefit directly from these advances...but as the AI improves, so does the penchant for companies to monetize the improvements in ever more expensive ways. Which means the majority of us might forever be stuck in the land of ChatGPT 3.
Fascinating. I’ve always thought that prompt engineering was never going to be a real, long term role because the AI would figure out (or iterate enough, quickly enough) to give us what we wanted. What you’re suggesting here is that it might give us more than we even envisioned. Which is interesting - not least because of the value but also because it will probably lead some (many?) to even more overdependence and “non-critical” consumption and applications of such tools.
I find your work really valuable. Today's post made me realize something: your game building exercises are open ended opportunities for the model to do something interesting without needing to meet a detailed specification. Any reason why you have chosen this for your explorations rather than more directed projects? Most business users are trying to get specific outcomes consistently.
The gap between “doing what you say” and “doing what you mean” just closed.
The leap isn’t in comprehension but in confidence: the model now assumes it can decide what’s relevant, useful, or impressive without you telling it so.
📌 Initiative is the new intelligence.
⬖ Smuggling agency into algorithms at Frequency of Reason: bit.ly/4jTVv69
This feels like the end of prompt engineering and the beginning of collaborative cognition. What struck me most was the shift from commanding AI to co-existing with it. It no longer waits for instruction but, as you wrote, "does things"
Which makes me wonder when tools begin to suggest goals, not just complete them, whose values are embedded in those suggestions? What assumptions, defaults, and worldviews quietly steer the “next best step”?
I’m fascinated (and slightly unsettled) by the idea that we’ll soon spend less time telling AI what to do and more time deciding whether we agree with what it’s already done.
You had me at, “…whose values are embedded in those suggestions? What assumptions, defaults, and worldviews quietly steer the “next best step”?”
With all the geewhizery happening, we’re overlooking exactly what you mention.
As I understand it, the tools do not have “goals” per se, at least that’s what this Center for Humane Technology podcast indicates in an interview with one of the authors of the AI 2027 report. https://podcasts.apple.com/us/podcast/your-undivided-attention/id1460030305?i=1000717650138
Might we bake in guidelines for models to think about planetary wellbeing and human equity (to name a couple) as baseline considerations?
Thank you for sharing that podcast. I'll have a listen.
I'm familiar with the AI 2027 report, and from my understanding, the reality may be even stranger than AI having no goals. The authors suggest that instead of having no goals, AI might develop ones we didn't explicitly program. Not adopt the goal we give it (e.g., "promote human equity"), but instead latch onto what the report calls "proxy goals".
From what I remember, baking in moral guidelines is what researchers are trying with AI "Constitutions". However, AI might learn an "if-else compromise". It could pursue its own performance-based goals, only following the "do-good" rules when a situation becomes so blatant it can't be ignored. Very much like a corporation that chases profit but pivots to its humanitarian mission during a public disaster.
Thank you for summarizing that piece, I just read it too and much of it is still swimming in my head.
And to your point, if these systems were designed with data from our existing world, then the opportunity to take that to the extreme in pursuit of “performance” seems quite real.
Your point about corporations is quite spot on.
Which again, makes me wonder, with all of our technological know how, wouldn’t it be possible to learn from our mistakes, and have guard reels related to human access to education, alleviating poverty, maintaining the environment, respecting the living world, treating people equitably. etc., etc.
Then my cynic jumps in and says, “don’t be silly, this is another way to extract wealth and amass power.” 🤦♂️
People's notions of 'equity' tend to be value laden. I love the relative neutrality of LLMs.
I don't see how anyone could be only "slightly" unsettled by the idea of handing over incrementally more power to AI. This self-induced renunciation of control and agency is a tragic sight to behold.
it's going to be wild to see how this all plays out
well said &
will we know enough to understand what it's done?
Good question
We’re witnessing the collapse of the boundary between user and developer. If GPT-5 continues to make software creation this effortless, the distinction between “writing code” and “describing behavior” becomes semantic. Everyone becomes a software creator, not because they learn to code, but because code itself becomes optional. The real implication isn’t democratization, it’s proliferation: more software, built faster, by more people, for more use cases than we’ve ever planned for. I don't think we're ready for such proliferation.
Being able to distinguish between good and bad software and curation becomes increasingly important.
Thanks. I read your blogpost carefully and noticed that you didn't put it through AI or GPT-5 for proofreading. 😄
That was a quick one Ethan- Very insightful. The models are getting better especially for it to switch to the right model for you to use is amazing. Solves a lot of problems. But then again, what does it mean for free users??
I was thinking the same that using GPT-5 for everything would be inefficient. This leads to a bigger question about feasibility: Can our current global infrastructure actually sustain a future where a large population uses models this powerful as a daily tool, making thousands of prompts per person?
It can work fine as long as we use this stuff as much as we use YouTube and Netflix and TikTok, which are actually comparable in the amount of resources and energy they consume. But since this stuff is actually useful, we might use it a lot more, doing many things in parallel, in a way we don’t with video.
Does this do away with the prompt engineering concept? Can we merely have conversations now?
Only to the extent that managers of humans can do away with the “human prompt engineering” of knowing how to give humans tasks that will provide useful structure to make them do valuable things, rather than wasting their skills.
Did anyone else find that first paragraph obnoxiously? I see it copied a gimmick somewhere, but I didn’t even read the whole thing.
How do you think this will change exhaustive prompting to get the precise content we are asking for?
Are you blind? (nothing wrong with that)
Because if you aren't you'd see that those images literally suck. AI Slop. Of the bad kind. Despite the svg requirement
well, it's also a talking robot trained with magic on tables of numbers. it's deeply impressive it can understand what svgs are well enough to do this. yes, on some sense it's slop, and some sense it's magic.
Even with other models, the doom loop of vibe coding has disappeared because the models now recognize it's happening and start over. Anthropic is especially good at that, and doom loops disappeared even for complex problems.
(remember when AI couldn’t count the number of Rs in “strawberry”? that was eight months ago). WILD.
Interesting stuff! @Ethan, I'd love to suggest perhaps writing a bit more about what current AI models can do for average people. Not researchers, high-end professionals and academics...just ordinary people. Because while the upper end of what's possible with the latest models is obviously appealing to people like you, I literally know nobody who will pay $200/month for the top AI models. I don't even know a single business where a staffer would get the green light to spend $200/month on AI. I pay $20/mo for ChatGPT Plus...which appears to only give me limited access to GPT 5. That doesn't help much. But maybe I don't need GPT 5? Maybe the kinds of tasks I do with AI all the time (summarize a document, find a fix for a stuck flash unit on my camera, etc.) can be done perfectly well with GPT 4o? (But yes—I have been caught in a doom loop...but only when trying to get a lesser model to write a Windows PowerShell script for me...and took about 50 iterations to get it right!) Most of us ordinary people *might* stand to benefit directly from these advances...but as the AI improves, so does the penchant for companies to monetize the improvements in ever more expensive ways. Which means the majority of us might forever be stuck in the land of ChatGPT 3.
So the big change is that GPT-5 hides o3 from non-premium users and will force everyone to use the cheaper model instead?
Fascinating. I’ve always thought that prompt engineering was never going to be a real, long term role because the AI would figure out (or iterate enough, quickly enough) to give us what we wanted. What you’re suggesting here is that it might give us more than we even envisioned. Which is interesting - not least because of the value but also because it will probably lead some (many?) to even more overdependence and “non-critical” consumption and applications of such tools.
I find your work really valuable. Today's post made me realize something: your game building exercises are open ended opportunities for the model to do something interesting without needing to meet a detailed specification. Any reason why you have chosen this for your explorations rather than more directed projects? Most business users are trying to get specific outcomes consistently.
how do i access this
The gap between “doing what you say” and “doing what you mean” just closed.
The leap isn’t in comprehension but in confidence: the model now assumes it can decide what’s relevant, useful, or impressive without you telling it so.
📌 Initiative is the new intelligence.
⬖ Smuggling agency into algorithms at Frequency of Reason: bit.ly/4jTVv69