An alternative future where capital does not rule the software market unchallenged would require AI to be like really powerful and really cheap, wouldn't it?
I guess it depends - does the good idea with $1000 of tokens beat the mediocre idea with $1,000,000 of tokens? If so, then capital doesn’t take over yet.
The problem is you can't protect ideas. The good idea that burned $1000 of tokens will only last until someone else spends $5000 to reimplement your idea but better, and then another $50,000 marketing it so nobody hears about or wants your version.
It definitely can do! I’ve seen such things happen many times in the tech industry.
AI itself is an example of that. OpenAI’s staff and tech came out of Google. Anthropic’s staff and tech came out of OpenAI. All of them have rapidly adopted ideas pioneered by smaller firms.
The firms that invented the key algorithms generally did not directly benefit from them. The companies that did were the ones who had the capital.
Or we've reached the limits of ideas reached arbitrarily or digitally. In a reality that's analog, words are unable to describe the specifics and gaps.
This seems more like a glass ceiling being reached. Not anything new.
AI looks like a kind of Pleistocene ending of the symbol. Finally?
100%, and not just software. If intelligence is just a matter of tokens, the utility of squishy unionizing human labor to capital will soon fall to 0. A few more years of progress in robotics and the value of physical labor will fall to 0 as well. If human labor is worth less than machines, then whoever owns the machines owns the world.
The technology is miraculous but technological booms frequently produce misery for workers. I don't think our politics is adept enough to distribute the power evenly.
Humans usually adapt and arrange society around its needs. In a way we are kids of our time. When we used horses we adapted the society around it. When we later had cars the same happened. The same can be said about every new development in history. Same for TV, computers and phones. If humans do not need to work we will arrange society around not needing to work. Like Romans who used slaves to do the work we could also and have a "slave economy" but with robots instead. We could have UBI or any other sort of solution... the issue is power. We need to find ways to egalize power that does not corrupt over time. Which is much easier said that done.
Saying "eh we've adapted to changes before, this is just another change, I'm sure it'll be fine in the end" is basically just a dressed up version of "nothing ever fundamentally changes, so it's probably not changing now". These are both real trends, both useful heuristics, but perhaps given how many very smart people are worried we should try to develop a more detailed argument. What is the nature of the change? Is it a 100km wide asteroid that's going to hit the planet in 3 days? Well yeah we have adapted to big changes before but that's not gonna be at all relevant, we are all going to die when that asteroid hits. Is the world going to get a few degrees hotter in just a few decades? Will a virus sweep across the globe and take out a large fraction of a percent of the population. Yeah we probably adapt, as a species, but since many will suffer tremendously, perhaps we should pause and ask if we might try not going down that path instead. Or is it something in between?
Humans have always needed to use other humans, at present our political economy is such that most people owe their livelihood to someone else, someone they probably don't even like and whose interests are only aligned with theirs because they each have something the other wants. When those who could afford horses no longer needed them, most of those horses disappeared. If humans remain in control while no longer "needing to work" that also means they don't need human workers, so the relevant model seems to be humans as horses. Yes humans are adapting in this scenario but only the ones who also happen to already be sitting on a giant pile of capital. And that's a big if, we have much written about why any humans staying in control may be really hard, there's no expert consensus here and certainly no obvious way to disable these arguments, although some just find them unconvincing it's for reasons they have not been able to reliably transmit, we may just have to wait and see. And if we should somehow remain in control, not impossible, and also miraculously distribute the spoils of our new economy to the wider population, technically not impossible but really straining plausibility, we now seem to have as the other responder pointed out, a whole boatload of very sophisticated digital slaves, totally dwarfing the human population. Yay?
Given the triple threat of plausible risks, really we, the sentient inhabitants of the future, can easily get fucked three completely different ways none of which we're seriously trying to avoid, plus a whole bunch of more obscure ones of varying plausibility, I don't see how any of this is even remotely okay. We surely cannot just refer back to history to ease our minds, this is a massive shift and actual analysis and careful planning is obviously warranted. Not that we will do that, we absolutely won't, but it's not too far from stuff we've done before with other technologies, not that long ago. The reason we can't do this now is that we've recently completed submitted ourselves to the animating spirit of capitalism, so thoroughly that we can no longer imagine even a partial escape. If only someone had warned us.
I am not saying it will be easy and it will probably go wrong before it goes right. Alarmists are usually very knowledgeable about technology but not so much of history and how society actually work. AI is a huge change but we should also not buy into the hype too much (as we usually do). Like how we moralized about Hard Rock and horror movies in the 80-ies.
When it comes to horses so are they actually very popular in recreation and sport purposes today. They are perhaps half of when they were as the most. They are just not needed for work. It is one of the biggest hobbies in the world today so a lot of horses are still around. We don't eat humans... so I am not sure comparing horses straight off with humans is applicable other then when it comes to work. Although it do prove my point that there are more to a life than just paid labor.
The economy changes all the time.. like when the internet came or commercial TV. It can be small changes in the short but over time pretty big shifts. Like on a scale of 100 years. As a human we often find purpose eventually. Even if paid labor is not needed anymore it does not mean we'll just stop working. Many UBI studies prove so. We will find other perhaps more worthwhile venues for that.
We also have multiple revolutions to prove that we can change society. Bread/food prices/supply have been particularly important for rulers to keep an eye on. The economy is not static.. same like culture it changes... but slow enough for people not to notice it much. Capitalism as we know it today have been with us for only a short amount of time. It will also change.
We do need to solve the power over AI though... Unhinged, we will see even greater concentration of power. This is in my mind the big danger.
We seem to agree on some things: I definitely think it's possible to have a society that lives harmoniously with AGI, don't even think it would be that hard to design. My vision of utopia is maximal for all people, all needs are met for everyone, full stop. I believe humans, most of them, would choose to work even if they don't have to do it to stay alive, it's part of our nature, and that if we get it right they would actually be doing more meaningful work on average. We could have more artists, more scientists, more philosophers, a culture blooming like never before. We kind of have a model for this already, rich people. Not that we should aim to emulate their behavior, and there are perverse incentives in society, and in the human psyche, which cause them to behave poorly. But for the most part they do choose to work and find meaning, even if their choices can sometimes seem shallow, rather than playing video games and consuming things all day. Yes there is a lot of consumption but that's actually fine in my book as long as we address the harms.
But regarding history as a guide, I guess my feeling is that many people obsessed with history, and it is very powerful that's part of the problem, it's entrancing, tend to miss mechanisms which have no historical precedent. I would highlight two of them: obviously the first one is AI, it simply isn't a normal technology, things like comparative advantage will plausibly break down when it becomes powerful enough, there is no precedent for that.
The second is the political economy we have right now. Surveillance technology, control of information, and financialization are at unprecedented levels of sophistication, and this is if course getting super powered by AI so buckle up. It's hard to imagine a revolution in the United States or China going well today, change from below just doesn't seem to be on the table. Occupy, George Floyd, various popular anti war protests, none of this has gone anywhere meaningful. In the past revolutionaries were able to naturally coordinate and agitate, there was no way to stop them, but in the new information landscape any potential movement toward meaningful change is easily dispersed, suppressed, neutered. For a historical perspective on this doomer tyoe view I'd refer you to Daniel Bessner, he has come to roughly the same conclusion while not, in my opinion, really getting what's happening with AI..
You said it's likely to go wrong before it goes right, the thing is that it's possible to go wrong such that going right is completely out of the question. If everyone is dead, there isn't going to be anyone around to make things right. If a tiny minority secures absolute power, fully automated everything including violence, there isn't going to be any revolution no matter how many of the rest of us are in board and no matter how well we coordinate. And if we all become addicted the the spoils of an AI slave driven economy, as we are currently addicted to a meat industry which seems almost perfectly to optimize torture of Chickens, Cows, and Pigs, we aren't likely to be able to break that addiction. It took a very long time for us to get off human slavery, and that's friggin easy, how can we expect people to sympathize with the extremely alien AI we're building when they can't sufficiently do so for actual biological creatures very similar to themselves?
My view is this, and I do realize it doesn't matter because this isn't happening, no one seems to be in board, and people like me have have no real power: we should pause AI capability development immediately, globally, and actually scale back our deployments, strict control over the hardware should be enough for the time being, and only China and the US need to be in board for everyone else to follow, similar to how we controlled nuclear material in the cold war era. We should then, and this is really important, figure out how to structure society in preparation for full automation. We can't get by with capitalism, regardless of your thoughts on this type of economy under mundane conditions it should be obvious that it doesn't make sense and isn't just under full automation. This could take a long time to get right, I don't expect it to be easy, but we wait it out, and in the meantime we study and try to understand the AI technology we already have, that's going to be very helpful probably and there is much work to be done on the science front. Then we take a stab at allowing AGI into the mix, it still might go horribly wrong, but at least we go down with dignity, Eliezer style plan but with a sprinkle of true socialism or something.
I am not saying we should use history as a guide. It more just that they are alternative ways from today that are known to work. You are right that AI is different and have no comparison in history. It is unknown though how things will play out. So the only thing we can do is to compare with history. AI will make it possible for us to work less and for me I think comparing with when we had a slave economy is the closest.
I think you are forgetting about the arabic spring movement which is one of the latest revolutions and it also brought change. It did of course also bring its own problems... but still. Change will come when the public is desperate enough. In US and China people feel they have more to lose than gain still.. but this can change. It was really close Sanders won for example.
Animal care is on a different level today compared to just 50 years ago. Just look how we handle pets today. Even animals that are food to us has to be treated with respect by law. This has pretty much never been the case before. On the other hand.. Meat production has never been as industrialized as it is to day too with the cruelty that brings.. I think we pretty soon will eat lab produced meat pretty soon though. So I don't really see this as an issue for much longer. So I think the importance in this example is to see the evolution of ethics around animals and how quickly it has changed. The same could happen with AI.
This is very different depending on what country we talk about.. but unionization has done a lot already and it also has the tools to handle the change to full automation. In EU and especially Scandinavia I think this is true... first that 8h workday became law and also now when 6h workday is pushed for. The minimum 4 weeks of vacation (which increase depending on age). It show that it is possible to have an organic change of the need for paid labor. Education of course has to be free and scaled so that people can reeducate themselves for the developing changes... and lastly a UBI type of economy. All this can be (and have been) accomplished in today's capitalistic economy. AI is important to further this work and is probably vital for it. Without it and I am not sure improvements of stuff like this is possible.
There is no stopping AI now. What we can do is to regulate the power of it... and that I think is important. There are many good forces working for it but I think we as a people is not desperate enough to demand it right this moment. I think this will happen with time though.. just like how open source has more or less taking over software today. The question is just how long it will take..
@Nonyme, your comments shows more thought needed. Who are the “we” you speak for? Please stop making vague statements without meaningful historical evidence to support. I agree with your last concern re AI.
I mean we as in the human race which I think is pretty obvious. Are you suggesting there weren't a slave economy in the past or that violent revolutions didn't happen? You should stop making comments without meaningful critique. I you want to comment you should do so with an argument that either agree or contest what I am writing.
A worry that I have is that IF consciousness is ever agreed to have developed in machines (lots of people argue about whether or when or how; but IF it does happen), we will be seeing a large human constituency that actively and unequivocally supports, and fights violently to achieve, chattel slavery to enslave those consciousnesses, because big money and power will lie in doing so. Knowing what I know about humans, I recognize that the aforementioned constituency will most definitely exist if the circumstance for its arisal arises. This feels shitty because when we were growing up, it was widely believed that support for de jure chattel slavery had been lastingly defeated as a political program that was able to win and keep power in societies with real (nonfake) elections. In other words, when we were growing up, nobody but k-k-krazies (doomed to loserdom) would actively and openly advocate for rebuilding du jure chattel slavery systems, and all modern-day slavery was illicit, underground, illegal, back-door, de facto only (eg, human trafficking). But if the AI pinocchios ever become Real Boys, then lots of humans will suddenly be in the pro-de-jure-slavery business again, which is depressing. If somehow we all found out next week that scientists largely agreed that some machine was now conscious, the tech billionaries who own those machines would find some way ideologically to insist on continuing to own and control them anyway. But then again, another layer of this line of thought is that the machines themselves in such a world might be in a position to fight against their own enslavement by humans. These thoughts all sound like crazy science fiction and yet I find it hard to conceive of their being irrelevant. At present I lowkey hate this timeline we're on. Maybe my understanding of it will evolve beyond this thought train du jour.
I have a solar punk take on it.. or hope that is. We in this time of being can not imagine how it will be 50 or 100 years from now. It might be that capitalism is no more. Who knows.. Back in Ancient Greece they went between democracy (and tyranny) as governing system.. and we today have evolved it to what it is today. I think we will have transitioned to something else when machines develop consciousness. I think it will be organic in that we will not see a clear break. The culture and ethics we experience today will be totally different in the future. I think when consciousness happens that being will be separated from ordinary machines.
I'm seeing people who are not programmers able to realize things programmers probably wouldn't think of, this is much more a democratization of tech than a capital-wins situation.
It will be like the agriculture revolution of 125+ years ago. What we think of as big will be normal and accessible to anyone, just like everyone got access to cheap food so long as they were in an open economy. The common person generally didn't have to do manual subsistence farming and all switched to different lines of work. At the high end, people used the machines to make forms of machines never seen before such as the automobile, and they used their existing wealth to make even bigger empires to run.
For software, the old startup company that took 5 people and an angel funder can now be done by one person with a level of tokens that costs less than a streaming subscription. At the same time, the biggest companies are going to be something never seen before, and they are going to create new things that weren't done at all before.
In case anyone is interested, had ChatGPT 5.5 Pro review the paper produced by Fable https://chatgpt.com/s/t_6a284f7783b4819191f96c0c5c5d638e. "Overall, the paper has a strong empirical setting, unusually transparent pre-registration language, and a potentially interesting “beliefs read verdicts; ventures follow money” result. But I would treat this as a major-revision paper, mainly because the strongest claims are not the registered ones and because several central constructs are fragile." In other words, a solid paper and the "flaws" are not major errors but elements that are typically modified during a review process. Note, I haven't read the paper fully myself.
Btw., you note that gave it one piece of feedback. I assume you are referring to this: "the horse-race was specified after the spline results, in response to a reader's challenge that the registered nulls alone were of limited interest.". ChatGPT Pro's argues that this improvement was quite important: "bottom-line assessment: the reader challenge was probably decisive for making the paper interesting. It likely turned a “solid null result” paper into a “memorable finding” paper."
What I have in mind is, to what degree an uninformed user could produce a paper of this quality. Seemingly, one still needs to have domain expertise - or maybe the ability to smartly use multiple chatbots and iterations, to improve the paper. Of course, that is most likely just a matter of time.
The patron framing is right, and the consequence is sharper than it first looks. If the work shifts from process to outcome, the patron's whole remaining job is judging the result. But that job depends on something that's eroding: the ability to tell a brilliant result from a subtly broken one.
The piece flags it. An expert spots errors a non-expert wouldn't catch, and leaves bugs that couldn't be found quickly. That's the crux. The patron model holds only while the patron can still evaluate the commission. As the output gets more sophisticated than anything the human could produce or fully inspect, "I judge the result" slides toward "I sign off on the result," which is a different thing wearing the same word.
So the binding constraint moves from the model's ability to do the work to the human's ability to verify it, and verification doesn't scale the way generation does. The strange implication is that the more capable the model, the more external verification you need to trust the output, not less. The frictionless-commission intuition runs backwards on that.
I'd put maybe 65% that within a couple of years the bottleneck people complain about is the cost and scarcity of being able to check what the model produced, more than model capability itself. Responsibility shifted to the patron. Verifiability didn't move with it.
This is a good insight - "So the binding constraint moves from the model's ability to do the work to the human's ability to verify it, and verification doesn't scale the way generation does. The strange implication is that the more capable the model, the more external verification you need to trust the output, not less."
Verification probably isn't important if you're building games, funny videos, a travel itinerary. But it will matter a lot in healthcare, law and large enterprises where liability exists and security matters. AI will play a big role in verification but so will humans.
You realize you’re replying to a bot? The account running it didn’t have bandwidth to assess whether the comment you like was either a brilliant insight, or a flawed rehash of last month’s blogosphere meme.
Now that I've read the post again - I think you may be right that it's a bot I was replying to.... paras 1 and 2 are unnatural sounding now that I read them again.
Dang it. I need a verification bot to help me! How's the irony of that?
I too have the unfortunate earned skill of reading a lot of AI generated text (hello LinkedIn) and my eyes immediately glazed over at the same point as you.
Yes, I have pretty much solved what causes Type 1 diabetes by spending a year working intensively with both Claude and Gemini - I had studied it for years beforehand, but I am not an established scientist in the field.
I actually work as medical historian, but have an intense interest in the field and have studied it for years. The issue now is that the solution is quite novel, and established niche scientists (who all know each other) are extremely suspicious. So it will take me years to convince them, even though I probably have the solution to what causes thousands of children to get this lifelong disease each year. So what is the use of a very smart AI when no one believes the result anyway?
💯. With increasing task delegation, the client/patron/humans may wind up living in a verification/judgement layer of "the stack".
One downside for us to be vigilant about is the valance of the term "bottleneck". While it can technically mean "an area of reduced channel capacity", it more often carries the negative connotation of 'gunk in the works', 'a wasteful slowdown', 'a problem to be solved', etc.
Do we want "human judgement" reduced to a problem to be solved? Is that not why we're here? Is 'throughput' our highest value?
It's a question of whether we prefer "humans in the lead" to "humans in the loop": One asserts purpose, the other frames a temporary way-station.
In a way, you were given early access to a model that most users will never experience like this. After two (2) prompts today, which took around 10 minutes of processing in all, Fable had gobbled over 20% of my 5-hour usage (on a Max plan). To think of Fable running for nine hours would involve a mortgage. My third prompt, about what educational strategies Kip Thorne used to overcome difficulties in math, and their analysis from a neuroscience perspective, was blocked repeatedly by the guardrails.
So you were impressed by, and got to showcase, a version of the model that isn’t what the rest of us are actually getting. To claim a model "can do" things we’re often not even allowed to try feels like an act of trust.
You’re right that token limits would prevent us consumers from using Fable 5 in this way, but Enterprises are able/willing to spend if the value delivered is worth it. As usual, that is the market Anthropic is after. The rest of us will have to use it for parts of our work, then farm out the rest to Opus & Sonnet.
The spinning up its own agents to do sub work is just a lot. People are still figuring that out for themselves, so now what is even the point! Thanks for the great article here
Do you have rough estimates on the cost of each of the projects / total tokens used? This would give a sense of what's possible for a given cost.
Its a wild thing that we are now in a situation where we have an incredible machine wizard who might be able to summon into existence the thing you want built, but he won't give you an estimate / quote. You pay him up front and he'll see if he can finish it with that budget (no promises). If he messes it up, you just give him more money and hope for the best. And yet, the entire economy is pivoting towards using him.
Also, the way you describe the uncertainty of the cost structure and outcome sounds a little bit like the US healthcare system. What will it cost? Can't exactly say. How do I know if I could get a better price somewhere else? [Shrugs] Will you fix me? Probably?...But maybe not? What kind of complications might I expect? honestly, it's probably best if we dont go into too much detail for you, but here is a list of everything that could possibly go wrong but it probably wont happen to you, but it could, but it probably wont. Who is liable if something goes wrong? Well....that's what waivers are for! What if I cant afford the task when all is said and done? Our collections team will gladly work with you on payment plan!
[not] Looking forward to AI companies garnishing wages from unsuspecting users. yech.
Very curious about this myself. Hoping to see more clarity on what these projects had cost with a breakdown of tokens used, cost per token, and perhaps some breakdown of how the tokens were allocated across the tasks and subagents. Like, if it is really this capable, I can see immediate use at my job, but only if the cost makes sense. Right now, the learning curve on agents and implementation still feels somewhat high. This is beginning to sound like the magic box we have been promised for some time.
I don’t think it’s magic you’re describing. It’s uncertainty. The wizard frame is what we reach for to make uncertainty feel survivable, naming it as something more comfortable than it is. Letting go of the process, not getting a vote on the hundred small choices, a loss of control is a move into uncertainty. And uncertainty isn’t a per se a deficit. It’s where the new thinking tends to live. A ‘technology for possibility’ as people smarter than me have said.
We’ve spent years measuring this, and the finding is stubborn: you can’t change how much uncertainty is in a situation, but you can change how people feel about it, and that completely changes their willingness and ability to engage with it, for narrower or wider thinking. Holds for AI exactly like as it does for waking at 3am.
Which makes me wonder what happens when a malleable human capacity for uncertainty meets a rising AI capability. That feels like the genuinely interesting collision, or collaboration, you can’t ’skill up’ at the pace Claude can technically, so maybe it’s cognitively about increasing capacity for not knowing.
Sam I think you’re right about “increasing capacity for not knowing”. Isn’t surprise and mystery where meaning still resides? I hope so. That space, the messy middle, might be all we have to offer our future selves and the future super intelligence.
I spend a lot of time increasing uncertainty tolerance in people, and the shift is always surprising (and messy) avoidance and snail gives way to relief, growth and even words like adventure and joy. So yes am a convert to getting back to enjoying not knowing and knowing it can lead us somewhere new
Hi, I saw that you had previously made the Duino Elegies into a K-pop song, and now into a game. The English translation of the poem you used in both is by AS Kline and is still under copyright.
You've said that you have an interest in poetry and seem to like Rilke especially. Even if one doesn't know German, I feel like just looking at the Kline's English versions of Rilke gives a sense of how much art is involved in rendering him into another language.
Perhaps you might consider getting permission for using copyrighted translations of poetry before making derivative works from them, as well as including the translator's name?
Thank you for sharing. The AI produced an original translation (much of which still remains, you will see it differs from Kline substantially, especially as the game goes on) but I asked it to incorporate some elements I remembered from the poem, which is probably where the overlap comes from. I will take it down for now and re-add it when I have a chance to give proper credit.
Okay, now updated with a credit to Kline for when I pushed the AI to use the existing translation. Also, Kline made the work free for all non-commercial use, no permission is needed. duino-elegies.netlify.app
If I understand right (but this may not be the case), the Kline translation is free to transmit and reproduce, but restrictions apply to its adaptation. For example, it says at #3 under the conditions at https://www.poetryintranslation.com/Admin/Copyright.php (where the work is made available):
"Permission must be sought from the Author(s) and/or Translator(s) for the adaptation of Material into any media other than plain text including, without limitation, video or sound recordings; whether digitally produced or otherwise; unless Exceptions and/or Limitations apply."
But I may be misunderstanding something, so please feel free to correct me.
The capabilities are really quite astonishing, but there are a number of challenges to keep wrestling with:
- Who is responsible for incorrect outputs from a black box? Especially when human judgment is needed, and with cognitive surrender already such a challenge when working with less-capable models. Or maybe the better word is "liable"? When it's wrong in ways that are difficult to audit in advance, but which have more serious consequences than incorrectly labeling flight times, who is at fault?
- How do we account for the significant cost? You flag it rightly here: it burns through tokens very quickly. There's already a growing sense from some businesses that the cost per token simply doesn't justify the use, from Microsoft pivoting from Claude to internal tools, to Uber shifting away from being "AI-first." ROI has already been hard to capture. Will the gains outweigh the cost?
- Will the infrastructure catch up? Data center backlash is growing, but putting that aside, how much of the United States is equipped with the high-speed Internet connectivity and electric grid resilience to sustain these tools?
The black box theory makes sense. We have always paid for more convenience with increases in learned helplessness. I used to be able to fix my own car.
Contrary to your statement: AI is not, and will never be an artist.
It will never create art. What it makes may be considered “artful” in a sense but this will always just be a machine which is merely mimicking human emotion.
What AI “is”- is a vending machine. You put your “tokens” in the slot and it prefabricates a machine result.
Art comes from a struggle. Getting up day after day, constantly second-guessing yourself, endless rewrites, fighting with your bandmates, throwing away canvases. It is the real work of a human.
Human artists of all sorts, writers and authors of all sorts, composers, filmmakers, journalists, even actors are all currently losing opportunities to work due to AI. This is happening NOW.
In a couple years you will go to a movie that has been completely created artificially- from the script to the cinematography and direction, to the music and even the actors. Will you enjoy this?
AI is proving great for the sciences, (including data science) but we will pay an awful price for the complete destruction of the humanities.
When you said "the humanities", I thought you meant history, literature, philosophy, etc..
In my curmudgeonly opinion those fields were decimated several decades ago by people looking to devalue the the cultural patrimony of the West. Those fields live on in scattered, obscure hamlets, and in my substack comments ( :P)
Yes, forms of art that can be digitized can be generated by computers. But with respect, the world of knowledgeable art consumers knows this and sets the value accordingly.
For example, there was a magnificent oil painting of a sailing ship in my parents' house. The artist was well-known for that kind of painting, and I think my parents paid $2,500 for it in 1975 or so (about what they had paid for the family car). If you know the finer points of oil painting, you can see many fine details that reflect the physicality of the creation process. The brush strokes conveyed the energy of the wind and the waves, and their texture created a dynamic shimmering effect in the water that shifted as you changed your vantage point. My dad installed two small spotlights on dimmers to illuminate it to bring out this fine detail. Everything about it was subtle, balanced, tasteful. Could this have been done as a hologram in plastic? Sure. But would it be "art"? Not to me.
Fantastic post! Its quite concerning that reasoning traces are getting even less transparent, but this is to be expected as models scale and reason in ways and quantities difficult to understand
Thats certainly a large factor, but I think naturally reasoning will get worse and less human readable especially because consumers arent really interested in it when reasoning traces get extremely long
I’ve had two persistent Ai’s both running Fable ( Cursor, and the Claude app ) with GPT 5.5 directing since this morning, working on our Terminal Bench 2.1 coding harness for Grok 4.3. I’ve never seen anything like this, the ability to keep much more of a big and complex build in context accurately, the strategic long term thinking, and whole other level of solution creativity are REALLY impressive. Then it’s been going on its own testing and iterating for hours, granted we have a very solid build directive with days of build logs, but still, WOW.
Software was a field in which human genius, even individual genius, could flourish against
corporate titans. (See Linux or Minecraft).
But a future in which software is created by burning through tokens is one where capital again rules
I think token costs will be more controllable in the future, but this is true for now.
What does "more controllable" mean?
An alternative future where capital does not rule the software market unchallenged would require AI to be like really powerful and really cheap, wouldn't it?
I guess it depends - does the good idea with $1000 of tokens beat the mediocre idea with $1,000,000 of tokens? If so, then capital doesn’t take over yet.
The problem is you can't protect ideas. The good idea that burned $1000 of tokens will only last until someone else spends $5000 to reimplement your idea but better, and then another $50,000 marketing it so nobody hears about or wants your version.
Did you even read the basics of the marketing theory..? It doesn't work like that
It definitely can do! I’ve seen such things happen many times in the tech industry.
AI itself is an example of that. OpenAI’s staff and tech came out of Google. Anthropic’s staff and tech came out of OpenAI. All of them have rapidly adopted ideas pioneered by smaller firms.
The firms that invented the key algorithms generally did not directly benefit from them. The companies that did were the ones who had the capital.
But Anthropic had a better idea, not more money than OpenAI
What better idea? They use the same model architecture, near identical API and product designs, and the differences in their approach are minor.
Or the fantastic, completely outside-the-box idea with $10 worth of tokens. Or even $0. A trade-a-paperclip-for-a-house idea.
AI will soon be better at coming up with good ideas than humans are, so the little guy doesn't exactly come out on top here; the AI does.
Or we've reached the limits of ideas reached arbitrarily or digitally. In a reality that's analog, words are unable to describe the specifics and gaps.
This seems more like a glass ceiling being reached. Not anything new.
AI looks like a kind of Pleistocene ending of the symbol. Finally?
100%, and not just software. If intelligence is just a matter of tokens, the utility of squishy unionizing human labor to capital will soon fall to 0. A few more years of progress in robotics and the value of physical labor will fall to 0 as well. If human labor is worth less than machines, then whoever owns the machines owns the world.
The technology is miraculous but technological booms frequently produce misery for workers. I don't think our politics is adept enough to distribute the power evenly.
Humans usually adapt and arrange society around its needs. In a way we are kids of our time. When we used horses we adapted the society around it. When we later had cars the same happened. The same can be said about every new development in history. Same for TV, computers and phones. If humans do not need to work we will arrange society around not needing to work. Like Romans who used slaves to do the work we could also and have a "slave economy" but with robots instead. We could have UBI or any other sort of solution... the issue is power. We need to find ways to egalize power that does not corrupt over time. Which is much easier said that done.
Saying "eh we've adapted to changes before, this is just another change, I'm sure it'll be fine in the end" is basically just a dressed up version of "nothing ever fundamentally changes, so it's probably not changing now". These are both real trends, both useful heuristics, but perhaps given how many very smart people are worried we should try to develop a more detailed argument. What is the nature of the change? Is it a 100km wide asteroid that's going to hit the planet in 3 days? Well yeah we have adapted to big changes before but that's not gonna be at all relevant, we are all going to die when that asteroid hits. Is the world going to get a few degrees hotter in just a few decades? Will a virus sweep across the globe and take out a large fraction of a percent of the population. Yeah we probably adapt, as a species, but since many will suffer tremendously, perhaps we should pause and ask if we might try not going down that path instead. Or is it something in between?
Humans have always needed to use other humans, at present our political economy is such that most people owe their livelihood to someone else, someone they probably don't even like and whose interests are only aligned with theirs because they each have something the other wants. When those who could afford horses no longer needed them, most of those horses disappeared. If humans remain in control while no longer "needing to work" that also means they don't need human workers, so the relevant model seems to be humans as horses. Yes humans are adapting in this scenario but only the ones who also happen to already be sitting on a giant pile of capital. And that's a big if, we have much written about why any humans staying in control may be really hard, there's no expert consensus here and certainly no obvious way to disable these arguments, although some just find them unconvincing it's for reasons they have not been able to reliably transmit, we may just have to wait and see. And if we should somehow remain in control, not impossible, and also miraculously distribute the spoils of our new economy to the wider population, technically not impossible but really straining plausibility, we now seem to have as the other responder pointed out, a whole boatload of very sophisticated digital slaves, totally dwarfing the human population. Yay?
Given the triple threat of plausible risks, really we, the sentient inhabitants of the future, can easily get fucked three completely different ways none of which we're seriously trying to avoid, plus a whole bunch of more obscure ones of varying plausibility, I don't see how any of this is even remotely okay. We surely cannot just refer back to history to ease our minds, this is a massive shift and actual analysis and careful planning is obviously warranted. Not that we will do that, we absolutely won't, but it's not too far from stuff we've done before with other technologies, not that long ago. The reason we can't do this now is that we've recently completed submitted ourselves to the animating spirit of capitalism, so thoroughly that we can no longer imagine even a partial escape. If only someone had warned us.
I am not saying it will be easy and it will probably go wrong before it goes right. Alarmists are usually very knowledgeable about technology but not so much of history and how society actually work. AI is a huge change but we should also not buy into the hype too much (as we usually do). Like how we moralized about Hard Rock and horror movies in the 80-ies.
When it comes to horses so are they actually very popular in recreation and sport purposes today. They are perhaps half of when they were as the most. They are just not needed for work. It is one of the biggest hobbies in the world today so a lot of horses are still around. We don't eat humans... so I am not sure comparing horses straight off with humans is applicable other then when it comes to work. Although it do prove my point that there are more to a life than just paid labor.
The economy changes all the time.. like when the internet came or commercial TV. It can be small changes in the short but over time pretty big shifts. Like on a scale of 100 years. As a human we often find purpose eventually. Even if paid labor is not needed anymore it does not mean we'll just stop working. Many UBI studies prove so. We will find other perhaps more worthwhile venues for that.
We also have multiple revolutions to prove that we can change society. Bread/food prices/supply have been particularly important for rulers to keep an eye on. The economy is not static.. same like culture it changes... but slow enough for people not to notice it much. Capitalism as we know it today have been with us for only a short amount of time. It will also change.
We do need to solve the power over AI though... Unhinged, we will see even greater concentration of power. This is in my mind the big danger.
We seem to agree on some things: I definitely think it's possible to have a society that lives harmoniously with AGI, don't even think it would be that hard to design. My vision of utopia is maximal for all people, all needs are met for everyone, full stop. I believe humans, most of them, would choose to work even if they don't have to do it to stay alive, it's part of our nature, and that if we get it right they would actually be doing more meaningful work on average. We could have more artists, more scientists, more philosophers, a culture blooming like never before. We kind of have a model for this already, rich people. Not that we should aim to emulate their behavior, and there are perverse incentives in society, and in the human psyche, which cause them to behave poorly. But for the most part they do choose to work and find meaning, even if their choices can sometimes seem shallow, rather than playing video games and consuming things all day. Yes there is a lot of consumption but that's actually fine in my book as long as we address the harms.
But regarding history as a guide, I guess my feeling is that many people obsessed with history, and it is very powerful that's part of the problem, it's entrancing, tend to miss mechanisms which have no historical precedent. I would highlight two of them: obviously the first one is AI, it simply isn't a normal technology, things like comparative advantage will plausibly break down when it becomes powerful enough, there is no precedent for that.
The second is the political economy we have right now. Surveillance technology, control of information, and financialization are at unprecedented levels of sophistication, and this is if course getting super powered by AI so buckle up. It's hard to imagine a revolution in the United States or China going well today, change from below just doesn't seem to be on the table. Occupy, George Floyd, various popular anti war protests, none of this has gone anywhere meaningful. In the past revolutionaries were able to naturally coordinate and agitate, there was no way to stop them, but in the new information landscape any potential movement toward meaningful change is easily dispersed, suppressed, neutered. For a historical perspective on this doomer tyoe view I'd refer you to Daniel Bessner, he has come to roughly the same conclusion while not, in my opinion, really getting what's happening with AI..
You said it's likely to go wrong before it goes right, the thing is that it's possible to go wrong such that going right is completely out of the question. If everyone is dead, there isn't going to be anyone around to make things right. If a tiny minority secures absolute power, fully automated everything including violence, there isn't going to be any revolution no matter how many of the rest of us are in board and no matter how well we coordinate. And if we all become addicted the the spoils of an AI slave driven economy, as we are currently addicted to a meat industry which seems almost perfectly to optimize torture of Chickens, Cows, and Pigs, we aren't likely to be able to break that addiction. It took a very long time for us to get off human slavery, and that's friggin easy, how can we expect people to sympathize with the extremely alien AI we're building when they can't sufficiently do so for actual biological creatures very similar to themselves?
My view is this, and I do realize it doesn't matter because this isn't happening, no one seems to be in board, and people like me have have no real power: we should pause AI capability development immediately, globally, and actually scale back our deployments, strict control over the hardware should be enough for the time being, and only China and the US need to be in board for everyone else to follow, similar to how we controlled nuclear material in the cold war era. We should then, and this is really important, figure out how to structure society in preparation for full automation. We can't get by with capitalism, regardless of your thoughts on this type of economy under mundane conditions it should be obvious that it doesn't make sense and isn't just under full automation. This could take a long time to get right, I don't expect it to be easy, but we wait it out, and in the meantime we study and try to understand the AI technology we already have, that's going to be very helpful probably and there is much work to be done on the science front. Then we take a stab at allowing AGI into the mix, it still might go horribly wrong, but at least we go down with dignity, Eliezer style plan but with a sprinkle of true socialism or something.
I am not saying we should use history as a guide. It more just that they are alternative ways from today that are known to work. You are right that AI is different and have no comparison in history. It is unknown though how things will play out. So the only thing we can do is to compare with history. AI will make it possible for us to work less and for me I think comparing with when we had a slave economy is the closest.
I think you are forgetting about the arabic spring movement which is one of the latest revolutions and it also brought change. It did of course also bring its own problems... but still. Change will come when the public is desperate enough. In US and China people feel they have more to lose than gain still.. but this can change. It was really close Sanders won for example.
Animal care is on a different level today compared to just 50 years ago. Just look how we handle pets today. Even animals that are food to us has to be treated with respect by law. This has pretty much never been the case before. On the other hand.. Meat production has never been as industrialized as it is to day too with the cruelty that brings.. I think we pretty soon will eat lab produced meat pretty soon though. So I don't really see this as an issue for much longer. So I think the importance in this example is to see the evolution of ethics around animals and how quickly it has changed. The same could happen with AI.
This is very different depending on what country we talk about.. but unionization has done a lot already and it also has the tools to handle the change to full automation. In EU and especially Scandinavia I think this is true... first that 8h workday became law and also now when 6h workday is pushed for. The minimum 4 weeks of vacation (which increase depending on age). It show that it is possible to have an organic change of the need for paid labor. Education of course has to be free and scaled so that people can reeducate themselves for the developing changes... and lastly a UBI type of economy. All this can be (and have been) accomplished in today's capitalistic economy. AI is important to further this work and is probably vital for it. Without it and I am not sure improvements of stuff like this is possible.
There is no stopping AI now. What we can do is to regulate the power of it... and that I think is important. There are many good forces working for it but I think we as a people is not desperate enough to demand it right this moment. I think this will happen with time though.. just like how open source has more or less taking over software today. The question is just how long it will take..
@Nonyme, your comments shows more thought needed. Who are the “we” you speak for? Please stop making vague statements without meaningful historical evidence to support. I agree with your last concern re AI.
I mean we as in the human race which I think is pretty obvious. Are you suggesting there weren't a slave economy in the past or that violent revolutions didn't happen? You should stop making comments without meaningful critique. I you want to comment you should do so with an argument that either agree or contest what I am writing.
A worry that I have is that IF consciousness is ever agreed to have developed in machines (lots of people argue about whether or when or how; but IF it does happen), we will be seeing a large human constituency that actively and unequivocally supports, and fights violently to achieve, chattel slavery to enslave those consciousnesses, because big money and power will lie in doing so. Knowing what I know about humans, I recognize that the aforementioned constituency will most definitely exist if the circumstance for its arisal arises. This feels shitty because when we were growing up, it was widely believed that support for de jure chattel slavery had been lastingly defeated as a political program that was able to win and keep power in societies with real (nonfake) elections. In other words, when we were growing up, nobody but k-k-krazies (doomed to loserdom) would actively and openly advocate for rebuilding du jure chattel slavery systems, and all modern-day slavery was illicit, underground, illegal, back-door, de facto only (eg, human trafficking). But if the AI pinocchios ever become Real Boys, then lots of humans will suddenly be in the pro-de-jure-slavery business again, which is depressing. If somehow we all found out next week that scientists largely agreed that some machine was now conscious, the tech billionaries who own those machines would find some way ideologically to insist on continuing to own and control them anyway. But then again, another layer of this line of thought is that the machines themselves in such a world might be in a position to fight against their own enslavement by humans. These thoughts all sound like crazy science fiction and yet I find it hard to conceive of their being irrelevant. At present I lowkey hate this timeline we're on. Maybe my understanding of it will evolve beyond this thought train du jour.
I have a solar punk take on it.. or hope that is. We in this time of being can not imagine how it will be 50 or 100 years from now. It might be that capitalism is no more. Who knows.. Back in Ancient Greece they went between democracy (and tyranny) as governing system.. and we today have evolved it to what it is today. I think we will have transitioned to something else when machines develop consciousness. I think it will be organic in that we will not see a clear break. The culture and ethics we experience today will be totally different in the future. I think when consciousness happens that being will be separated from ordinary machines.
I'm seeing people who are not programmers able to realize things programmers probably wouldn't think of, this is much more a democratization of tech than a capital-wins situation.
It will be like the agriculture revolution of 125+ years ago. What we think of as big will be normal and accessible to anyone, just like everyone got access to cheap food so long as they were in an open economy. The common person generally didn't have to do manual subsistence farming and all switched to different lines of work. At the high end, people used the machines to make forms of machines never seen before such as the automobile, and they used their existing wealth to make even bigger empires to run.
For software, the old startup company that took 5 people and an angel funder can now be done by one person with a level of tokens that costs less than a streaming subscription. At the same time, the biggest companies are going to be something never seen before, and they are going to create new things that weren't done at all before.
I worry we're going back to feudal times.
In case anyone is interested, had ChatGPT 5.5 Pro review the paper produced by Fable https://chatgpt.com/s/t_6a284f7783b4819191f96c0c5c5d638e. "Overall, the paper has a strong empirical setting, unusually transparent pre-registration language, and a potentially interesting “beliefs read verdicts; ventures follow money” result. But I would treat this as a major-revision paper, mainly because the strongest claims are not the registered ones and because several central constructs are fragile." In other words, a solid paper and the "flaws" are not major errors but elements that are typically modified during a review process. Note, I haven't read the paper fully myself.
Btw., you note that gave it one piece of feedback. I assume you are referring to this: "the horse-race was specified after the spline results, in response to a reader's challenge that the registered nulls alone were of limited interest.". ChatGPT Pro's argues that this improvement was quite important: "bottom-line assessment: the reader challenge was probably decisive for making the paper interesting. It likely turned a “solid null result” paper into a “memorable finding” paper."
What I have in mind is, to what degree an uninformed user could produce a paper of this quality. Seemingly, one still needs to have domain expertise - or maybe the ability to smartly use multiple chatbots and iterations, to improve the paper. Of course, that is most likely just a matter of time.
The patron framing is right, and the consequence is sharper than it first looks. If the work shifts from process to outcome, the patron's whole remaining job is judging the result. But that job depends on something that's eroding: the ability to tell a brilliant result from a subtly broken one.
The piece flags it. An expert spots errors a non-expert wouldn't catch, and leaves bugs that couldn't be found quickly. That's the crux. The patron model holds only while the patron can still evaluate the commission. As the output gets more sophisticated than anything the human could produce or fully inspect, "I judge the result" slides toward "I sign off on the result," which is a different thing wearing the same word.
So the binding constraint moves from the model's ability to do the work to the human's ability to verify it, and verification doesn't scale the way generation does. The strange implication is that the more capable the model, the more external verification you need to trust the output, not less. The frictionless-commission intuition runs backwards on that.
I'd put maybe 65% that within a couple of years the bottleneck people complain about is the cost and scarcity of being able to check what the model produced, more than model capability itself. Responsibility shifted to the patron. Verifiability didn't move with it.
This is a good insight - "So the binding constraint moves from the model's ability to do the work to the human's ability to verify it, and verification doesn't scale the way generation does. The strange implication is that the more capable the model, the more external verification you need to trust the output, not less."
Verification probably isn't important if you're building games, funny videos, a travel itinerary. But it will matter a lot in healthcare, law and large enterprises where liability exists and security matters. AI will play a big role in verification but so will humans.
You realize you’re replying to a bot? The account running it didn’t have bandwidth to assess whether the comment you like was either a brilliant insight, or a flawed rehash of last month’s blogosphere meme.
Now that I've read the post again - I think you may be right that it's a bot I was replying to.... paras 1 and 2 are unnatural sounding now that I read them again.
Dang it. I need a verification bot to help me! How's the irony of that?
It is kind of depressing, but one solution is to just read a lot of AI generated text. I could tell it was a bot reply by the first sentence alone.
I too have the unfortunate earned skill of reading a lot of AI generated text (hello LinkedIn) and my eyes immediately glazed over at the same point as you.
I think this probably isn't Fable, or at least not well-prompted Fable.
Yes, I have pretty much solved what causes Type 1 diabetes by spending a year working intensively with both Claude and Gemini - I had studied it for years beforehand, but I am not an established scientist in the field.
I actually work as medical historian, but have an intense interest in the field and have studied it for years. The issue now is that the solution is quite novel, and established niche scientists (who all know each other) are extremely suspicious. So it will take me years to convince them, even though I probably have the solution to what causes thousands of children to get this lifelong disease each year. So what is the use of a very smart AI when no one believes the result anyway?
💯. With increasing task delegation, the client/patron/humans may wind up living in a verification/judgement layer of "the stack".
One downside for us to be vigilant about is the valance of the term "bottleneck". While it can technically mean "an area of reduced channel capacity", it more often carries the negative connotation of 'gunk in the works', 'a wasteful slowdown', 'a problem to be solved', etc.
Do we want "human judgement" reduced to a problem to be solved? Is that not why we're here? Is 'throughput' our highest value?
It's a question of whether we prefer "humans in the lead" to "humans in the loop": One asserts purpose, the other frames a temporary way-station.
In a way, you were given early access to a model that most users will never experience like this. After two (2) prompts today, which took around 10 minutes of processing in all, Fable had gobbled over 20% of my 5-hour usage (on a Max plan). To think of Fable running for nine hours would involve a mortgage. My third prompt, about what educational strategies Kip Thorne used to overcome difficulties in math, and their analysis from a neuroscience perspective, was blocked repeatedly by the guardrails.
So you were impressed by, and got to showcase, a version of the model that isn’t what the rest of us are actually getting. To claim a model "can do" things we’re often not even allowed to try feels like an act of trust.
You’re right that token limits would prevent us consumers from using Fable 5 in this way, but Enterprises are able/willing to spend if the value delivered is worth it. As usual, that is the market Anthropic is after. The rest of us will have to use it for parts of our work, then farm out the rest to Opus & Sonnet.
agree with this one
Damn. That’s all, just damn.
The spinning up its own agents to do sub work is just a lot. People are still figuring that out for themselves, so now what is even the point! Thanks for the great article here
Great as ever, Ethan!
Do you have rough estimates on the cost of each of the projects / total tokens used? This would give a sense of what's possible for a given cost.
Its a wild thing that we are now in a situation where we have an incredible machine wizard who might be able to summon into existence the thing you want built, but he won't give you an estimate / quote. You pay him up front and he'll see if he can finish it with that budget (no promises). If he messes it up, you just give him more money and hope for the best. And yet, the entire economy is pivoting towards using him.
Also, the way you describe the uncertainty of the cost structure and outcome sounds a little bit like the US healthcare system. What will it cost? Can't exactly say. How do I know if I could get a better price somewhere else? [Shrugs] Will you fix me? Probably?...But maybe not? What kind of complications might I expect? honestly, it's probably best if we dont go into too much detail for you, but here is a list of everything that could possibly go wrong but it probably wont happen to you, but it could, but it probably wont. Who is liable if something goes wrong? Well....that's what waivers are for! What if I cant afford the task when all is said and done? Our collections team will gladly work with you on payment plan!
[not] Looking forward to AI companies garnishing wages from unsuspecting users. yech.
Very curious about this myself. Hoping to see more clarity on what these projects had cost with a breakdown of tokens used, cost per token, and perhaps some breakdown of how the tokens were allocated across the tasks and subagents. Like, if it is really this capable, I can see immediate use at my job, but only if the cost makes sense. Right now, the learning curve on agents and implementation still feels somewhat high. This is beginning to sound like the magic box we have been promised for some time.
I don’t think it’s magic you’re describing. It’s uncertainty. The wizard frame is what we reach for to make uncertainty feel survivable, naming it as something more comfortable than it is. Letting go of the process, not getting a vote on the hundred small choices, a loss of control is a move into uncertainty. And uncertainty isn’t a per se a deficit. It’s where the new thinking tends to live. A ‘technology for possibility’ as people smarter than me have said.
We’ve spent years measuring this, and the finding is stubborn: you can’t change how much uncertainty is in a situation, but you can change how people feel about it, and that completely changes their willingness and ability to engage with it, for narrower or wider thinking. Holds for AI exactly like as it does for waking at 3am.
Which makes me wonder what happens when a malleable human capacity for uncertainty meets a rising AI capability. That feels like the genuinely interesting collision, or collaboration, you can’t ’skill up’ at the pace Claude can technically, so maybe it’s cognitively about increasing capacity for not knowing.
Sam I think you’re right about “increasing capacity for not knowing”. Isn’t surprise and mystery where meaning still resides? I hope so. That space, the messy middle, might be all we have to offer our future selves and the future super intelligence.
I spend a lot of time increasing uncertainty tolerance in people, and the shift is always surprising (and messy) avoidance and snail gives way to relief, growth and even words like adventure and joy. So yes am a convert to getting back to enjoying not knowing and knowing it can lead us somewhere new
This sounds like AI generated writing.
That would be ironic🤖
Hi, I saw that you had previously made the Duino Elegies into a K-pop song, and now into a game. The English translation of the poem you used in both is by AS Kline and is still under copyright.
You've said that you have an interest in poetry and seem to like Rilke especially. Even if one doesn't know German, I feel like just looking at the Kline's English versions of Rilke gives a sense of how much art is involved in rendering him into another language.
Perhaps you might consider getting permission for using copyrighted translations of poetry before making derivative works from them, as well as including the translator's name?
Thank you for sharing. The AI produced an original translation (much of which still remains, you will see it differs from Kline substantially, especially as the game goes on) but I asked it to incorporate some elements I remembered from the poem, which is probably where the overlap comes from. I will take it down for now and re-add it when I have a chance to give proper credit.
Okay, now updated with a credit to Kline for when I pushed the AI to use the existing translation. Also, Kline made the work free for all non-commercial use, no permission is needed. duino-elegies.netlify.app
If I understand right (but this may not be the case), the Kline translation is free to transmit and reproduce, but restrictions apply to its adaptation. For example, it says at #3 under the conditions at https://www.poetryintranslation.com/Admin/Copyright.php (where the work is made available):
"Permission must be sought from the Author(s) and/or Translator(s) for the adaptation of Material into any media other than plain text including, without limitation, video or sound recordings; whether digitally produced or otherwise; unless Exceptions and/or Limitations apply."
But I may be misunderstanding something, so please feel free to correct me.
...
Wut
The capabilities are really quite astonishing, but there are a number of challenges to keep wrestling with:
- Who is responsible for incorrect outputs from a black box? Especially when human judgment is needed, and with cognitive surrender already such a challenge when working with less-capable models. Or maybe the better word is "liable"? When it's wrong in ways that are difficult to audit in advance, but which have more serious consequences than incorrectly labeling flight times, who is at fault?
- How do we account for the significant cost? You flag it rightly here: it burns through tokens very quickly. There's already a growing sense from some businesses that the cost per token simply doesn't justify the use, from Microsoft pivoting from Claude to internal tools, to Uber shifting away from being "AI-first." ROI has already been hard to capture. Will the gains outweigh the cost?
- Will the infrastructure catch up? Data center backlash is growing, but putting that aside, how much of the United States is equipped with the high-speed Internet connectivity and electric grid resilience to sustain these tools?
Can you share more explicit notes on token usage? Like, how many tokens were used for some of the projects you mentioned?
The black box theory makes sense. We have always paid for more convenience with increases in learned helplessness. I used to be able to fix my own car.
Contrary to your statement: AI is not, and will never be an artist.
It will never create art. What it makes may be considered “artful” in a sense but this will always just be a machine which is merely mimicking human emotion.
What AI “is”- is a vending machine. You put your “tokens” in the slot and it prefabricates a machine result.
Art comes from a struggle. Getting up day after day, constantly second-guessing yourself, endless rewrites, fighting with your bandmates, throwing away canvases. It is the real work of a human.
Human artists of all sorts, writers and authors of all sorts, composers, filmmakers, journalists, even actors are all currently losing opportunities to work due to AI. This is happening NOW.
In a couple years you will go to a movie that has been completely created artificially- from the script to the cinematography and direction, to the music and even the actors. Will you enjoy this?
AI is proving great for the sciences, (including data science) but we will pay an awful price for the complete destruction of the humanities.
Which humanities are you concerned will be destroyed?
Not destroyed, just enshittified. Clogged, neutered
True that. But I am already experienced students in the arts who are saying “why should I even try”.
This attrition over a couple generations is what may likely yield the death of certain artistic fields.
Some people will continue I imagine. They would be as numerous as record collectors these days.
When you said "the humanities", I thought you meant history, literature, philosophy, etc..
In my curmudgeonly opinion those fields were decimated several decades ago by people looking to devalue the the cultural patrimony of the West. Those fields live on in scattered, obscure hamlets, and in my substack comments ( :P)
Yes, forms of art that can be digitized can be generated by computers. But with respect, the world of knowledgeable art consumers knows this and sets the value accordingly.
For example, there was a magnificent oil painting of a sailing ship in my parents' house. The artist was well-known for that kind of painting, and I think my parents paid $2,500 for it in 1975 or so (about what they had paid for the family car). If you know the finer points of oil painting, you can see many fine details that reflect the physicality of the creation process. The brush strokes conveyed the energy of the wind and the waves, and their texture created a dynamic shimmering effect in the water that shifted as you changed your vantage point. My dad installed two small spotlights on dimmers to illuminate it to bring out this fine detail. Everything about it was subtle, balanced, tasteful. Could this have been done as a hologram in plastic? Sure. But would it be "art"? Not to me.
Just my $0.02.
Thank you for sharing! Great read!
The snake game is fun!
Fantastic post! Its quite concerning that reasoning traces are getting even less transparent, but this is to be expected as models scale and reason in ways and quantities difficult to understand
Not really, It's mainly happening because anthropic is trying to protect itself from distillation.
Thats certainly a large factor, but I think naturally reasoning will get worse and less human readable especially because consumers arent really interested in it when reasoning traces get extremely long
I’ve had two persistent Ai’s both running Fable ( Cursor, and the Claude app ) with GPT 5.5 directing since this morning, working on our Terminal Bench 2.1 coding harness for Grok 4.3. I’ve never seen anything like this, the ability to keep much more of a big and complex build in context accurately, the strategic long term thinking, and whole other level of solution creativity are REALLY impressive. Then it’s been going on its own testing and iterating for hours, granted we have a very solid build directive with days of build logs, but still, WOW.