I teach in a doctoral program. Your description of AI producing work without "necessarily representing underlying effort" matches perfectly to the dissertation writing process. Our entire assessment uses words as a proxy as an indicator of effort, intelligence, and care. Yes, committees do work with students extensively over a long period of time so we do know and see their thinking process- but it is not necessarily captured in a formal way. I believe this is an existential threat to doctoral programs if we do not change our assessments or the dissertation itself. I actually think it opens up new opportunities to explore research in new ways we have never dreamed of.
Ethan, this is a fantastic piece. I'm motivated, at last, to try and replicated this myself, and it's an unusual piece of writing that does that.
What strikes me most about this is the shift to the importance of asking the right questions. Asking questions isn't a skill in high demand in most of business, and only rarely in academia. Getting instant feedback on asking questions is a change.
Using ChatGPT 4, I feel like it is, or maybe more correctly creates a convincing facsimile of being, AGI already. It is an AGI limited in the extremely important sense of not being able to form new memories and hence not being able to learn, which prevents its fully successful use in all kinds of potential roles like virtual friend, agent instead of chatbot, experienced worker instead of brand new intern that needs its hand held forever. Inability to form memory and hence to learn is built into the LLM transformer model, we would need a totally different architecture to solve it rigorously. But large context windows (say the length of a regular novel) and APIs that fill them with important information from previous chats would allow a quick and dirty hack that would give us a ChatGPT with a spotty memory instead of no memory.
At least in my experience, right now ChatGPT doesn’t feel threatening for the most skilled workers. It’s just not as smart as them, not as creative, and not as good at structured, rigorous thought and planning. But today, right now, except for the still very important memory issue, it is already competitive with millions upon millions of white collar workers who are not the most skilled and not most elite. Sure, it’s not an agent (yet), but it can help one person do the work of 2 people or 5 people or 10 people, and this will have enormous effects going forward as liability issues are worked out and tools like copilot make organizational use impossible to forestall.
And if right now elite workers are safe, who is to say that a Q* powered ChatGPT 5 or 6 in half a year or a few years won’t surpass them or be good enough to start replacing them? Right now we are in the last “normal” time when it comes to organizations and white collar work, the last twilight when it is possible to ignore what is coming. But I have a feeling we will look back from 2030 or 2035 with bemusement to how different things were in 2024
Bing chat, Bing chat entreprise, Copilot, Copilot studio…Microsoft keep changing name and it drives me crazy when I try to explain things to my colleagues
There are so many excellent insights in this post--thanks for your leadership.
I hope you'll address the issue of "quality" in a future post.
Theoretically, as you say here and have said before, AI would "free up" time on less important tasks we are for some reason adamant about preserving (?), so we, humans, can spend more time on the important stuff--even quality. But any behaviorist will tell you that if there aren't immediate and tangible benefits to offering quality products (can we really market quality anymore?), the temptation to turn the time saved by AI into more time to play golf is probably gonna win out.
What AI seems to be revealing is that businesses are like unions in one crucial respect: what's important is time spent, not the quality of the product. The *motivation* for this emphasis on time spent is obviously directly oppositional: the company wants less time put in to increase profit margins, while the union is concerned with the human costs of work speedup (picture the post office, letters and packages flying by). But the quality of the product is really secondary to both. Consumers (both union and management as they go to the same store) will continue to lose.
Will we have to wait until AI actually works better than humans until quality will matter--that is, when an algorithm might suggest that actually, AI might increase quality, too, without any damage to your tee-time or hours on the bass boat?
Ethan, I love that you're out there doing this sort of experimentation. These are truly earth-shaking innovations, and this moment we're living through is just plain *faster* than every other time before.
"This time is different" are perhaps the four most dangerous words in the English language, but "this time is faster" seems altogether completely appropriate. Profound societal changes are already (slowly) beginning to happen, and this revolution won't be anything close to uniform. Your students are already far ahead of the curve, and it'll be tough for anyone to close that gap once AI becomes embedded in their (our) way of life.
As an English teacher, I am excited about how AI can/will change the classroom experience. But as a worker, I am concerned about what this will do to my skill-set. My skill has always been communication - verbal, written, et al. -- what happens when a well-organized and delivered presentation can be done by AI in a matter of seconds? What skill do I need to learn to keep myself relevant for the next 30 years?
I have already begun experimenting with AI in the classroom and writing about it, and have taken a course in prompt engineering from Vanderbilt. Should I learn programming - on a basic level? Data science? I fear that so much of my identity is due to be erased from relevance.
This is an excellent distillation of the state-of-the-state-of-AI to date. (At least, it makes sense to me.) AI is a tool that may make some kinds of output easier, but it doesn’t do the *thinking* that leads humans to create progressive, even revolutionary work. Did you write “Kratzenberg” deliberately? Well played, sir.
That is one of the worries - that people are getting incredibly lazy and use ChatGPT for ***EVERYTHING***.... AI is incredibly western biased, devoid of cultural, regional, generational nuanced interpretations. All this "AI is everywhere" is going to create a sterile, no-innovation type of business world where people will become inflexible dumb workers with no personal confidence to get anything done without an AI directing them.
It would be nice to see a deep dive into the energy cost for this "5 tasks in 59 seconds".... the data centers running OpenAI queries are incredibly energy hungry. That is a dirty side of the business nobody wants to discuss, and be realistic on whether we should really get an AI LLM to do all the "thinking" for us.
Another great piece. Your point on organizations and senior executives thinking about how best to deploy the technology in a responsible way (to employees, clients, regulatory obligations) is a key one. It’s amazing to me how many organizations are not experimenting with this yet. Waiting until the path is clearer means you’ll be too late
I teach in a doctoral program. Your description of AI producing work without "necessarily representing underlying effort" matches perfectly to the dissertation writing process. Our entire assessment uses words as a proxy as an indicator of effort, intelligence, and care. Yes, committees do work with students extensively over a long period of time so we do know and see their thinking process- but it is not necessarily captured in a formal way. I believe this is an existential threat to doctoral programs if we do not change our assessments or the dissertation itself. I actually think it opens up new opportunities to explore research in new ways we have never dreamed of.
AI is my best colleague, my help and friend. It is creepy how I cannot imagine my life without it any more. I love this blog, I learned so much!
Ethan, this is a fantastic piece. I'm motivated, at last, to try and replicated this myself, and it's an unusual piece of writing that does that.
What strikes me most about this is the shift to the importance of asking the right questions. Asking questions isn't a skill in high demand in most of business, and only rarely in academia. Getting instant feedback on asking questions is a change.
Using ChatGPT 4, I feel like it is, or maybe more correctly creates a convincing facsimile of being, AGI already. It is an AGI limited in the extremely important sense of not being able to form new memories and hence not being able to learn, which prevents its fully successful use in all kinds of potential roles like virtual friend, agent instead of chatbot, experienced worker instead of brand new intern that needs its hand held forever. Inability to form memory and hence to learn is built into the LLM transformer model, we would need a totally different architecture to solve it rigorously. But large context windows (say the length of a regular novel) and APIs that fill them with important information from previous chats would allow a quick and dirty hack that would give us a ChatGPT with a spotty memory instead of no memory.
At least in my experience, right now ChatGPT doesn’t feel threatening for the most skilled workers. It’s just not as smart as them, not as creative, and not as good at structured, rigorous thought and planning. But today, right now, except for the still very important memory issue, it is already competitive with millions upon millions of white collar workers who are not the most skilled and not most elite. Sure, it’s not an agent (yet), but it can help one person do the work of 2 people or 5 people or 10 people, and this will have enormous effects going forward as liability issues are worked out and tools like copilot make organizational use impossible to forestall.
And if right now elite workers are safe, who is to say that a Q* powered ChatGPT 5 or 6 in half a year or a few years won’t surpass them or be good enough to start replacing them? Right now we are in the last “normal” time when it comes to organizations and white collar work, the last twilight when it is possible to ignore what is coming. But I have a feeling we will look back from 2030 or 2035 with bemusement to how different things were in 2024
AI is a toy. Garbage in, garbage out with a smattering of bullshit to boot.
Bing chat, Bing chat entreprise, Copilot, Copilot studio…Microsoft keep changing name and it drives me crazy when I try to explain things to my colleagues
There are so many excellent insights in this post--thanks for your leadership.
I hope you'll address the issue of "quality" in a future post.
Theoretically, as you say here and have said before, AI would "free up" time on less important tasks we are for some reason adamant about preserving (?), so we, humans, can spend more time on the important stuff--even quality. But any behaviorist will tell you that if there aren't immediate and tangible benefits to offering quality products (can we really market quality anymore?), the temptation to turn the time saved by AI into more time to play golf is probably gonna win out.
What AI seems to be revealing is that businesses are like unions in one crucial respect: what's important is time spent, not the quality of the product. The *motivation* for this emphasis on time spent is obviously directly oppositional: the company wants less time put in to increase profit margins, while the union is concerned with the human costs of work speedup (picture the post office, letters and packages flying by). But the quality of the product is really secondary to both. Consumers (both union and management as they go to the same store) will continue to lose.
Will we have to wait until AI actually works better than humans until quality will matter--that is, when an algorithm might suggest that actually, AI might increase quality, too, without any damage to your tee-time or hours on the bass boat?
Ethan, I love that you're out there doing this sort of experimentation. These are truly earth-shaking innovations, and this moment we're living through is just plain *faster* than every other time before.
"This time is different" are perhaps the four most dangerous words in the English language, but "this time is faster" seems altogether completely appropriate. Profound societal changes are already (slowly) beginning to happen, and this revolution won't be anything close to uniform. Your students are already far ahead of the curve, and it'll be tough for anyone to close that gap once AI becomes embedded in their (our) way of life.
As an English teacher, I am excited about how AI can/will change the classroom experience. But as a worker, I am concerned about what this will do to my skill-set. My skill has always been communication - verbal, written, et al. -- what happens when a well-organized and delivered presentation can be done by AI in a matter of seconds? What skill do I need to learn to keep myself relevant for the next 30 years?
I have already begun experimenting with AI in the classroom and writing about it, and have taken a course in prompt engineering from Vanderbilt. Should I learn programming - on a basic level? Data science? I fear that so much of my identity is due to be erased from relevance.
This is an excellent distillation of the state-of-the-state-of-AI to date. (At least, it makes sense to me.) AI is a tool that may make some kinds of output easier, but it doesn’t do the *thinking* that leads humans to create progressive, even revolutionary work. Did you write “Kratzenberg” deliberately? Well played, sir.
Once again, Ethan crushes it. I have already forwarded this to many.
Just amazing!
"The AI does the work they do not want to do."
What's not to like?
That is one of the worries - that people are getting incredibly lazy and use ChatGPT for ***EVERYTHING***.... AI is incredibly western biased, devoid of cultural, regional, generational nuanced interpretations. All this "AI is everywhere" is going to create a sterile, no-innovation type of business world where people will become inflexible dumb workers with no personal confidence to get anything done without an AI directing them.
It would be nice to see a deep dive into the energy cost for this "5 tasks in 59 seconds".... the data centers running OpenAI queries are incredibly energy hungry. That is a dirty side of the business nobody wants to discuss, and be realistic on whether we should really get an AI LLM to do all the "thinking" for us.
I love your newsletter, thank you so much. Can you recommend others with practical advice on how to use ChatGPT?
Another great piece. Your point on organizations and senior executives thinking about how best to deploy the technology in a responsible way (to employees, clients, regulatory obligations) is a key one. It’s amazing to me how many organizations are not experimenting with this yet. Waiting until the path is clearer means you’ll be too late