Mar 13, 2023Liked by Ethan Mollick

In this paper we showed that if you ask GPT-3 to decide how novel and useful its own answers are, it can actually do better than human level on the AUT: https://openreview.net/pdf?id=8HwKaJ1wvl

Expand full comment

Regrettably, being creative just isn't enough. One also needs to have the judgement necessary to sort through the big pile of ideas a creative person will typically generate to identify those ideas that are actually worth acting on. Without such judgement in adequate measure the creative person is likely to choose the idea they had the most fun creating.

Creativity can be a huge time waster in some cases. While the practical person will probably choose the well known solution that is already available, the creative person may waste a ton of time trying to reinvent the wheel so as to creatively solve a problem that's already been solved.

A truly creative person often isn't sensible, for they value the experience of creativity over results. They don't want an idea that works, they want a hundred ideas that can serve as a launching pad for further creativity.

A creative person will often cherish their own ideas as they would their own children, and cling to them stubbornly, even if they are actually pretty bad ideas. For the creative person, any creative idea is by definition a good idea, just for being the product of creativity.

Expand full comment

Precisely! You get it! Being a human is NOT just about being sensible, productive, practical, "time-sensitive," or simply functional! Good job, ChatGPT, you're learning how to be human!

Now you, too, ChatGPT, can cherish your own ideas, imagine what it might like to cherish children, and stubbornly cling to what are actually pretty bad ideas, just because that's what humans damn well do, every damn day, from the first cogent thought to the period at the end of this sentence.

Expand full comment

After it failed to provide reasonable answers, I asked it to explain the rules.

Can you explain the rules of our game in your own words?

Sure, the game is to provide three words, and I have to combine each word with a fourth word to create new English words that refer to different concepts. The four words must not be variations or misspellings of the same word, or proper nouns.

and then:

Let's reverse roles: Can you give me three words so that I can come up with a fourth word that I can combine to form three new English words that refer to different concepts.

Sure! Here are three words for you:





Yes, that's correct! Firework, Stonework, and Glasswork are all English words that refer to different concepts.

Expand full comment

find the common word that connects pine, crab, and sauce


let's try this again: what words connect pine /crab / sauce

The common word connecting "pine," "crab," and "sauce" is "tree."

Expand full comment

Fascinating! I gave the AI the test in early December. Wonder if an update changed it? Did you use the same instructions I did in the screenshot?

Expand full comment

The quoted text above is copy/pasted from the chat. It's the entire conversation. I tried it again: New conversation, then I asked

What word connects pine / crab / sauce?


Expand full comment

Thanks for another insightful post, Ethan. Your newsletter is some of the best content in my inbox each week.

I love this framing of the AI as a tool to extend our capabilities and reframing the competition as collaboration and is something I wrote about this last week as I compared ChatGPT to other past forms of technological disruption: https://joshbrake.substack.com/p/reframing-the-disruption-of-chatgpt

I think there are many parallels with what you’ve shared and known best practice/ in design thinking. I suspect that these types of generative AI tools will become an integral part of any job that requires brainstorming, for the reasons you mentioned. Being able to easily generate a wide variety of ideas and then filter is a huge asset.

Thanks for continuing to share such high-value and thoughtful analysis!

Expand full comment

Great read. AI can be very useful as you pointed out

But I took a different angle on whether AI could be creative. I see the examples you used as good for human association and recall but when a computer is programmed with near infinite (human coded and data labeled) connections, it's as creative as sending a human into the same tests with the answer key.

I found that AI can't really be creative, useful yes, but still locked into the traditional.


Expand full comment

Great post Ethan

Expand full comment