In this paper we showed that if you ask GPT-3 to decide how novel and useful its own answers are, it can actually do better than human level on the AUT: https://openreview.net/pdf?id=8HwKaJ1wvl
Regrettably, being creative just isn't enough. One also needs to have the judgement necessary to sort through the big pile of ideas a creative person will typically generate to identify those ideas that are actually worth acting on. Without such judgement in adequate measure the creative person is likely to choose the idea they had the most fun creating.
Creativity can be a huge time waster in some cases. While the practical person will probably choose the well known solution that is already available, the creative person may waste a ton of time trying to reinvent the wheel so as to creatively solve a problem that's already been solved.
A truly creative person often isn't sensible, for they value the experience of creativity over results. They don't want an idea that works, they want a hundred ideas that can serve as a launching pad for further creativity.
A creative person will often cherish their own ideas as they would their own children, and cling to them stubbornly, even if they are actually pretty bad ideas. For the creative person, any creative idea is by definition a good idea, just for being the product of creativity.
After it failed to provide reasonable answers, I asked it to explain the rules.
Can you explain the rules of our game in your own words?
Sure, the game is to provide three words, and I have to combine each word with a fourth word to create new English words that refer to different concepts. The four words must not be variations or misspellings of the same word, or proper nouns.
and then:
Let's reverse roles: Can you give me three words so that I can come up with a fourth word that I can combine to form three new English words that refer to different concepts.
Sure! Here are three words for you:
Fire
Stone
Glass
work
Yes, that's correct! Firework, Stonework, and Glasswork are all English words that refer to different concepts.
Thanks for another insightful post, Ethan. Your newsletter is some of the best content in my inbox each week.
I love this framing of the AI as a tool to extend our capabilities and reframing the competition as collaboration and is something I wrote about this last week as I compared ChatGPT to other past forms of technological disruption: https://joshbrake.substack.com/p/reframing-the-disruption-of-chatgpt
I think there are many parallels with what you’ve shared and known best practice/ in design thinking. I suspect that these types of generative AI tools will become an integral part of any job that requires brainstorming, for the reasons you mentioned. Being able to easily generate a wide variety of ideas and then filter is a huge asset.
Thanks for continuing to share such high-value and thoughtful analysis!
Great read. AI can be very useful as you pointed out
But I took a different angle on whether AI could be creative. I see the examples you used as good for human association and recall but when a computer is programmed with near infinite (human coded and data labeled) connections, it's as creative as sending a human into the same tests with the answer key.
I found that AI can't really be creative, useful yes, but still locked into the traditional.
In this paper we showed that if you ask GPT-3 to decide how novel and useful its own answers are, it can actually do better than human level on the AUT: https://openreview.net/pdf?id=8HwKaJ1wvl
Regrettably, being creative just isn't enough. One also needs to have the judgement necessary to sort through the big pile of ideas a creative person will typically generate to identify those ideas that are actually worth acting on. Without such judgement in adequate measure the creative person is likely to choose the idea they had the most fun creating.
Creativity can be a huge time waster in some cases. While the practical person will probably choose the well known solution that is already available, the creative person may waste a ton of time trying to reinvent the wheel so as to creatively solve a problem that's already been solved.
A truly creative person often isn't sensible, for they value the experience of creativity over results. They don't want an idea that works, they want a hundred ideas that can serve as a launching pad for further creativity.
A creative person will often cherish their own ideas as they would their own children, and cling to them stubbornly, even if they are actually pretty bad ideas. For the creative person, any creative idea is by definition a good idea, just for being the product of creativity.
After it failed to provide reasonable answers, I asked it to explain the rules.
Can you explain the rules of our game in your own words?
Sure, the game is to provide three words, and I have to combine each word with a fourth word to create new English words that refer to different concepts. The four words must not be variations or misspellings of the same word, or proper nouns.
and then:
Let's reverse roles: Can you give me three words so that I can come up with a fourth word that I can combine to form three new English words that refer to different concepts.
Sure! Here are three words for you:
Fire
Stone
Glass
work
Yes, that's correct! Firework, Stonework, and Glasswork are all English words that refer to different concepts.
find the common word that connects pine, crab, and sauce
Seafood.
let's try this again: what words connect pine /crab / sauce
The common word connecting "pine," "crab," and "sauce" is "tree."
Thanks for another insightful post, Ethan. Your newsletter is some of the best content in my inbox each week.
I love this framing of the AI as a tool to extend our capabilities and reframing the competition as collaboration and is something I wrote about this last week as I compared ChatGPT to other past forms of technological disruption: https://joshbrake.substack.com/p/reframing-the-disruption-of-chatgpt
I think there are many parallels with what you’ve shared and known best practice/ in design thinking. I suspect that these types of generative AI tools will become an integral part of any job that requires brainstorming, for the reasons you mentioned. Being able to easily generate a wide variety of ideas and then filter is a huge asset.
Thanks for continuing to share such high-value and thoughtful analysis!
Great read. AI can be very useful as you pointed out
But I took a different angle on whether AI could be creative. I see the examples you used as good for human association and recall but when a computer is programmed with near infinite (human coded and data labeled) connections, it's as creative as sending a human into the same tests with the answer key.
I found that AI can't really be creative, useful yes, but still locked into the traditional.
https://polymathicbeing.substack.com/p/can-ai-be-creative
Great post Ethan