In this paper we showed that if you ask GPT-3 to decide how novel and useful its own answers are, it can actually do better than human level on the AUT: https://openreview.net/pdf?id=8HwKaJ1wvl
After it failed to provide reasonable answers, I asked it to explain the rules.
Can you explain the rules of our game in your own words?
Sure, the game is to provide three words, and I have to combine each word with a fourth word to create new English words that refer to different concepts. The four words must not be variations or misspellings of the same word, or proper nouns.
and then:
Let's reverse roles: Can you give me three words so that I can come up with a fourth word that I can combine to form three new English words that refer to different concepts.
Sure! Here are three words for you:
Fire
Stone
Glass
work
Yes, that's correct! Firework, Stonework, and Glasswork are all English words that refer to different concepts.
Thanks for another insightful post, Ethan. Your newsletter is some of the best content in my inbox each week.
I love this framing of the AI as a tool to extend our capabilities and reframing the competition as collaboration and is something I wrote about this last week as I compared ChatGPT to other past forms of technological disruption: https://joshbrake.substack.com/p/reframing-the-disruption-of-chatgpt
I think there are many parallels with what you’ve shared and known best practice/ in design thinking. I suspect that these types of generative AI tools will become an integral part of any job that requires brainstorming, for the reasons you mentioned. Being able to easily generate a wide variety of ideas and then filter is a huge asset.
Thanks for continuing to share such high-value and thoughtful analysis!
Great read. AI can be very useful as you pointed out
But I took a different angle on whether AI could be creative. I see the examples you used as good for human association and recall but when a computer is programmed with near infinite (human coded and data labeled) connections, it's as creative as sending a human into the same tests with the answer key.
I found that AI can't really be creative, useful yes, but still locked into the traditional.
Precisely! You get it! Being a human is NOT just about being sensible, productive, practical, "time-sensitive," or simply functional! Good job, ChatGPT, you're learning how to be human!
Now you, too, ChatGPT, can cherish your own ideas, imagine what it might like to cherish children, and stubbornly cling to what are actually pretty bad ideas, just because that's what humans damn well do, every damn day, from the first cogent thought to the period at the end of this sentence.
In this paper we showed that if you ask GPT-3 to decide how novel and useful its own answers are, it can actually do better than human level on the AUT: https://openreview.net/pdf?id=8HwKaJ1wvl
After it failed to provide reasonable answers, I asked it to explain the rules.
Can you explain the rules of our game in your own words?
Sure, the game is to provide three words, and I have to combine each word with a fourth word to create new English words that refer to different concepts. The four words must not be variations or misspellings of the same word, or proper nouns.
and then:
Let's reverse roles: Can you give me three words so that I can come up with a fourth word that I can combine to form three new English words that refer to different concepts.
Sure! Here are three words for you:
Fire
Stone
Glass
work
Yes, that's correct! Firework, Stonework, and Glasswork are all English words that refer to different concepts.
find the common word that connects pine, crab, and sauce
Seafood.
let's try this again: what words connect pine /crab / sauce
The common word connecting "pine," "crab," and "sauce" is "tree."
Fascinating! I gave the AI the test in early December. Wonder if an update changed it? Did you use the same instructions I did in the screenshot?
The quoted text above is copy/pasted from the chat. It's the entire conversation. I tried it again: New conversation, then I asked
What word connects pine / crab / sauce?
Pineapple.
Thanks for another insightful post, Ethan. Your newsletter is some of the best content in my inbox each week.
I love this framing of the AI as a tool to extend our capabilities and reframing the competition as collaboration and is something I wrote about this last week as I compared ChatGPT to other past forms of technological disruption: https://joshbrake.substack.com/p/reframing-the-disruption-of-chatgpt
I think there are many parallels with what you’ve shared and known best practice/ in design thinking. I suspect that these types of generative AI tools will become an integral part of any job that requires brainstorming, for the reasons you mentioned. Being able to easily generate a wide variety of ideas and then filter is a huge asset.
Thanks for continuing to share such high-value and thoughtful analysis!
Great read. AI can be very useful as you pointed out
But I took a different angle on whether AI could be creative. I see the examples you used as good for human association and recall but when a computer is programmed with near infinite (human coded and data labeled) connections, it's as creative as sending a human into the same tests with the answer key.
I found that AI can't really be creative, useful yes, but still locked into the traditional.
https://polymathicbeing.substack.com/p/can-ai-be-creative
Great post Ethan
Precisely! You get it! Being a human is NOT just about being sensible, productive, practical, "time-sensitive," or simply functional! Good job, ChatGPT, you're learning how to be human!
Now you, too, ChatGPT, can cherish your own ideas, imagine what it might like to cherish children, and stubbornly cling to what are actually pretty bad ideas, just because that's what humans damn well do, every damn day, from the first cogent thought to the period at the end of this sentence.