RSS Feed
September 11th, 2023 
Awake Goy Who’s Better At Generating Innovative Ideas, ChatGPT Or MBA Students?
By Mish Shedlock of MishTalk
Wharton professors say Ideas Are Dimes a Dozen and they put that theory to a test. But how does one determine a good idea? And what does better mean?
Wharton notes the difference between consistency and better. For example, an airplane pilot who lands aircraft with average smoothness 10 times out of 10 is better than one who is brilliant 9 times out of 10 and crashes once.
With ideas, one fantastic idea and 10 poor ones is better than 10 average ones. With that backdrop let’s dive into the article.
Abstract: ChatGPT-4 can generate ideas much faster and cheaper than students, and the ideas are on average of higher quality (as measured by purchase-intent surveys) and exhibit higher variance in quality. More important, the vast majority of the best ideas in the pooled sample are generated by ChatGPT and not by the students. Providing ChatGPT with a few examples of highly rated ideas further increases its performance. We discuss the implications of these findings for the management of innovation.
Introduction: Generative artificial intelligence has made remarkable advances in creating life-like images and coherent, fluent text. OpenAI’s ChatGPT chatbot, based on the GPT series of large language models (LLM) can equal or surpass human performance in academic examinations and tests for professional certifications (OpenAI, 2023).
Despite their remarkable performance, LLMs sometimes produce text that is semantically or syntactically plausible but is, in fact, factually incorrect or nonsensical (i.e., hallucinations). The models are optimized to generate the most statistically likely sequences of words with an injection of randomness. They are not designed to exercise any judgment on the veracity or feasibility of the output.
In what applications can we leverage artificial intelligence that is brilliant in many ways yet cannot be trusted to produce reliably accurate results? One possibility is to turn their weaknesses – hallucinations and inconsistent quality – into a strength (Terwiesch, 2023).
ChatGPT can generate ides far faster than humans. This gives them a huge edge in coming up with a few great ideas. For this study the professors gave ChatGPT and the students the same prompt.
I believe I would get a big zero in coming up with a truly innovative product idea.
Could you do this? How fast?
System Prompt “You are a creative entrepreneur looking to generate new product ideas. The product will target college students in the United States. It should be a physical good, not a service or software. I’d like a product that could be sold at a retail price of less than about USD 50. The ideas are just ideas. The product need not yet exist, nor may it necessarily be clearly feasible. Number all ideas and give them a name. The name and idea are separated by a colon.”
User Prompt “Please generate ten ideas as ten separate paragraphs. The idea should be expressed as a paragraph of 40-80 words.”
Do LLMs Enhance Productivity in Generating Ideas?
The answer to this question is straightforward. ChatGPT-4 is very efficient at generating ideas. This question does not require much precision to answer. Two hundred ideas can be generated by one human interacting with ChatGPT-4 in about 15 minutes. A human working alone can generate about five ideas in 15 minutes (Girotra et al., 2010). Humans working in groups do even worse. In short, the productivity race between humans and ChatGPT is not even close.
Still, the old saying that ideas are a dime a dozen is perhaps a tad optimistic. A professional working with ChatGPT-4 can generate ideas at a rate of about 800 ideas per hour. At a cost of USD 500 per hour of human effort, a figure representing an estimate of the fully loaded cost of a skilled professional, ideas are generated at a cost of about USD 0.63 each, or USD 7.50 (75 dimes) per dozen. At the time we used ChatGPT-4, the API fee for 800 ideas was about USD 20. For that same USD 500 per hour, a human working alone, without assistance from an LLM, only generates 20 ideas at a cost of roughly USD 25 each, hardly a dime a dozen. For the focused idea generation task itself, a human using ChatGPT-4 is thus about 40 times more productive than a human working alone.
What Is The Quality Distribution of the Ideas Generated Using LLMs?
A “stochastic parrot” can generate ideas, and LLMs do so shockingly productively. But we don’t care about quantity alone. More typically, the objective of idea generation is to generate at least a few truly exceptionally good ideas. In most innovation settings, we’d rather have 10 great ideas and 90 terrible ideas than 100 ideas of average quality.
We, therefore, care about the quality distribution of the ideas, and in particular, the quality of the best few ideas in a sample. Of course, we might as well also measure the mean and standard deviation of the three sets of ideas, and we do so. Two useful measures of the extreme values are: What is the average quality of the ideas in the top decile of each of the three samples? Which sources provided the ideas comprising the top 10 percent of the ideas in the pooled sample?
Chat-GPT generated the best-rated idea in our sample, with an 11% higher purchase probability than the best human idea. The average quality of the top decile in each of the three pools also follows the same pattern as average quality— seeded Chat-GPT ≻ ChatGPT ≻ Humans. Overall, we have 400 ideas, with an equal number generated by ChatGPT and humans. In the top 40 ideas (top decile) a full 35 (87.5%) are those generated by ChatGPT.
ChatGPT vs the Screen Actors Guild
The Screen Actors Guild (SAG) joined the Writers Guild (WGA) in coordinated strikes. The writers demand protection from Artificial Intelligence. Articles abound.
- Deadline: September 6: WGA & SAG-AFTRA To Rally For California Bill That Would Grant Unemployment Insurance To Strikers
- Deadline: September 6: SAG-AFTRA Says Dual Strikes Against Video Games & Film/TV Industry “Makes Sense” As Key Issues Of Wages & AI “Mirror” Each Other
- MovieWeb: September 7: Bill Maher Says Writers Are Not ‘Owed a Living’ as the WGA & SAG-AFTRA Strikes Continue
Hoot of the Day from the World Socialist Organization
The World Socialist Website reports US film and television writers’ and actors’ anger reaching the boiling point
The struggle by 11,000 film and television writers, members of the Writers Guild of America (WGA), is now in its fifth month, while 65,000 actors in the Screen Actors Guild-American Federation of Television and Radio Artists (SAG-AFTRA) are nearing the end of their second month on the picket lines.
The militant determination of the writers and actors to fight for decent living standards and a more meaningful future for art and culture have been met with intransigence and outright cruelty by the entertainment mega-corporations united in the Alliance of Motion Picture and Television Producers (AMPTP). The companies have made clear their willingness to drive thousands of artists into misery and out of the arts and entertainment industry.
Similarly, with all its phony rhetoric about “solidarity,” SAG-AFTRA has prevented video game workers from joining the strike of their fellow union members and only this week began a strike authorization vote. Moreover, the use of “interim agreements,” allowing hundreds of productions to go ahead, has created a surreal situation of internal scabbing that weakens or negates the purpose of strike action.
Outright cruelty!?
I discussed this on July 24, asking If the Screen Actors and Writers Strikes Went on Forever, Who Would Care?
The strike started on July 14. Did you notice? Care? I don’t watch TV so I am not a good judge.
In a podcast, Maher expressed some sympathy for the writers.
“I feel for my writers. I love my writers. I’m one of my writers. But there’s a big other side to it. And a lot of people are being hurt besides them — a lot of people who don’t make as much money as them in this bipartisan world we have where you’re just in one camp or the other, there’s no in between.
You’re either for the strike like they’re f—ing Che Guevara out there, you know, like, this is Cesar Chavez’s lettuce picking strike — or you’re with Trump. There’s no difference — there’s only two camps. And it’s much more complicated than that.”
But I side with Bill Maher who says writers are not “owed” a living and that the strike demands can be excessive and unrealistic.
The strike demands of the United Auto Workers are also excessive and unrealistic.
United Auto Workers (UAW) Demands
- 32-hour workweek
- 46 percent pay raise over 4 years
- Right to strike over plant closures
- Increased retiree benefits
- Defined pension plan for all workers
- Cost of living adjustments
Bernie Sanders Comments and an Accurate Rebuttal
The irony in the UAW case is Biden is recklessly pursuing an avenue faster than infrastructure allows and that will cost UAW jobs, but increase them elsewhere, in a highly inflationary manner. Note that Biden’s Green Energy Inflation Reduction Act Needs a Big Bailout Already “What, me worry?” Some on Twitter predict, even cheer for my demise to AI writers for my stance against the UAW. Like Bill Maher. I’m not worried. Unlike Bill Maher, I am so small no one would even want to bother to try to replace me. When I started this article, I had no idea it would morph into the Screen Actors Guild or the UAW. On a day to day basis, I have no idea what I am going to write about. Could AI have produced this article better? Even if so, would it bother? In retrospect, I am terrible at producing ideas for products, but pretty good at commenting on the global economic news. If I am replaced by AI, so be it. No one is owed a living. Not the Screen Actors Guild, not the UAW, and not me.
UAW Gearing Up for a Strike, It Could be Long and Nasty
On August 29, I commented UAW Gearing Up for a Strike, It Could be Long and Nasty
Bloomberg estimates the UAW demands would add $80 billion to costs.
If the Big Three automakers gave into UAW demand, they would all go bankrupt in short order.
The fact is, EVs are easier to produce. That means fewer workers. But the workers want protection from losing their jobs. The SAG wants protection from ChatGPT.
It’s really the same story. Change happens. It’s disruptive.
Biden’s Green Energy Inflation Reduction Act Needs a Big Bailout Already
The irony in the UAW case is Biden is recklessly pursuing an avenue faster than infrastructure allows and that will cost UAW jobs, but increase them elsewhere, in a highly inflationary manner.
Note that Biden’s Green Energy Inflation Reduction Act Needs a Big Bailout Already
“What, me worry?”
Some on Twitter predict, even cheer for my demise to AI writers for my stance against the UAW.
Like Bill Maher. I’m not worried. Unlike Bill Maher, I am so small no one would even want to bother to try to replace me.
When I started this article, I had no idea it would morph into the Screen Actors Guild or the UAW. On a day to day basis, I have no idea what I am going to write about. Could AI have produced this article better? Even if so, would it bother?
In retrospect, I am terrible at producing ideas for products, but pretty good at commenting on the global economic news.
If I am replaced by AI, so be it. No one is owed a living. Not the Screen Actors Guild, not the UAW, and not me.
Tyler Durden
Sun, 09/10/2023 – 20:30 Source
Related posts:
Views: 0
Posted in 
Tags: 
