- Item 1
- Item 2
- Item 3
- Item 4
Stanford/MIT Study: GPT Boosts Support Agent Productivity by up to 35%
Researchers from Stanford and MIT reveal the substantial effects of AI on customer service agents, increasing productivity and closing skill gaps, which may bring far-reaching consequences for the labor landscape.
A Stanford & MIT study has shown vast increases in customer support agent productivity from the deployment of AI software. Photo illustration: Artisana
🧠 Stay Ahead of the Curve
Stanford and MIT researchers reveal AI's significant impact on customer service agents' productivity in a groundbreaking study.
The study demonstrates AI's ability to reduce skill gaps, particularly benefiting low-skill agents and new hires in customer support.
As AI raises performance standards, the broader labor market may face changes in job requirements, compensation, and role availability.
April 26, 2023
A new study by researchers from Stanford and MIT presents a first-of-its-kind analysis of the effects of large language model (LLM)-powered software on over 5,000 customer service agents at a Fortune 500 company. The software, which was powered by OpenAI’s GPT-3, delivered a significant impact that may foreshadow accelerated shifts in the labor landscape driven by generative AI's rapid influence.
A Pioneering Wide-Scale, Long-Term Analysis
The study examined more than 3 million chat sessions conducted by 5,179 agents over a year-long period encompassing both pre- and post-AI deployment. Its uniqueness stems from its focus on real-world conditions, rather than controlled lab tests, and its consideration of AI's impact across an extended timeframe on thousands of real workers.
Although the specific software remains unnamed, the researchers disclosed that it was based on OpenAI’s GPT-3 and fine-tuned on customer service interactions. The agents, primarily located overseas, worked for an undisclosed Fortune 500 company specializing in business process software.
Boosting Productivity for New and Low-Skill Agents
The researchers observed a 13.8% increase in the number of chats agents successfully resolved per hour. Specifically, the gains the researchers witnessed were:
A decline in the time it takes an agent to handle an individual chat.
An increase in the number of chats that an agent is able to handle per hour.
A small increase in the share of chats that are successfully resolved.
The improvement was particularly notable among lower-performing agents, who experienced a 35% increase in successful resolutions per hour. This substantial reduction in skill gaps allowed less-skilled workers to perform nearly as well as their highest-skilled counterparts, the researchers found.
Conversely, higher-skilled agents saw little to no increase in productivity. In some cases, they even experienced a decrease, which researchers hypothesized was due to the AI tool's distracting nature.
Notably, new agents also experienced significant performance improvements. Agents with two months of tenure, aided by AI, performed on par with agents who had been working for over six months without AI assistance.
Implications for Knowledge Worker Industries
Generative AI's rapid emergence is already transforming various industries and the work professionals do within them. We have reported on the declining job opportunities for creative artists in China as AI image generation gains widespread adoption in the gaming industry. The impact of generative AI on other roles is just beginning to surface.
For customer service agents, increased retention rates and better performance among lower-skilled agents are clear outcomes of generative AI implementation. These improvements could lead to cost savings for companies as they benefit from higher performance levels and longer employee tenures.
However, the elevation of the lowest performers to the level of the highest performers may have other consequences for the broader labor market in customer support. As AI improves the quality of the least-skilled agents, job requirements and compensation may change as companies no longer need to compete for top talent.
The researchers highlight the importance of considering compensation for workers who contribute data to AI systems. “Our findings raise questions about whether and how workers should be compensated for the data that they provide to AI systems,” the team writes, “High-skill workers, in particular, play an important role in model development but see smaller direct benefits in terms of improving their own productivity.”
As generative AI continues to make a splash in numerous roles, overall efficiency gains may result in a reduction of available support agent roles. Notably, the software examined in the study was based on GPT-3. With the rapid advancements in LLMs, even more capable software utilizing OpenAI's recently released GPT-4 could have profound consequences, including a further increase in support agent efficiency.
Given the early stage of generative AI, the researchers agree: numerous consequences “deserve further scrutiny.”
NewsAI and Media Titans Quietly Hash Out Future of Content Licensing
June 16, 2023
ResearchIn Largest-Ever Turing Test, 1.5 Million Humans Guess Little Better Than Chance
June 09, 2023
NewsHigh-Profile AI Leaders Warn of “Risk of Extinction” from AI
May 30, 2023
NewsKey Takeaways from OpenAI CEO Sam Altman's Senate Testimony
May 16, 2023
NewsOpenAI Readies Open-Source Model as Competition Intensifies
May 15, 2023
ResearchChatGPT Trading Algorithm Delivers 500% Returns in Stock Market
May 10, 2023
NewsLeaked Google Memo Claiming “We Have No Moat, and Neither Does OpenAI” Shakes the AI World
May 05, 2023
NewsChegg’s Stock Tumble Serves as Wake Up Call on the Perils of AI
May 03, 2023
NewsHollywood Writers on Strike Grapple with AI’s Role in Creative Process
May 02, 2023
ResearchGPT AI Enables Scientists to Passively Decode Thoughts in Groundbreaking Study
May 01, 2023
NewsChatGPT Grows in Popularity as Bing and Bard Flatline
April 27, 2023
NewsSnap's My AI Feature Faces Unexpected Backlash from Users
April 24, 2023
News"Next to Impossible": OpenAI's ChatGPT Faces GDPR Compliance Woes
April 20, 2023
NewsMicrosoft's AI Chip Strategy Reduces Costs and Nvidia Dependence
April 18, 2023
News4 Million Accounts Compromised by Fake ChatGPT App
April 17, 2023
NewsEU's AI Act: Stricter Rules for Chatbots on the Horizon
April 14, 2023
ResearchStudy: Assigning Personas Creates a Sixfold Increase in ChatGPT Toxicity
April 13, 2023
ResearchGPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500,000 and 20,000 hours
April 11, 2023
ResearchGenerative Agents: Stanford's Groundbreaking AI Study Simulates Authentic Human Behavior
April 10, 2023
ResearchBye-Bye, Mechanical Turk? How ChatGPT is Making Humans Obsolete
April 09, 2023
NewsMayor Threatens Landmark Defamation Lawsuit Against OpenAI's ChatGPT
April 06, 2023
NewsOpenAI's ChatGPT Suspended in Italy Amid Privacy and Cybersecurity Concerns
March 31, 2023
NewsCiting "Profound Risks to Society," Prominent AI Experts Call for Pause
March 29, 2023
NewsEuropol Warns of ChatGPT's Dark Side as Criminals Exploit AI Potential
March 28, 2023
CultureAs Online Users Increasingly Jailbreak ChatGPT in Creative Ways, Risks Abound for OpenAI
March 27, 2023
NewsAI Researchers Voice Disappointment at GPT-4’s Lack of Openness
March 16, 2023