WindAwake/Shutterstock
Text mining of quarterly earnings call transcripts has become a staple of investment research, as investors search the transcripts for clues about a company’s future profitability, among other things.
While some information is relatively straightforward to extract, other information is trickier. Take corporate investment policies, for example—those often aren’t stated simply, or at all. But Georgia State’s Manish Jha and Baozhong Yang, Georgia State PhD student Jialin Qian, and Chicago Booth’s Michael Weber designed a method using OpenAI’s ChatGPT that they suggest is capable of discovering these sometimes-hidden policies.
Earnings calls are the cocktail parties of corporate communications. In contrast to the formal prose in annual filings, the candid speak during earnings calls is how company managers often reveal their strategic plans and give estimates for their company’s future sales and profits. Particularly in the Q&A session between management and analysts, they sometimes dish out valuable private information.
But not every piece of information is stated plainly. For example, an executive may say, “We are investing in growth initiatives.” While the line doesn’t directly state as much, this might imply some large, upcoming capital expenditures that could affect near-term profitability. It can take an experienced analyst with the skills of a cold-war diplomat in understanding the unspoken intentions behind statements to discern such investment policy changes.
The researchers find that ChatGPT also has the ability to uncover such information. They gathered almost 75,000 transcripts of quarterly earnings calls from nearly 3,900 US public companies from 2006 to 2020. They also obtained financial and balance-sheet data and stock-return information.
They matched these transcripts to data from the Duke CFO Survey, focusing on the responses to a question about anticipated changes to a company’s investment policy. After the matching, they had a sample of about 1,300 quarterly transcripts with supporting data.
The researchers then fed the transcripts into ChatGPT 3.5, prompting it to report back on—using a 5-point scale ranging from “decrease substantially” to “increase substantially”—how much the company planned to change its capital spending over the following 12 months. They translated the responses into a score for each transcript, and then validated their system by comparing these scores to the corresponding CFO survey responses about corporate investment plans. They find the scores and responses to be highly correlated. (They also had ChatGPT give a one-sentence explanation for each scoring decision, which they compared with the score derived by ChatGPT, for a random sample, to ensure accuracy.)
The researchers fed nearly 75,000 earnings call transcripts into ChatGPT and asked it to provide a score indicating how much companies planned to change their capital spending over the coming year. The average score across all companies closely tracked the average change in the companies’ actual capital spending.
ChatGPT was able to decipher the corporate policy changes from the transcripts with a high degree of accuracy, the researchers write. The combination of ChatGPT’s huge knowledge base and “understanding” of context from its training makes it well-suited to analyzing conference calls, the researchers find. They contend that ChatGPT is even better suited than humans, as it does not have the capacity constraints humans have when it comes to processing a large amount of text in a short period of time. It doesn’t get tired and can remain consistent and objective in its assessments.
The study finds that a 1 standard deviation increase in a company’s investment score was associated with a 4 percent increase in expected capital spending over the following 12 months. It was also associated with a decrease of 1.8 percent in a company’s annualized return in the quarter subsequent to the earnings call, which makes sense considering that high-investment stocks generate lower returns than low-investment stocks.
The researchers suggest that their method can successfully forecast corporate investment for up to nine quarters into the future. The investment scores derived by ChatGPT can also predict how a company executive will respond to the CFO survey.
And ChatGPT’s usefulness is not limited to changes in investment policies. The researchers went back and tested their method on other corporate policy changes including dividends and employment and find it successful for those areas as well.
The empirical results suggest that “the market does not fully incorporate information already contained in public corporate earnings calls, and an advanced AI model like ChatGPT is able to extract such information efficiently,” the researchers write. Not only does it extract the information, they note, but it also provides information that is not currently captured by existing measures of investment. Particularly as the number of companies participating in surveys in general dwindles, the researchers’ method could become an attractive new tool for investors to add to their arsenal.
Manish Jha, Jialin Qian, Michael Weber, and Baozhong Yang, “ChatGPT and Corporate Policies,” Working paper, July 2023.
Chicago Booth’s Steve Kaplan says that private-equity firms frequently invest and grow companies more effectively than other owners.
What’s So Bad About Private Equity?
Harvard law professor John Coates discusses the potential dangers of financial consolidation.
Capitalisn’t: When a Few Financial Institutions Control Everything
Venture capitalists tend to rely too heavily on founders’ backgrounds.
How Startup Investors Could Back More WinnersYour Privacy
We want to demonstrate our commitment to your privacy. Please review Chicago Booth's privacy notice, which provides information explaining how and why we collect particular information when you visit our website.