The Evolution of AI in Finance

In separate but related work, University of Notre Dame’s Tim Loughran and Bill McDonald explored sentiment analysis of 10-Ks, finding in a 2011 paper that the existing dictionary of words used to determine the sentiment of a text was not well suited to the financial domain. For example, words such as liability, cost, and tax were scored as negative for sentiment using the traditional dictionary, but these words are not necessarily negative when used in a financial context. Loughran and McDonald in turn created a dictionary tailored to finance.

Other researchers have developed new techniques for analyzing textual data. Boston University’s Tarek Alexander Hassan, Tilburg University’s Stephan Hollander, Frankfurt School of Finance and Management’s Laurence van Lent, and London Business School’s Ahmed Tahoun (then a research scholar at Booth) published research in 2019 that used a simple algorithm for assessing political risk in earnings call transcripts. It counted bigrams (two-word combinations including the constitution or public opinion) used in conjunction with the words risk and uncertainty, or their synonyms, to identify potential risks to companies. The higher the count, the greater the political risk for the company, the research finds. Subsequent papers resulted in a startup, NL Analytics, that works with central banks and international organizations to use these methods for economic surveillance.

The jumps that led to deeper understanding

Finance and accounting have long sought to learn from text. Economists have, too, and originally used a “bag-of-words” model. This relies on counting word frequency in a text—for example, how many times does a document include the words capital and spending? In this case, the more frequently these words occur, the more likely it is that the document discusses corporate policies.

This method is straightforward: in 1963, the late Frederick Mosteller and the late David L. Wallace used it to argue that James Madison, not Alexander Hamilton, had written 12 of the 85 essays and articles in the Federalist Papers whose authorship had been in dispute. By counting commonly used words in Madison’s and Hamilton’s known texts, they could compare them with the count of those words in the disputed articles in the Federalist Papers.

The method is also limited, however. It doesn’t take into account potentially important information such as grammar or the order in which words appear. As a result, it’s unable to capture much in terms of a document’s context. A company’s 10-K filing might report that “Increased transportation costs have offset our revenue gains,” and bag-of-words may interpret this as a positive statement—after all, the word increased and the phrase revenue gains might seem confident. But it misses the fact that increased taken with costs is negative and that offset changes the meaning of revenue gains.

Researchers at Google took a big step toward incorporating this context in 2013 when the company introduced word2vec, a neural network–based model that learns vector representations of words and captures the semantic relationships between them. Vectorization enabled ML models to process and understand text in a more meaningful way. If you have three related words, such as man, king, and woman, word2vec can find the next word most likely to fit in this grouping, queen, by measuring the distance between the vectors assigned to each word.

And in a 2017 paper, a team of researchers led by Ashish Vaswani, who was then at Google Brain, introduced what’s known by practitioners of deep learning as transformer architecture. Transformers form the basis of the large language models we know today and represent a significant improvement over previous architectures in their ability to understand and generate human language, which word-based models could not do.

One prominent LLM, BERT (bidirectional encoder representations from transformers), is used to understand the context of words but was not designed to generate text. It works by considering the words that appear before and after a particular word to decipher its meaning.

Meanwhile, GPT (generative pretrained transformer) is able to predict the most likely next word in a sequence based on the text leading up to it. For example, finish this sentence: “Why did the chicken cross the _____?” Your brain automatically fills in the blank with the word road as the most probable next word, even though many other words would work here, including street, highway, or maybe even yard. GPT does the same thing. Its parameters can be set, however, so that it doesn’t always choose the highest-probability word. This allows more creativity in the text it generates.

Now these LLMs, too, are tools that are being applied to finance, enabling researchers and practitioners in the field to extract increasingly valuable insights from data of all kinds.