EDGAR-CORPUS: Billions of Tokens Make The World Go Round

Introduction

Natural Language Processing (nlp) for economics and finance is a rapidly developing research area Hahn et al. (2018, 2019); Chen et al. (2020); El-Haj et al. (2020). While financial data are usually reported in tables, much valuable information also lies in text. A prominent source of such textual data is the Electronic Data Gathering, Analysis, and Retrieval system (edgar) from the us Securities and Exchange (sec) website that hosts filings of publicly traded companies.See https://www.sec.gov/edgar/searchedgar/companysearch.html for more information. In order to maintain transparency and regulate exchanges, sec requires all public companies to periodically upload various reports, describing their financial status, as well as important events like acquisitions and bankruptcy.

In this paper, we release edgar-corpus, a novel financial corpus containing all the us annual reports (10-K filings) from 1993 to 2020.edgar-corpus is available at: https://zenodo.org/record/5528490 Each report is provided in an easy-to-use json format containing all 20 sections and subsections (items) of a sec annual report; different items provide useful information for different tasks in financial nlp. To the best of our knowledge, edgar-corpus is the largest publicly available financial corpus (Table 1). In addition, we use edgar-corpus to train and release word2vec embeddings, dubbed edgar-w2v. We experimentally show that the new embeddings are more useful for financial nlp tasks than generic glove embeddings (Pennington et al., 2014) and other previously released financial word2vec embeddings (Tsai et al., 2016). Finally, to further facilitate future research in financial nlp, we open-source edgar-crawler, the Python toolkit we developed to download and extract the text from the annual reports of publicly traded companies available at edgar.edgar-crawler is available at: https://github.com/nlpaueb/edgar-crawler

Related Work

There are few textual financial resources in the nlp literature. Händschke et al. (2018) published joco, a corpus of non-sec annual and social responsibility reports for the top 270 us, uk, and German companies. Daudert and Ahmadi (2019) released cofif, the first financial corpus in the French language, comprising annual, semestrial, trimestrial, and reference business documents.

While some previous work has published document collections from edgar, those collections come with certain limitations. Kogan et al. (2009) published a collection of the Management’s Discussion and Analysis Sections (Item 7) for all sec company annual reports from 1996 to 2006. Tsai et al. (2016) updated that collection to include reports up to 2013 while also providing word2vec embeddings. Finally, Lee et al. (2014) released a collection of 8-K reports from edgar, which announce significant firm events such as acquisitions or director resignations, from 2002 until 2012.

Compared to previous work, edgar-corpus contains all 20 items of the annual reports from all publicly traded companies in the us, covering a time period from 1993 to 2020. We believe that releasing the whole annual reports (with all 20 items) will facilitate several research directions in financial nlp Loughran and McDonald (2016). Also, edgar-corpus is much larger than previously published financial corpora in terms of tokens, number of companies, and year range (Table 1).

Creating edgar-corpus

Publicly-listed companies are required to submit 10-K filings (annual reports) every year. Each 10-K filing is a complete description of the company’s economic activity during the corresponding fiscal year. Such reports also provide a full outline of risks, liabilities, corporate agreements, and operations. Furthermore, the documents provide an extensive analysis of the relevant sector industry and the marketplace as a whole.

A 10-K report is organized in 4 parts and 20 different items (Table 2). Extracting specific items from documents with hundreds of pages usually requires manual work, which is time- and resource-intensive. To promote research in all possible directions, we extracted all available items using an extensive pre-processing and extraction pipeline.

In more detail, we developed edgar-crawler, which we used to download the 10-K reports of all publicly traded companies in the us between the years 1993 and 2020. We then removed all tables to keep only the textual data, which were html-stripped,We use Beautiful Soup (https://beautiful-soup-4.readthedocs.io/en/latest). cleaned and split into the different items by using regular expressions. The resulting dataset is edgar-corpus.

While there exist toolkits to download annual filings from edgar, they do not support the extraction of specific item sections.For example, sec-edgar can download complete html reports (with images and tables), but it does not produce clean item-specific text. This is particularly important since researchers often rely on certain items in their experimental setup. For example, Goel and Gangolly (2012), Purda and Skillicorn (2015), and Goel and Uzuner (2016) perform textual analysis on Item 7 to detect corporate fraud. Katsafados et al. (2020) combine Item 7 and Item 1A to detect Initial Public Offering (ipo) underpricing, while Moriarty et al. (2019) combine Item 1 with Item 7 to predict mergers and acquisitions.

Apart from edgar-corpus, we also release edgar-crawler, the toolkit we developed to create edgar-corpus, to facilitate future research based on textual data from edgar. edgar-crawler consists of two Python modules that support its main functions:

edgar_crawler.py is used to download 10-K reports in batch or for specific companies that are of interest to the user.

extract_items.py extracts the text of all or particular items from 10-K reports. Each item’s text becomes a separate json key-value pair (Figure 1).

2 Word embeddings

To facilitate financial nlp research, we used edgar-corpus to train word2vec embeddings (edgar-w2v), which can be used for downstream tasks, such as financial text classification or summarization. We used word2vec’s skip-gram model Mikolov et al. (2013a, b) with default parameters as implemented by gensim (Řehůřek and Sojka, 2010) to generate 200-dimensional word2vec embeddings for a vocabulary of 100K words. The word tokens are generated using spaCy Honnibal et al. (2020). We also release edgar-w2v.The edgar-w2v embeddings are available at: https://zenodo.org/record/5524358

To illustrate the quality of edgar-w2v embeddings, in Figure 2 we visualize sampled words from seven different entity types, i.e., location, industry, company, year, month, number, and financial term, after applying dimensionality reduction with the umap algorithm McInnes et al. (2018). The financial terms are randomly sampled from the Investopedia Financial Terms Dictionary.https://www.investopedia.com/financial-term-dictionary-4769738. In addition, companies and industries are randomly sampled from well-known industry sectors and publicly traded stocks. Finally, the words belonging to the remaining entity types are randomly sampled from gazetteers. Figure 2 shows that words belonging to the same entity type form clear clusters in the 2-dimensional space indicating that edgar-w2v embeddings manage to capture the underlying financial semantics of the vocabulary.

To further highlight the semantics captured by edgar-w2v embeddings, we retrieved the 5 nearest neighbors, according to cosine similarity, for commonly used financial terms (Table 3).We exclude obvious top-scoring neighbors of singular/plural pairs such as market/markets or investor/investors. As shown, all the nearest neighbors are highly related to the corresponding term. For instance, the word economy is correctly associated with terms indicating the slowdown of the economy happening during the past few years, e.g., downturn, recession, or slowdown. Also, market is correctly related with words such as marketplace, industry, and prices.

Experiments on financial NLP tasks

We also compare edgar-w2v embeddings against generic glove (Pennington et al., 2014) embeddingsWe use the 200-dimensional glove embeddings from https://nlp.stanford.edu/data/glove.6B.zip. and the financial embeddings of Tsai et al. (2016) in three financial nlp tasks. For each task, we use the same model and we only alter the embeddings component. In addition, we use the same pre-processing during the creation of the vocabulary of the embeddings in each case.

finsim-3 Juyeon Kang and Gan (2021) provides a set of business and economic terms and the task is to classify them into the most relevant hypernym from a set of 17 possible hypernyms from the Financial Industry Business Ontology (fibo).https://spec.edmcouncil.org/fibo/. Example hypernyms include Credit Index, Bonds, and Stocks. We tackle the problem with a multinomial logistic regression model, which, given the embedding of an economic term, classifies the term to one of the 17 possible hypernyms. Since finsim-3 is a recently completed challenge, the true labels for the test data were not available. Therefore, we use a stratified 10-fold cross-validation. We report accuracy and the average rank of the correct hypernym as evaluation measures. For the latter, the 17 hypernyms are sorted according to the model’s probabilities. A perfect model, i.e., one that would always rank the correct hypernym first, would have an average rank of 1.

Financial tagging (fint) is an in-house sequence labeling problem for financial documents. The task is to annotate financial reports with word-level tags from an accounting taxonomy. To tackle the problem, we use a bilstm encoder operating over word embeddings with a shared multinomial logistic regression that predicts the correct tag for each word from the corresponding bilstm state. We report the F1 score micro-averaged across all tags.

fiqa Open Challenge Maia et al. (2018) is a sentiment analysis regression challenge over financial texts. It contains financial tweets annotated by domain experts with a sentiment score in the range , with 1 denoting the most positive score. For this problem, we employ a bilstm encoder which operates over word embeddings, and a linear regressor operating over the last hidden state of the bilstm. Since we do not have access to the test set in this task, we use a 10-fold cross-validation. We evaluate the results using Mean Squared Error (MSE) and R-squared ($R^{2}$).

Across all tasks, edgar-w2v outperforms glove, showing that in-domain knowledge is critical in financial nlp problems (Table 4). The gains are more substantial in finsim-3 and fint, which rely to a larger extent on understanding highly technical economics discourse. Interestingly, the in-domain embeddings of Tsai et al. (2016) are comparable to the generic glove embeddings in two of the three tasks. One possible reason is that Tsai et al. (2016) employed stemming during the creation of the embeddings vocabulary, which might have contributed noise to the models due to loss of information.

Conclusions and Future Work

We introduced and released edgar-corpus, a novel nlp corpus for the financial domain. To the best of our knowledge, edgar-corpus is the largest financial corpus available. It contains textual data from annual reports published in edgar, the repository for all us publicly traded companies, covering a period of more than 25 years. All the reports are split into their corresponding items (sections) and are provided in a clean, easy-to-use json format. We also released edgar-crawler, a toolkit for downloading and extracting the reports. To showcase the impact of edgar-corpus, we used it to train and release edgar-w2v, which are financial word2vec embeddings. After illustrating the quality of edgar-w2v embeddings, we also showed their usefulness in three financial nlp tasks, where they outperformed generic glove embeddings and other financial embeddings.

In future work, we plan to extend edgar-crawler to support additional types of documents (e.g., 10-Q, 8-K) and to leverage edgar-corpus to explore transfer learning for the financial domain, which is vastly understudied.

Disclaimer

This publication contains information in summary form and is therefore intended for general guidance only. It is not intended to be a substitute for detailed research or the exercise of professional judgment. Member firms of the global EY organization cannot accept responsibility for loss to any person relying on this article.

References