7.4 Financial NLP & Transcripts

Basic Prerequisites: Statistics & Inference (regression on the feature matrix) and Financial ML Pitfalls & Low-SNR (why a text-derived signal dies without leak-proof evaluation). The inputs being mined are described in Data Sources & Corporate Data. (these are the folder-level prerequisites for pages 02–06; page 01 states its own, smaller, entry requirements)

1. Intuition & Practical Objective

Over 80% of actionable corporate information arrives as unstructured text: 10-K/10-Q filings, earnings-call transcripts, press releases, analyst reports, and news wires. The discretionary analyst’s skill is reading between the lines - tone, hedging, evasiveness, and year-over-year phrasing shifts. Financial NLP is that reading made algorithmic, repeatable, and testable at scale: quantify a document’s sentiment, measure how much the words changed from last quarter, and regress those text features on subsequent returns.

The empirical spine is three results every quant should know cold:

1. Tetlock (2007) - the negative-word fraction of the Wall Street Journal’s “Abreast of the Market” column predicts down-pressure on returns and is followed by reversal; media pessimism is a measurable, market-moving feature.
2. Loughran & McDonald (2011) - generic sentiment dictionaries (Harvard’s General Inquirer, Henry’s list) are badly miscalibrated for finance: “tax”, “cost”, “liability”, “share”, “gross”, and “unrealized” get mislabeled because they are not actually negative in an accounting context. Finance needs its own dictionary - the LM lists - built by hand-sorting 10-K word frequencies.
3. Gentzkow, Kelly & Taddy (2019) - “Text as Data” unifies the machinery: any text corpus is a sparse count matrix, and the statistical question is always the same - which low-dimensional function of the counts predicts an outcome (a return, a label, a topic) out of sample.

The one-sentence essence. “Text is a signal when the words a firm chose - not just its reported numbers - carry incremental information, and that signal is extracted by turning documents into numeric feature vectors (dictionaries → bags of words → embeddings) and regressing the outcome on them without leaking the future.”

This folder is the hub. It (a) gives the fast bag-of-words → TF-IDF → sentiment → regression pipeline and its two killer lookup numbers (the LM negative-fraction and the $p\gg N$ overfitting cost) below, and (b) routes to six sub-pages that climb from raw intuition to transformer-based earnings-call analysis.

2. Mathematical Ground Truth & Quick Lookup

Notation. Document $d$ as a count vector over the vocabulary: ${\mathbf{x}}_d\in {\mathbb{N}}^{|V|}$, where $x_{d,t}$ = number of times term $t$ appears in $d$. $N$ = number of documents, $d{f}_t$ = document frequency (docs containing $t$).

TF–IDF weight (the standard feature rescaling that tames ubiquitous words):

Cosine similarity (the Loughran–McDonald/Tetlock workhorse for “how similar are two documents”, and the basis of the Cohen–Malloy–Nguyen linguistic-change metric):

Net sentiment / tone score (the single most-used text feature):

Text-to-signal regression (the pricing of the feature):

3. Computational Implementation - the whole pipeline in ~40 lines

The full ingest → clean → features → model chain, reproduced identically on the sub-pages (stdlib + numpy only).

4. Failure Modes & First-Principles Breakdowns (hub signposts)

The full analysis lives in 05 · Failure Modes & Practice. In one line each:

1. Dictionary misfit - applying a general (non-finance) lexicon to 10-Ks mislabels “liability”, “tax”, “gross” as negative; the first principle is that a word’s valence is domain-conditional (Loughran–McDonald 2011).
2. Look-ahead from restated/leaked text - a 10-K you read today contains restatements a trader did not have at the close; a transcript’s full text can embed a number only released after the call. Text must be timestamped point-in-time exactly like numbers (see Data Sources & Corporate Data).
3. Overfitting with many n-grams - a vocabulary of thousands of sparse binary features on a few hundred filings is the canonical $p\gg N$ problem (ESL Ch 18): in-sample fit climbs to $R^2=1$ while out-of-sample collapses (this folder’s run: $1.000$ vs $-0.433$).

5. References

• Loughran, Tim & McDonald, Bill, “When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks,” Journal of Finance 66(1):35–65, 2011
• Tetlock, Paul C., “Giving Content to Investor Sentiment: The Role of Media in the Stock Market,” Journal of Finance 62(3):1139–1168, 2007
• Gentzkow, Matthew, Kelly, Bryan & Taddy, Matt, “Text as Data,” Journal of Economic Literature 57(3):535–574, 2019
• Araci, Dogu, “FinBERT: Financial Sentiment Analysis with Pre-trained Language Models,” arXiv:1908.10063, 2019; and Yang, Yi, Uy, Mark C.S. & Huang, Allen, “FinBERT: A Pretrained Language Model for Financial Communications,” arXiv:2006.08097, 2020
• Cohen, Lauren, Malloy, Christopher & Nguyen, Quoc, “Lazy Prices,” Journal of Finance 75(3):1371–1415, 2020
• Hastie, Tibshirani & Friedman, The Elements of Statistical Learning

6. Connected Graph Bridges

• Foundational base: Statistics & Inference · Financial ML Pitfalls & Low-SNR (the evaluation discipline every text signal inherits)
• Data provenance: Data Sources & Corporate Data (point-in-time filings - where the text comes from) · Accounting Quality & Red Flags (earnings management the text may reveal)
• Sibling topic: Alternative Data Pipelines & Evaluation (alt-data hygiene)
• Sub-pages (in-folder): 01 From Zero · 02 Bag-of-Words & LM Dictionary · 03 Sentiment & Tone · 04 Embeddings & Transformers · 05 Failure Modes · 06 Advanced Extensions (Earnings-Call Analysis)

Beginner: start at 01 · Practitioner: start at 05