7.4.1 Financial NLP from Zero

Basic Prerequisites: None beyond basic statistics; Index Hub for the map.

---

1. Intuition & Practical Objective

Why should a quant care about words at all? The numbers are right there in the 10-Q. The answer is incremental information: reported numbers are standardized, delayed, and gamed, but the words a firm chooses are partly uncontrolled - and uncontrolled information is what a signal is made of.

Three mechanisms make text genuinely informative:

1. Tone is priced but under-reacted. Tetlock (2007) showed that a single column’s pessimism predicts downward price pressure and then reversal - i.e. media text contains information the market does not fully absorb instantly. If it were fully absorbed, there would be nothing to measure. The residual drift is the alpha.

2. Text carries what the table of numbers cannot. A CEO’s prepared remarks and, more so, the live Q&A, contain forecasts, guidance language, hedging (“we expect”, “potentially”, “assuming”), evasiveness, and sentiment about future states. Cohen–Malloy–Nguyen (2020) find that firms that silently change their 10-K wording - with no number changing - underperform by over 7% a year: the wording itself is the disclosure.

3. Language choice is a discretionary accounting output. Earnings numbers are accrual estimates a manager chooses (Accounting Quality & Red Flags); the words are chosen even more freely. Both Dechow–Ge–Schrand (2010) and Healy–Wahlen (1999) model earnings quality as the gap between reported and real performance - text is the observable record of how management narrates that gap, and obfuscation is itself diagnostic.

The objective of this page is one idea: a document is a vector, and the counts in that vector are already a tradable feature. No deep learning required to start - the histogram of words already separates a bullish from a bearish call.

---

2. Mathematical Ground Truth & Derivations

The bag-of-words representation. A document $d$ is mapped to a count vector over a fixed vocabulary:

$${\mathbf{x}}_d=(x_{d,1},x_{d,2},\dots ,x_{d,|V|}),x_{d,t}=\#\{\text{occurrences of term }t\text{ in }d\}.$$

Order is discarded (“bag”), so “results beat estimates” and “estimates beat results” produce the same vector - the cheapness that makes bag-of-words scale to millions of documents, and the approximation embeddings will later fix (04).

From counts to a score. With a sentiment lexicon (a set $P$ of positive, $N$ of negative terms) the natural feature is the net-tone fraction:

$$\text{tone}(d)=\frac{{\sum }_{t\in P}{x}_{d,t}-{\sum }_{t\in N}{x}_{d,t}}{{\sum }_t{x}_{d,t}}.$$

This is the feature Tetlock regressed on returns and the feature the whole dictionary track of financial NLP is built on. Its statistical content: a fraction, so document length cancels - a 5-page and a 50-page filing are comparable.

Why the market doesn’t arbitrage it away. If tone forecast returns with certainty, prices would adjust on release and the forecast vanish. What survives is the weak but real correlation with drift that does not get fully traded out because it is noisy, slow, and mixed with the fundamental signal - exactly the low-SNR regime of Financial ML Pitfalls & Low-SNR.

---

3. Computational Implementation - counts already separate the two calls

The simplest possible text feature: count the words. Run this and see that the histogram alone separates a bullish from a bearish release - no model, no training.

The bull call nets $+0.292$; the bear call $-0.304$ - opposite signs, clean separation, from nothing but word counts and a tiny lexicon. That is the entire bag-of-words intuition; everything later is about making these counts calibrated (02) and regressed on returns (03).

---

4. Failure Modes & First-Principles Breakdowns

1. “More words is more informative” is false - rarefaction. A 50-page boilerplate 10-K is less informative per word than a 2-page press release. The first principle is that signal is fractional: divide by length (as §2 does) or the count is dominated by drafting verbosity, not information. Failing to normalize is the entry-level error.
2. Dictionary valence is domain-conditional. “Liability”, “tax”, “gross”, “unrealized” are accounting-normal words but look negative to a generic lexicon. Counting them as pessimism fabricates sentiment that does not exist - the exact misfit Loughran–McDonald (2011) measure and the reason 02 exists.
3. Timing: the text’s release date, not the call date, is the signal timestamp. A transcript exists publicly only after the call ends (often after close); a filing after the market is open is not tradeable at the open. Using the wrong timestamp is look-ahead by construction - see 05 and Data Sources & Corporate Data.

---

5. References

• Tetlock, Paul C., “Giving Content to Investor Sentiment,” Journal of Finance 62(3), 2007
• Loughran, Tim & McDonald, Bill, “When Is a Liability Not a Liability?” Journal of Finance 66(1), 2011
• Cohen, Lauren, Malloy, Christopher & Nguyen, Quoc, “Lazy Prices,” Journal of Finance 75(3), 2020
• Gentzkow, Matthew, Kelly, Bryan & Taddy, Matt, “Text as Data,” JEL 57(3), 2019
• Dechow, Patricia, Ge, Weili & Schrand, Catherine, “Understanding Earnings Quality,” 2010

---

6. Connected Graph Bridges

• Base: Financial ML Pitfalls & Low-SNR (the low-SNR regime text signals live in)
• Continue: 02 · Bag-of-Words & LM Dictionary · Index Hub
• Earnings context: Accounting Quality & Red Flags