7.4.3 Sentiment & Tone

Basic Prerequisites: 02 · Bag-of-Words & LM Dictionary (the lexicon) and OLS (Statistics & Inference).

---

1. Intuition & Practical Objective

Sentiment is the feature that made financial NLP famous. The objective: compress a whole document into one (or a few) numbers that a regression can price. Two competing vocabularies define the field:

• Tetlock’s media pessimism - the fraction of negative words in a news column, shown (2007) to exert downward pressure on returns followed by reversal. His construction is deliberately crude - negative-word fraction from a single lexicon - and that is its power: robust, reproducible, and out-of-sample meaningful.
• The LM net-tone score - with the finance-calibrated dictionary (02), the same fraction becomes tone = (pos − neg)/words, the workhorse feature for filings and transcripts.

Beyond raw sentiment, tone & readability matter: hedging quantifiers (“may”, “potentially”, “subject to”), the LM uncertainty and litigious lists, and readability indices (Fog, Flesch) capture how clearly management says things. A readable 10-K correlates with performance; a fog index that jumps signals obfuscation - a red-flag input to Accounting Quality & Red Flags.

The page’s payoff is one demonstration: build the tone feature, then regress next-day drift on it and see a clean, positive, priceable slope - the exact operation behind Tetlock and the whole dictionary track.

---

2. Mathematical Ground Truth & Derivations

The sentiment feature. With positive set $P$ and negative set $N$ (from a lexicon, LM for finance):

$$\text{tone}(d)=\frac{{\sum }_{t\in P}{x}_{d,t}-{\sum }_{t\in N}{x}_{d,t}}{{\sum }_t{x}_{d,t}},\text{neg-frac}(d)=\frac{{\sum }_{t\in N}{x}_{d,t}}{{\sum }_t{x}_{d,t}}.$$

Both are length-normalized so documents of different size are comparable; $\text{neg-frac}$ is Tetlock’s original scalar.

The regression. Let $y_d$ be the forward return after document $d$‘s release. The signal is linear in the text feature:

$$y_d={\beta }_0+{\beta }_1\text{tone}(d)+{\epsilon }_d.$$

OLS estimates $\beta$ by minimizing ${\sum }_d(y_d-{\beta }_0-{\beta }_1{\text{tone}}_d{)}^2$; the closed form in slope-intercept coordinates is

$${\hat{\beta }}_1=\frac{{\sum }_d({\text{tone}}_d-\overline{\text{tone}})(y_d-\bar{y})}{{\sum }_d({\text{tone}}_d-\overline{\text{tone}}{)}^2},{\hat{\beta }}_0=\bar{y}-{\hat{\beta }}_1\overline{\text{tone}}.$$

A positive, stable ${\hat{\beta }}_1$ - not a one-off - is the quantitative claim “tone is priced.” $R^2$ is the fraction of return variance explained; on single text features it is routinely tiny (a few percent or less), which is expected and not a failure - the alpha is in the residual the market leaves (low-SNR regime).

Readability (Fog index) - the canonical proxy for “how hard is this to read,” used in earnings-quality research:

$$\text{Fog}=0.4(\underset{\text{avg sentence length}}{\underbrace{\frac{\text{words}}{\text{sentences}}}}+\underset{\text{complex-word fraction}}{\underbrace{100\cdot \frac{\text{complex words}}{\text{words}}}}).$$

Higher Fog = harder text; large year-over-year increases in a 10-K’s Fog are the obfuscation signal Loughran–McDonald and earnings-quality work exploit.

---

3. Computational Implementation - sentiment scores, then the tone→return regression

Builds the net-tone feature for six news/transcript snippets and regresses a synthetic next-day drift on it, exactly as §2 specifies (numpy lstsq).

The monotone ordering is the point: sentiment −0.571 ↔ drift −4.0, sentiment +0.429 ↔ drift +8.0, and OLS recovers a slope of +14.244 (a +0.1 tone move ≈ +1.4% drift) with $R^2=0.79$ on this clean sample. On real data the slope survives but $R^2$ collapses toward a few percent - that residual is the tradeable low-SNR signal.

---

4. Failure Modes & First-Principles Breakdowns

1. Lexicon misfit dominates everything. With a generic (Harvard/Henry) dictionary the feature mislabels “liability”/“tax”/“gross” as negative, biasing the slope and potentially inverting the sign you think you’ve found (Loughran–McDonald 2011). First principle: the feature is only as calibrated as the lexicon, and valence is domain-conditional.
2. Regressing on the wrong return window / timestamp leaks the future. If the transcript’s sentiment is computed from the full call text but the return window opens when the call starts, you’ve used answers delivered an hour later. First principle: the feature at time $t$ may use only text knowable at $t$ - see 05.
3. Reading $R^2$ as “the signal.” A small $R^2$ on a text feature is not failure - it is the expected state of a weak, slowly-traded signal. The error is to conclude “no signal” from tiny $R^2$, or to conclude “huge alpha” from an in-sample $R^2$ that is only high because you overfit the n-grams (05).
4. Boilerplate inflation. In filings, much text is mandatory and sentiment-free; naive scores average it in and dilute the signal. Prefer active text (MD&A, Q&A) - the earnings-call refinement is in 06.

---

5. References

• Tetlock, Paul C., “Giving Content to Investor Sentiment,” Journal of Finance 62(3), 2007
• Loughran, Tim & McDonald, Bill, “When Is a Liability Not a Liability?” Journal of Finance 66(1), 2011
• Loughran, Tim & McDonald, Bill, “Measuring Readability in Financial Disclosures,” Journal of Finance 69(4), 2014
• Gentzkow, Kelly & Taddy, “Text as Data,” JEL 57(3), 2019

---

6. Connected Graph Bridges

• Back: 02 · Bag-of-Words & LM Dictionary · Index Hub
• Forward: 04 · Embeddings & Transformers · 06 · Earnings-Call Analysis
• Red-flag context: Accounting Quality & Red Flags