8.6.6 Advanced Extensions

Basic Prerequisites: 04 · Reproducibility and 05 · Failure Modes.

---

1. Intuition & Practical Objective

The research-to-production gap is the set of environmental differences between the notebook where a strategy is discovered and the server where it is deployed. Each difference is a failure mode from 05 waiting to happen. This page is the launchpad to the tooling that closes it: environment management (pin every dependency), config management (pin every parameter), and experiment tracking (record every run’s inputs and outputs so any result can be attributed and reproduced).

Three extensions of the discipline:

1. Environment management - uv/Poetry/conda-lock lockfiles and Docker images turn “same code” into literally the same binaries, so a dependency bump cannot silently drift the result.
2. Config management - parameters (seed, universe, cost model, z_max threshold) are themselves inputs to $f(D,C,E)$; they must be versioned alongside data and code, not edited in place in a notebook cell.
3. Experiment tracking - every run records its full $(D,C,E,\text{config})$ plus the output hash. With that record, any result is auditable: you can re-run it, and you can answer “which runs used data version v2?“.

The one-line job: make the research and production environments identical by construction - so that the pipeline that found the edge is the same pipeline that trades it.

---

2. Mathematical Ground Truth & Derivations

Parallel reduction is where reproducibility breaks first. When a computation is split across workers (threads, cores, GPU blocks) and results are summed, the order of aggregation becomes an input to the result. IEEE-754 doubles have ~16 significant digits; adding a small term after a huge one absorbs it. Formally, two different reduction orders over partial sums $\{s_0,\dots ,s_{K-1}\}$ can yield

$$R_A={\sum }_{\text{order A}}{s}_i\ne R_B={\sum }_{\text{order B}}{s}_i,$$

with the difference up to the magnitude of the absorbed low-order terms. A nondeterministic scheduler (whichever thread finishes first) therefore breaks reproducibility even with all inputs pinned - the canonical reason frameworks expose a fixed reduction order or a pairwise/tree reduction.

Experiment tracking is a function, not a log. Each run is a point in input space mapping to an output:

$$\text{run}:(D,C,E,\theta )\mapsto O,O=f(D,C,E,\theta ).$$

“Reproducing a result” = recovering the recorded input tuple and re-evaluating $f$. Versioning the tuple (data hash, commit, lockfile, config) is what makes the mapping invertible - the mathematical content of an experiment tracker.

The research-to-production gap as a difference of triples. Research runs on $(D_r,C_r,E_r)$, production on $(D_p,C_p,E_p)$. The gap is $\Vert \text{distance}((D_r,C_r,E_r),(D_p,C_p,E_p))\Vert$: every nonzero component is a source of divergence that pinning removes. Container images + lockfiles drive each component to zero by construction.

---

3. Computational Implementation - parallel reduce, two orders, bit-verified

Simulate a threaded map-reduce (threads as workers producing shard partial sums), then aggregate the four shards in two different orders and show the answer changes - the failure mode experiment tracking must record (the reduction order) alongside the inputs. Stdlib only; uses threading (safe under python3 -c).

The two lines are the entire research-to-production story in miniature: reordering the same shards changes the answer, but pinning the order makes it bit-reproducible. An experiment tracker that records only the inputs - not the reduction order - cannot reproduce $1.0$ vs $0.0$. Recording the full run tuple (inputs + order + environment) is what makes any result auditable.

---

4. Failure Modes & First-Principles Breakdowns

1. Research and production environments diverge. The most common gap: research in a notebook with hand-pinned versions, production in a container with different (often newer) dependencies. One bump in a math library can shift results - see §2. Close it with lockfiles + images so $E_r=E_p$.
2. Config edited in place, not versioned. A z_max threshold or universe list changed in a notebook cell is a silent change to $f$‘s inputs. It must live in a versioned config that the tracker records, or no run can be reproduced.
3. Nondeterministic parallel reduction. If the aggregation order depends on thread scheduling, even a fully pinned run is not reproducible. Use a fixed/tree reduction and record it (this page’s demo).
4. Tracking outputs but not inputs. A log of “result = 0.0342” with no $(D,C,E,\theta )$ record is a tombstone, not an audit trail. Track the tuple, and the output is implicit.

---

5. References

• Astral uv / Poetry / conda-lock / Docker
• López de Prado - Advances in Financial Machine Learning
• NautilusTrader Documentation (nautilustrader.io)
• Needham & Simons - DuckDB in Action

---

6. Connected Graph Bridges

• Back: 05 · Failure Modes & Practice · Index Hub
• Forward (the gap’s destination): Production Risk Guards & Kill Switches · Backtesting Engines
• Environment foundation: Python Quant Stack (the stack the lockfile manages)