The Doob–Meyer Decomposition: Separating Signal from Noise

Trend extraction feels intuitive. You look at a time series and ask: what part is structure, and what part is randomness?

But in stochastic systems, this question is fundamentally ill-posed unless you specify how information arrives over time.

The Doob–Meyer decomposition formalizes this boundary. It says that every “reasonable” stochastic process admits a canonical split into:

1. A component that is predictable given the past, and
2. A component that is unpredictable in the strongest possible sense.

The catch—often glossed over outside probability theory—is that this split is relative to a filtration. Change the information set, and the decomposition changes.

This post argues for a precise position:

• Signal versus noise is not an intrinsic property of a process.
• It is a property of a process viewed through an information structure.
• The Doob–Meyer theorem tells you exactly where that structure enters.

What you’ll get:

• Why drift is undefined without a martingale reference.
• How Doob–Meyer formalizes “trend” as conditional predictability.
• How this connects to filtering, state-space models, and modern time-series practice.

The Ill-Posedness of “Trend”

Let be an adapted stochastic process on a filtered probability space .

A naïve decomposition

is meaningless unless both terms are defined relative to .

To see why, note that for any adapted process , one can always write:

This identity already suggests a split into a predictable and an unpredictable part. But:

• The first term depends entirely on what is measurable at time .
• The second term is only “noise” relative to that same information set.

Without fixing the filtration, there is no canonical notion of drift.

Martingales as the Zero-Signal Baseline

A martingale formalizes the idea of no predictable trend.

A process is a martingale if:

Equivalently, the increment satisfies:

This is stronger than “mean zero noise.” It says:

Given everything you know at time , there is no systematic way to predict the future increment.

In this sense, martingales play the same conceptual role that white noise plays in classical time-series—but without stationarity, independence, or Gaussian assumptions.

If you want to define signal, you must define it as deviation from a martingale.

The Doob–Meyer Decomposition (Formal Statement)

Let be a càdlàg, integrable submartingale with respect to . Then there exists a unique decomposition:

such that:

1. is a martingale.
2. is a predictable, càdlàg, increasing process with .
3. The decomposition is unique up to indistinguishability.

The conditions matter:

• Submartingale: ensures “average growth” exists.
• **Predictability of :** ensures structure is known before it happens.
• Uniqueness: guarantees this is not just one decomposition among many.

This is not smoothing. It is not projection in . It is a structural theorem.

Drift as Conditional Structure

The increasing process captures all systematic, predictable growth of .

In discrete time, this becomes especially transparent. If is a submartingale, then:

and

Each increment of is the one-step-ahead conditional expectation of the increment of .

This makes the interpretation precise: * : what the process was expected to do, given the past. * : what happened beyond that expectation.

Nothing in this definition refers to smoothness, frequency, or parametric form. Only predictability matters.

Why Drift Is Not Intrinsic

Because is -measurable, it depends entirely on the filtration.

• Enlarge the filtration more structure moves from into .
• Shrink the filtration predictable structure collapses back into noise.

This resolves a common modeling confusion:

If residuals show structure, it is not evidence that “noise is structured.” > It is evidence that the filtration is too small.

This is exactly what happens when: 1. Adding latent state to a model reduces residual autocorrelation. 2. Incorporating exogenous variables turns apparent noise into drift.

The Doob–Meyer theorem says this is not heuristic—it is inevitable.

Relation to Filtering and State-Space Models

State-space models can be viewed as **parametric guesses for **.

A linear Gaussian state-space model asserts:

with the latent state expanding the filtration.

Kalman filtering is then a recursive approximation to the Doob–Meyer decomposition under Gaussian assumptions.

From this perspective: * Filtering constructing a richer filtration. * Residual diagnostics testing whether the remainder behaves like a martingale.

This reframes model checking: the goal is not “white noise,” but martingale difference noise.

Conceptual Parallel: Spectral and Koopman Lenses

There is a direct analogy to spectral and Koopman methods:

• Spectral analysis separates dynamics by frequency.
• Koopman analysis separates dynamics by eigenfunctions.
• Doob–Meyer separates dynamics by predictability.

Each framework answers a different question. None of them define structure in isolation. The mistake is treating any decomposition as absolute rather than conditional on a lens.

Practical Rules of Thumb

1. If you cannot state your filtration, you cannot define drift.
2. Trend extraction without an information model is ill-posed.
3. Martingale noise is the correct irreducible baseline.
4. White noise is a special—and fragile—case.
5. Structured residuals mean missing state, not bad randomness.
6. Better smoothing cannot replace a richer filtration.

Closing

The Doob–Meyer decomposition is often introduced as a technical lemma en route to stochastic calculus. That framing misses its conceptual weight.

It is a theorem about what can be known in stochastic systems—and when.

Signal exists only relative to information. Noise is what remains once predictability is exhausted.

The decomposition does not discover this boundary. It enforces it.