Uncertainty Under Regime Shifts: Why Adaptive Conformal Beats Parametric Volatility

In production forecasting, we often fixate on point accuracy. We obsess over minimizing RMSE or MAE, tweaking architectures until the forecast line tracks the actuals reasonably well. But in high-stakes environments—energy grids, logistics, or algorithmic trading—the point forecast is rarely enough. The real question isn't just "What is the value?" but "How wrong could we be?"

Most forecasting models implicitly assume stationarity. They assume the rules governing the past will apply neatly to the future. In many real-world systems, that assumption fails systematically due to regime changes, physical constraints, or structural breaks.

When this happens, uncertainty estimates are often the first thing to fail. While the point forecast might remain visually acceptable, the "95% confidence interval" silently stops covering 95% of the data. This post explores why standard parametric methods (like GARCH) break down under regime shifts and how Adaptive Conformal Inference (ACI) offers a robust, distribution-free alternative.

The Setup: One Forecaster, Two Wrappers

To isolate the problem of uncertainty quantification, we need to control the forecasting engine. For these experiments, we use a fixed Koopman-style linear operator model.

Without getting bogged down in the math, this model lifts time-series data into a higher-dimensional space where the dynamics are approximately linear. It provides a stable, consistent baseline for point forecasts.

The variable we are testing is the uncertainty wrapper. We compare two distinct approaches to generating prediction intervals around that fixed forecast:

1. Parametric Volatility (GARCH): The industry standard for modeling time-varying variance. It assumes the residuals follow a specific distribution (usually conditional normality) and that the volatility structure is stationary.
2. Adaptive Conformal Inference (ACI): A modern, distribution-free approach. It builds intervals based on empirical residuals rather than assumed distributions. Crucially, it uses an online update mechanism to recalibrate residual quantiles on the fly.

The core difference lies in how they treat the residual, :

GARCH tries to model the variance of using a fixed set of parameters. ACI simply asks: "What quantile of the recent historical residuals do I need to cover the next point?"

Baseline Sanity Check: When Parametric Works (Retail)

Before breaking the model, let's look at a scenario where things work well: High-frequency retail sales data.

This system is characterized by stable, periodic dynamics. The spectral structure (as seen in the eigenvalue plot) is consistent. Because the underlying process is close to stationary, the residuals behave "nicely."

In this regime, both GARCH and ACI perform well. The coverage hovers near the nominal 95%, and the interval widths are comparable.

Key Takeaway: In regimes with stable dynamics, even strong parametric assumptions can appear well calibrated. If your data never undergoes structural shifts, parametric volatility is a perfectly valid choice.

The Breaking Point: Solar Power and the Hard Zero

The illusion of safety vanishes when we move to a system with structural boundaries. Solar power generation provides a perfect test case because of the "hard zero"—the sun goes down every night.

This introduces a systematic regime shift. The distribution of residuals during the day (high variance, potential cloud cover) is fundamentally different from the distribution at night (zero variance, deterministic). This violates the core assumptions of stationarity and homoskedasticity.

Why GARCH Struggles

As shown in the comparison above, GARCH struggles to handle the transition. Because it relies on historical volatility to predict future variance, it is slow to react.

• Lag: When the sun sets, GARCH still predicts high variance based on the afternoon's volatility.
• Inflation: To maintain average coverage, it inflates intervals globally. You end up with wide uncertainty bands at night, which is physically impossible.

Why ACI Succeeds

Adaptive Conformal Inference behaves differently. Because it makes no distributional assumptions, it doesn't "know" that variance should be continuous. It simply tracks the empirical error.

When the error drops to zero at night, the ACI intervals collapse to near-zero immediately. When variance spikes in the morning, the intervals expand.

We can rely on the conformal coverage guarantee:

Crucially, this guarantee holds marginally even when the data-generating process changes—provided the calibration step is allowed to adapt online.

A Critical Nuance: ACI does not make the point forecast better. The underlying Koopman model still produces the same errors. ACI simply reallocates the uncertainty correctly, tightening where confidence is high and expanding where it is low.

The Limits of Adaptation: Wind Power

It is important not to view Adaptive Conformal Inference as a magic wand. To demonstrate its limits, we look at wind power generation.

Unlike solar, which has a predictable regime shift, wind dynamics are chaotic and driven by complex atmospheric physics. In our experiment, we observe that while ACI ensures coverage, the quality of the intervals degrades.

The issue is visible in the residual autocorrelation plot (ACF). The Ljung-Box test rejects the null hypothesis of white noise. This indicates that the residuals contain structure that the linear Koopman model failed to capture.

• Coverage: ACI adapts and ensures we capture the true value 95% of the time.
• Utility: Because the residuals are dependent, the intervals become reactive rather than predictive. The intervals widen significantly to "catch" the wandering mean of the process.

This highlights a fundamental rule of forecasting:

Adaptation corrects calibration error, not modeling error.

If your residuals show strong autocorrelation, better uncertainty quantification is a band-aid. The root cause is a model that lacks the necessary state or complexity (e.g., needing spatial weather data) to capture the dynamics.

Summary and Rules of Thumb

Uncertainty is not an afterthought—it is usually the first thing to break when your data shifts. Based on these experiments, here are three rules of thumb for choosing your uncertainty wrapper:

1. Stable Dynamics (e.g., Retail): Parametric methods (GARCH) are sufficient and computationally efficient.
2. Regime Shifts / Constraints (e.g., Solar): Adaptive calibration (ACI) is essential. It prevents "smearing" uncertainty across regimes and respects physical boundaries better than parametric assumptions.
3. Persistent Residual Dependence (e.g., Wind): Adaptation has limits. If your residuals are heavily autocorrelated, you don't need a better uncertainty wrapper—you need a better model class.