sientia-dataops-laborious-legacy/docs/sientia_model_drift_jensen_shannon_change.md

Specification: sientia_model — Jensen–Shannon drift (DriftAnalysis)

This document describes what sientia_model.analytics.drift_analysis.DriftAnalysis should change so downstream consumers (e.g. Laborious calculate_drift → Postgres drift_metrics.value NOT NULL) no longer receive NaN for Jensen–Shannon on valid finite data.

Scope

• File: sientia_model/analytics/drift_analysis.py
• Method: _jensen_shannon_distance(self, ref: np.ndarray, cur: np.ndarray, bins: int = 20) -> float
• Callers: detect_univariate_drift uses this for the jensen_shannon method; results are written to value in the drift dataframe.

Problem

The current implementation:

1. Builds bin edges from ref only: np.histogram(ref, bins=bins, density=True).
2. Builds the chunk histogram with density=True on the same edges: np.histogram(cur, bins=edges, density=True).

When every value in cur falls outside the closed support implied by those edges (typical case: production chunk drifted above the reference max or below the reference min), NumPy yields all-zero counts for cur. With density=True, normalization does 0/0, producing NaN for the whole histogram, which propagates to float('nan') in detect_univariate_drift → SQL NULL where value is NOT NULL.

This appears in Laborious when:

• model_config.target excludes the main target column from univariate features, so another feature (e.g. sensor_2) is compared chunk-by-chunk against the full reference series for that feature.
• Reference and current ranges do not overlap for some chunks (strong drift or different scaling).

Other methods (kolmogorov_smirnov, wasserstein) do not use this histogram+density path, so they can stay finite while Jensen–Shannon alone becomes null.

Required behavior

1. Finite output for finite ref and cur after removing non-finite values, whenever both sides have at least one usable sample.
2. Explicit handling of out-of-range chunk mass: probability mass from cur that does not fall into any bin defined from ref must still be represented (so the chunk distribution sums to 1), analogous to NannyML’s continuous JS approach (tail / “leftover” mass).
3. Missing values: drop NaN from ref and cur before computing. If either side is empty after that, return float('nan') (callers may filter or map; schema may still forbid null — product decision outside this spec).

Recommended algorithm (replace current body)

1. ref = np.asarray(ref, float); cur = np.asarray(cur, float).
2. ref = ref[~np.isnan(ref)]; cur = cur[~np.isnan(cur)].
3. If ref.size == 0 or cur.size == 0: return float('nan').
4. hist_ref, edges = np.histogram(ref, bins=bins) — counts, not density=True.
5. p = hist_ref.astype(float) / ref.size (reference bin probabilities).
6. hist_cur, _ = np.histogram(cur, bins=edges); q = hist_cur.astype(float) / cur.size.
7. leftover = 1.0 - float(np.sum(q)). If leftover > 1e-15 (tolerance for float noise), append leftover to q and 0.0 to p so both remain proper discrete distributions over the same extended support.
8. Apply small smoothing (existing module constant EPSILON is fine): add EPSILON to p and q, renormalize each to sum 1.
9. m = 0.5 * (p + q); compute symmetric JS via KL terms as today, e.g. inner = 0.5 * (sum(p*log(p/m)) + sum(q*log(q/m))).
10. Return sqrt(max(inner, 0.0)) to guard against tiny negative inner from floating-point error.

Non-goals / notes

• Numerical parity with the old density=True implementation is not required; parity with NannyML or scipy JS is desirable but optional. The priority is finite, interpretable drift when the chunk is outside the reference histogram range.
• Multivariate drift in the same file is unchanged by this spec.
• Tests in sientia_model should cover: (a) chunk entirely above reference max, (b) entirely below reference min, (c) overlapping range, (d) ref or cur all-NaN after cleaning.

Reference (external)

• NannyML continuous JS uses count-based bin probabilities and a leftover mass bin; see ContinuousJensenShannonDistance in nannyml/drift/univariate/methods.py (_calculate, leftover = 1 - np.sum(...)).
• Historical NaN-in-reference issue: NannyML#339 / #340 (orthogonal to out-of-range mass, but relevant for input cleaning).