My ML Feature Used the Wrong Unit. The Threshold Was 400x Too Large.

The historical model and the live validator both calculated large_share, the fraction of sweep volume in unusually large trades. Their definitions had diverged.

The live code used the 75th percentile of per-second net aggressive volume as the cutoff for an individual trade. On the affected symbols, that threshold was 163–423 times too large. Almost no individual trade cleared it, so large_share was near zero in 97% or more of live events.

The public audit package includes the minimal reproducible fixture, the feature-contract checks, and the recorded aggregate results used here. It does not include the underlying market-data archive.

Evidence package: open the feature-contract tests and frozen raw-replay summary on GitHub.

For AAVEUSDT, the 75th percentile of individual trade quantity was 4.3 units. The 75th percentile of per-second NAV was 1,730 units. Both are volumes; they are not interchangeable.

large_thr = np.percentile(np.abs(dev["nav_abs"]), 75)
# nav_abs: per-second net aggressive volume

large_share = volume_of_trades_at_or_above(large_thr) / total_volume
# threshold is applied to individual trades

The correct calibration comes from individual trade quantities. The repaired live path refuses to score a symbol when it lacks a matching calibration artifact.

Three other feature contracts also differed

The feature-parity audit found three other differences between training and live:

The large_share defect was the main false-positive driver. In the first analytical triage of 14 October, fixing it alone reduced selected candidates from 40 to 5. Applying all four corrections left 2.

Those are diagnostic counts from before the October raw-trade archive was recovered; two feature values were approximated in that first pass.

We subsequently recovered Binance Futures aggTrades for 6–14 October. The raw reconstruction keeps the recorded candidate timestamps and uses the matching frozen artifacts: V1 live for 6–13 October (13,786 candidates) and V2 full baseline for 14 October (895).

It rebuilt the four affected feature families for 14,681 of 14,682 candidates. One LDOUSDT detection second was absent from the archive. The frozen artifacts reproduced recorded scores with a mean absolute difference of 1.04×10⁻⁶.

The repaired features selected 52 candidates. Their raw five-second Binance markout averaged +8.67 bps. With a 10 bps round-trip fee assumption, that becomes −1.33 bps. No strategy was promoted.

The raw archive resolves the four-feature reconstruction gap. A five-second Binance markout is a research measure. Candidate generation and the logged nav_abs / nav_overshoot convention remain fixed, so this is not an execution study.

Drift monitoring comes after parity

A KS test can show that two feature distributions differ. It does not establish whether one implementation used individual trades while another used one-second aggregates.

Before monitoring feature drift, run both implementations against the same recorded raw inputs:

for fixture in recorded_sweeps:
    historical = batch_features(fixture.trades, fixture.direction)
    live = streaming_features(fixture.trades, fixture.direction)

    assert historical.large_share == live.large_share
    assert historical.run_frac == live.run_frac
    assert historical.pre_vel == live.pre_vel
    assert historical.trailing_nav == live.trailing_nav

Record the threshold's unit, symbol, source period, and input hash beside the calibration artifact. Check feature parity before interpreting a model score.

Part of the HFM Research methodology series.