Teaching a Vehicle to Distrust Its Own Sensors

An automated vehicle fuses several noisy sensors into one estimate of where it is and how fast it is going, and then it acts on that estimate. So the estimate had better be trustworthy, and the hard question is what happens when it is not. A GPS fix jumps. An IMU drifts. A wheel slips. Two things decide whether the system is safe: does it notice, and does it fail safe instead of failing. Over the last stretch of an open, simulation-only build I put that whole integrity layer together, from noticing a bad reading to shipping the result so a stranger can reproduce every number. None of the underlying methods are mine to claim. The work, and the point of this write-up, is how the standard pieces get wired into something you can actually trust.

01 Deciding when to stop is harder than detecting the fault

Detecting a lying sensor is the easy half. A GPS jump produces a large, instant surprise against what the fusion expected, and a standard consistency check gates it out. The hard half is deciding what to do next, because that decision is where a system is safe or dangerous. Stop for every bit of sensor noise and the vehicle is useless. Keep driving on an answer it should not trust and it is unsafe.

The reframe that made this tractable was to stop counting faulty sensors and ask a different question: can I still check my own estimate. One lying sensor is outvoted by the others, isolated, and gated, and the estimate stays good. That is a degraded mode you drive through, not an emergency. The state that earns a stop is subtler. It is when the sensors that would catch a bad answer are themselves compromised, so there is nothing trustworthy left to pull the estimate back, and no way even to tell how wrong it has become. Three states fall out of that one honest criterion: healthy, degraded, and safe-stop, decided by whether any trustworthy correction remains.

02 The fault your own system calls healthy

The fast per-reading check has a deliberate blind spot, and the honest thing was to show it failing before fixing it. A moderate IMU bias sails straight through as healthy, because the fusion simply absorbs it. The good sensors quietly correct for the bias every step, the position estimate stays fine, and no single reading looks surprising enough to flag.

But absorbed is not the same as gone. To keep absorbing a biased prediction, the corrections have to pull the same way every step, and that constant one-sided pull leaves a fingerprint: the errors stop being random noise around zero and pick up a persistent lean, each one far too small to trip a per-reading alarm. So the second monitor watches the running average of that error over a window, using the filter's own prediction as a small digital twin of expected behaviour. A healthy sensor averages to zero. A bias pushes the average off zero long before any one reading looks odd. It trades speed for sensitivity, and it caught the exact bias the first check had waved through, about a second in. Loud faults want a fast check; quiet faults want a slow, patient one. A real integrity layer needs both.

03 One green run is a demo, not a measurement

Every milestone up to here ended with a sentence like caught the fault at onset with zero false alarms. That reads well and it is true, and it proves almost nothing, because a detector that fires on everything would pass that test too. A fault monitor only means something as three numbers, measured together: how often it catches real faults, how rarely it cries wolf, and how fast. Any one alone is gameable.

Because every fault here is injected, the ground truth is always known, so I could run the monitors across many random seeds and a sweep of fault sizes and get real measurements rather than estimates. The picture is a curve, not a point. False alarms are genuinely rare, three stray flags in fifty-four thousand readings on clean drives, about 0.006%, which is where a 99.9 percent threshold should land and what keeps the detection numbers honest. A GPS jump is invisible below the sensor noise, then climbs a sharp S-curve to caught every time, within a single sample, once it clears a few metres. The absorbed bias is caught by the slower drift monitor, with a higher floor and seconds to confirm rather than milliseconds. Two monitors, two honestly different operating envelopes.

04 Open versus closed is a seam, not a wall

The last step added no new detection idea. It did the two unglamorous things that turn four increments into something a stranger can trust and a business could build on. First, make every result reproducible from a clean checkout: one command reruns the scenarios and prints the metrics, another rebuilds every figure from the same runs, so nothing is a stored number sitting next to code that no longer produces it. If the write-up and the harness ever disagree, the harness wins.

$ python -m av_integrity report
integrity core: reference SafetyMonitor
M1  GPS jump (8 m): detected=True  RMSE 5.6 m -> 0.4 m with gating
M4  false-alarm rate (nominal): 0.006% / check
    GPS jump 5.0 m: detection 1.00, time-to-detect 0.00 s
    IMU bias 1.0:  detection 1.00, time-to-detect 3.95 s

Second, draw the line between the open part and the private part as one real, testable seam instead of a wall. Everything public is an honest, un-tuned reference. A tuned detector, better thresholds, a learned residual model, is worth keeping private. The temptation is a wall, a separate fork and a pile of contact-us, but walls rot: the two sides drift apart and you maintain two systems that no longer agree. The alternative is a single function that decides which monitor runs. With nothing installed it returns the public reference; point an environment variable at a private module and the same function loads that instead, behind the identical interface, graded by the exact same stress test. The private code never enters the public repository, and an improvement is an honest like-for-like comparison. That is the systems-integration idea the whole project has been about, turned on itself: the value is in owning the seam.