The most dangerous failure in an autonomous system is not a wrong decision. It is a confident decision, built on a sensor that is quietly lying. A GPS fix that jumped a few metres. An IMU that has started to drift. A wheel that is slipping while the encoder keeps counting. The state estimate still looks reasonable, so the system trusts it and acts. The gap between what the system believes and what is true is where autonomy gets dangerous. That gap is what I am building an answer to, in the open.
The dangerous failure isn't a bad decision.
It's a confident one, built on a sensor that's quietly wrong.
I have spent fifteen years where hardware meets software, and this is the failure that has stayed with me the most. Not the dramatic crash, but the quiet one: a system acting, confidently, on an input that had already gone wrong, while every dashboard still looked green. I have wanted to sit with the hard version of that problem for a long time, and building it in the open is how I am finally doing it.
01 The problem, and why it is a systems problem
Every autonomous system, a vehicle, a robot, a drone, fuses a handful of noisy sensors into one estimate of its own state, and then acts on that estimate. When a sensor degrades or lies, two questions decide whether the system is safe: does it notice, and does it fail safe when it does. Neither question is answered by a single clever algorithm. The answer lives across the sensors, the estimator, the fault logic, the safe-state behaviour, and the interface contracts between all of them. That is a systems-integration problem, and it is the work I care about and do best.
02 Why in the open, and why simulation-first
Building in the open is not about showing a finished thing. The value is the visible reasoning: the constraints, the trade-offs, the failures, and the interface decisions at each step. That is what I want to put on the table and be judged on.
Simulation-first, because the hard part of this problem is the software, the estimation, and the verification, not a machine shop. Simulation lets me iterate in hours and inject faults on demand. It also lets me measure honestly: because I inject the faults, I know the ground truth, so "did it catch the fault" has a real answer instead of a marketing number. The honest boundary: a simulation proves the reasoning and the method, not real-world performance. I will say so, and I will state where the model's fidelity ends.
03 The shape of the build
It grew one increment at a time. Each increment was a working piece, a public repository, and a build log that explains the decisions:
- A sensor-fusion core. Simulated sensors feed one estimate of the vehicle's state.
- A digital twin. A model that predicts what the sensors should read, so divergence between predicted and observed becomes a fault signal.
- Fault detection and isolation. Not just that something is wrong, but which sensor is lying and how badly.
- A safe-state layer. What the system does once it can no longer trust part of its own input.
- An honest evaluation harness. Detection rate, false-alarm rate, and time-to-detect across many injected-fault scenarios, including the ones it fails.
Across five milestones the flagship shipped a working piece and a build log at each step, explaining the architecture decisions, the interface contracts, and the trade-offs. The interface contracts are not a formality. They are where an integrator's discipline shows, and where most real systems quietly fail.
04 The build so far
The code and every build log are public. The repository is github.com/andrewmichelis/av-integrity (open, Apache-2.0), and each milestone has its own write-up. The five-milestone arc is now complete; below is the full set.
- M0 · the fusion core: Building a Sensor-Fusion Testbed You Can Trust. Fuse GPS, IMU, and wheel speed into one estimate, and prove it tracks the truth.
- M1 · the first integrity check: How I Made a Vehicle Notice Its GPS Was Lying. Score each reading by how surprising it is, and gate out a spoofed GPS at the first bad sample.
- M2–M5 · the integrity layer, end to end: Teaching a Vehicle to Distrust Its Own Sensors. Isolate a fault and fail safe on an honest criterion, catch the quiet faults a fast check misses with a model-based twin, turn single runs into detection and false-alarm rates, and ship it all as a reproducible open harness with the tuned core behind one seam.
05 What you get by following along
These build logs are not a project diary. Each one is written to be useful even if you never touch my code: the reasoning, the failure modes, and the transferable lessons. Alongside the build, I write standalone pieces on the ideas it touches, sensor fusion, digital twins, fault detection, and on lessons from fifteen years of shipping cyber-physical systems where hardware meets software.
The plan is a system that knows when it can't trust its own sensors and fails safe instead of failing, built in the open one honest increment at a time. The first build log covers the sensor-fusion core and how I check the estimate against ground truth. If you work on systems that have to trust their own sensors, this is worth following.