In Cyber-Physical Systems, Latency Is a Correctness Property

A control loop I was bringing up behaved perfectly in simulation and shook itself apart on the bench. Same gains, same model, same code. The difference was a few milliseconds. On hardware the feedback path picked up sensor read time, a bus hop, and a scheduler delay that the simulation had treated as instantaneous. The loop was not wrong about the world. It was wrong about when. It was correcting the vehicle's state as that state had been a few milliseconds earlier, and by the time each correction landed, the real state had already moved on. Nothing in the code was buggy. The system was unstable because a number arrived late, and late, in a loop that acts on what it reads, is a different thing from slow.

01 Eventually correct is not correct

In ordinary software, latency is a performance property. The answer is the same whether it takes one millisecond or one hundred, you just wait longer for it, so you chase the tail of the distribution to keep users happy and move on. A cyber-physical system quietly breaks that assumption, because the answer is about a physical quantity that keeps changing whether or not your software is ready for it.

A position, a velocity, a temperature measured at time t is a fact about the world at t. Deliver it at t + Δ and act on it, and you have applied a correct measurement to the wrong instant. The value did not degrade in transit. Its meaning did. "Eventually correct" is a promise about the number. It says nothing about whether the number still matches the world at the moment you use it, and in a loop that closes on a physical system, that match is the entire game. This is why latency here is not a service-level nicety you tune after the logic is right. It is part of whether the logic is right.

02 Where it actually bites

Three places, in the order they have cost me time.

Fusion timestamp alignment. When you fuse several sensors, each reading has to be placed at the time it was actually measured, not the time it reached your process. A GPS fix at 5 Hz arriving over a link that adds tens of milliseconds, and an IMU at 200 Hz that is nearly immediate, are describing the same trajectory at very different delays. Fuse them by arrival order and you fold each sensor's transport delay straight into the state estimate, where it comes out looking exactly like sensor noise: a bias you will spend days trying to calibrate out of the wrong component. In the open sensor-fusion monitor I have been building in public, the single most consequential line is not the filter math, it is stamping every measurement by its true measurement time and letting the estimator account for the known delay. Get the clock right and a mediocre filter is fine. Get it wrong and no filter saves you.

Control-loop deadlines. A feedback loop has a stability budget, and latency spends it. Delay in the loop eats phase margin directly, and past a threshold the loop that was calm in simulation oscillates on hardware, which is the failure I opened with. Jitter is usually worse than the latency itself. A constant delay you can measure and compensate for. A delay that varies from cycle to cycle you cannot, and a loop closed at a jittery average is really closed at its worst case, whether or not you designed for that.

The sensor-to-actuator budget. The number that finally matters is end to end: from the instant the world changes to the instant the actuator responds. Every hop in between spends part of a fixed deadline set by physics and safety. Sensor integration time, bus transport, fusion, the decision, actuation. Shaving the average off one stage while another quietly blows its tail does nothing for the deadline that counts. You do not have a latency figure until you have the worst-case sum along the whole path.

03 Treat latency as a spec, not a hope

If latency is a correctness property, it belongs in the contract for every data path, the same way a range or a unit does. Concretely, for each path that feeds a decision:

• give it a deadline and a staleness bound, and write them down next to the data, not in someone's memory;
• stamp data by measurement time and carry that timestamp end to end, so every consumer can reason about how old the value is;
• when data is late, decide on purpose: reject it, extrapolate with widened uncertainty, or hold last-known behind a flag, but never silently use a stale value as if it were fresh;
• specify and measure the worst case, not the average, because the tail is where the system actually fails;
• budget end to end and hold each stage to its slice.

The reject-late guard is only a few lines, and it turns a silent failure into a visible one:

def use(measurement, now, max_age):
    age = now - measurement.stamp          # measurement time, not arrival time
    if age > max_age:
        return reject(measurement, "stale") # a late value is a wrong value
    return fuse(measurement, age)          # let the estimator account for the known delay

There is nothing sophisticated here. The point is that "too late to be correct" becomes an explicit, testable condition instead of a number that quietly poisons the estimate three layers downstream.

04 What it changes

Once latency is a correctness property rather than a performance metric, a familiar class of bug stops being mysterious. "Works in simulation, shakes on hardware" is usually not a modelling error. It is a timing assumption the simulation made for free and the hardware charged for. The fix is rarely a faster machine. It is an honest budget, timestamps that mean what they say, and a deliberate answer to the question of what happens when a value shows up late.

Most of the hard integration bugs I have chased were not wrong values. They were right values, used at the wrong time. That is a smaller, sharper problem, and unlike "make it faster," it is one you can specify, measure, and catch before the bench does.