Operation continuity without a persistent identifier

Diagnosing multi-stage operations meant reconstructing the flow of a single operation through successive systems, from its start to its confirmation.

The working assumption was that one identifier linked all the messages of an operation. Every conclusion about the stage at which a problem arose depended on that assumption.

I based the analysis on four principles:

• Testing the identification assumption — before drawing conclusions about an operation’s flow, checking whether the assumed identifier really links all the messages of one operation.
• Reading fields in the context of the stage — the same information can change form between states or be replaced by other data, so identification considers what fields mean at a specific processing stage, not only their values.
• Correlation based on state transitions — consecutive messages are linked using transport- and application-layer information, the context of the operation’s source, its type and the dependencies between stages. Neither timing, session nor a single identifying value is standalone proof that messages belong to the same operation.
• Verification on a larger dataset — rules identified on an initial sample are checked on a much larger dataset, to rule out that they reflect the specifics of a small sample.

I built a tool that analyses binary messages at byte level, with the interpretation of their fields checked against the available specification.

The analysis showed that the assumed identifier changed in every complete case. Only a small share of operations allowed simple start-to-finish correlation, and the set of fields available in messages changed between stages.

On that basis I designed and deployed a mechanism that reconstructs the continuity of an operation from transitions between states. Reporting shows on what basis messages were linked. The mechanism makes it possible to analyse the time of individual processing stages and to recognise delays and hangs, including in operations that no single identifier links.

From around 2,000 cases I identified how the identifiers changed and developed the correlation rules; I then verified them on more than 170,000 cases. The proportions of the observed phenomena remained stable, confirming that the dependencies did not reflect the specifics of the initial sample.

Value: diagnosing multi-stage operations rests on a verified communication model rather than an unconfirmed assumption about the identifier.