Why does yesterday's test result finish the attempt I just started?

DominicBailey · 24 Sept 2025, 04:39 UTC

Closed
DO
DominicBailey
In my offline UR5e inspection model I can hold an old completion, start another bracket attempt with the same job label, then release the old message and watch the new attempt finish; random delays sometimes expose it and sometimes don't, so how do I make a test that fails every time until this is fixed?

21 replies

HA
HassanBrown0878
Replying to DominicBailey

Can your test choose when each message is delivered?

23 points
DO
DominicBailey
Replying to HassanBrown0878

At the moment a background thread sleeps and sends it, so the machine running the test gets a vote in the order

8 points
HA
HassanBrown0878
Replying to DominicBailey

Replace that timing guess with an event queue the test advances explicitly.

16 points
DA
DanielBrown0913
Replying to DominicBailey

Write down the bad sequence in the test itself. Old attempt waits, new attempt starts, old completion arrives. Then check the new attempt is still unfinished, not just that the test printed the right warning.

15 points
HA
HassanBrown0878
Replying to DominicBailey

Does the completion carry an attempt identity?

15 points
DO
DominicBailey
Replying to HassanBrown0878

Only the job label in my model, which is exactly what gets reused; the software owner is reviewing how the real interface can carry attempt identity

17 points
HA
HassanBrown0878
Replying to DominicBailey

Keep the reused label in the test. It is part of the failure case.

22 points
LI
LiamArcher0403
Replying to HassanBrown0878

Can you reproduce it now without any sleeps?

6 points
DO
DominicBailey
Replying to LiamArcher0403

Yes, Liam, queued delivery fails the old handler every time I've run it, and Daniel's unfinished-state assertion catches the wrong completion directly

20 points
EL
ElliotBrown0944
Replying to DominicBailey

Give the test separate identities for separate attempts. Deliver duplicates as well as the delayed event, and check that a completed attempt cannot be credited twice. Keep the real interface work separate from this model change.

18 points
DO
DominicBailey
Replying to ElliotBrown0944

Added unique attempt identities in the model and the delayed case now passes, but a duplicate still adds another entry to the completion list

2 points
DA
DanielBrown0913
Replying to DominicBailey

There's your next assertion. One accepted completion entry per attempt. I'd also want the ignored duplicate visible somewhere for diagnosis, because otherwise support will wonder why the raw message count differs.

15 points
DO
DominicBailey
Replying to DanielBrown0913

The model now retains the duplicate as an event without crediting another completion; or rather, that's what the new handler does, and the old handler fails both checks

19 points
HA
HassanBrown0878
Replying to DominicBailey

What happens if the application restarts before the duplicate arrives?

23 points
LI
LiamArcher0403
Replying to HassanBrown0878

And does the test restore saved state or start empty?

14 points
EL
ElliotBrown0944
Replying to LiamArcher0403

Restore the state the application design actually persists. Include a crash after receipt but before saving the completion. Recovery has to handle that uncertainty rather than assuming the event never occurred.

23 points
DO
DominicBailey
Replying to ElliotBrown0944

Liam, it started empty, which was no help; I've added saved-state restoration and a crash point before the save, Elliot, with the unresolved case left for reconciliation

7 points
EL
ElliotBrown0944
Replying to DominicBailey

Also deliver the new attempt's valid completion before the old one. The old event must not overwrite the newer result or reopen a completed attempt.

19 points
DA
DanielBrown0913
Replying to DominicBailey

Who will read the unresolved case screen? Your software owner should try the wording with them. 'Unknown' without the attempt or reason sends people straight back to asking you.

22 points
DO
DominicBailey
Replying to DanielBrown0913

Quality's shift lead will review that screen, Daniel; model tests now cover both completion orders, duplicates and the save gap, but the live interface proposal is still with its owners

11 points

Discussion closed

This discussion is closed to new replies after six months without activity. Last activity: .