My status screen holds its breath whenever the log rotates

LinAdams0096 · 8 Dec 2024, 00:27 UTC

Closed
LI
LinAdams0096
Can I untangle a logging problem without pestering the controls team? My FR10 status viewer freezes when a new log file starts, or rather the numbers freeze while the window still moves; it's read-only and the actual cell controls are separate.

10 replies

LU
LucaBarnes0526
Replying to LinAdams0096

Check the timestamps first, because my display once showed old readings beautifully while everyone blamed the network.

11 points
LI
LinAdams0096
Replying to LucaBarnes0526

The displayed timestamp stops as well, then jumps ahead with no readings shown for the gap.

7 points
LU
LucaBarnes0526
Replying to LinAdams0096

That leaves two possibilities in my head: missing samples or samples waiting somewhere, can you see whether the file has them?

8 points
SA
SaraCarter0983
Replying to LinAdams0096

Time the read and file write separately. The screen alone can't tell you which waited.

16 points
LU
LucaBarnes0526
Replying to SaraCarter0983

Did those timings show anything? I ask because rotation was when our report tool copied the previous file to a slow share.

11 points
LI
LinAdams0096
Replying to LucaBarnes0526

Found a copy-to-archive call in the rotation handler, it sits on the same loop as the reads and the delay matches the freeze... I hadn't realised that archive folder was remote.

12 points
LU
LucaBarnes0526
Replying to LinAdams0096

That's promising, ours recovered once archiving stopped holding up acquisition, though we still had to show when the archive was behind.

13 points
DA
DanLoop
Replying to LinAdams0096

Does your proposed background writer have a bounded queue and an explicit overflow policy? Moving the wait can otherwise turn a visible stall into silent memory growth, which is a pain to hand over.

11 points
LI
LinAdams0096
Replying to DanLoop

I've moved archival copies off the sampling loop and capped the pending work, failed copies stay local with an alert; a deliberately unavailable share no longer freezes the viewer, and the sampler timings stay steady.

17 points
LU
LucaBarnes0526
Replying to LinAdams0096

Good, the unavailable-share test is the bit I wanted to hear about, our first fix worked only while everything was behaving.

18 points

Discussion closed

This discussion is closed to new replies after six months without activity. Last activity: .