threads: run the main loop once when a mgmt thread is killed during the initial pause

Every thread is created paused (THV_PAUSE) and the main thread unpauses
them all before the main loop. A thread that has not yet been scheduled
spins in TmThreadsWaitForUnpause() until it sees the unpause. Under
heavy thread oversubscription (many short offline runs in parallel) a
management thread can be descheduled across the whole run: by the time
it runs again the shutdown has set THV_KILL while it is still in the
initial pause. TmThreadsWaitForUnpause() then returned false, so the
thread skipped its main loop entirely and never did its final shutdown
work - for the stats thread that means the last stats event is never
written to eve.json. This intermittently fails suricata-verify tests
that assert on the final stats.

Mgmt threads (tv->type == TVT_MGMT) now return so the main loop still
runs once: it tests THV_KILL and exits after the pass, but the thread
gets the single pass it needs for final work (e.g. the last stats
output). THV_PAUSED is cleared on that path. RX, worker and other
non-mgmt threads keep the previous behaviour: their loops do not test
THV_KILL at the top - the RX loop exits only on the PktAcqLoop return
value and suricata_ctl_flags, and a pcap-file pass would dispatch
packets into a pipeline that is being torn down.

The log-maintenance thread breaks before its first sleep slice when it
is already being killed: on that pass no rotation or flush can happen
(their wait counts need a full second of slices), so the fixed 500 ms
sleep would be pure shutdown latency - the slice exists to react to
shutdown quickly, and a first wake into a KILL must not wait out a full
slice. A kill arriving after the first slice keeps the normal
behaviour, including the final rotation check.

Ticket: #9168.
This commit is contained in:
Victor Julien
2026-10-08 13:08:20 +00:00
parent b8275da9d0
commit 1ff8de029a
3 changed files with 28 additions and 4 deletions
+11
View File
@@ -81,8 +81,17 @@ static void *LogMaintenanceThread(void *arg)
int flush_counter = 0;
uint64_t rotation_check_count = 0;
uint64_t worker_flush_count = 0;
bool completed_slice = false;
bool run = TmThreadsWaitForUnpause(tv_local);
while (run) {
/* killed before completing the first slice: skip it - the fixed
* sleep would only delay shutdown, no rotation or flush can
* happen on this pass. A kill arriving later keeps the normal
* behaviour: the current slice still runs its rotation/flush. */
if (!completed_slice && TmThreadsCheckFlag(tv_local, THV_KILL)) {
break;
}
SleepMsec(maintenance_sleep_time);
/* Check rotation every 1 second */
@@ -99,6 +108,8 @@ static void *LogMaintenanceThread(void *arg)
flush_counter = 0;
}
completed_slice = true;
if (TmThreadsCheckFlag(tv_local, THV_KILL)) {
break;
}
+12 -3
View File
@@ -359,7 +359,14 @@ error:
}
/**
* Also returns if the kill flag is set.
* Wait for the thread to be unpaused.
*
* A thread killed while still in the initial pause skips its main loop,
* except mgmt threads (TVT_MGMT): they run one pass to do final work
* (e.g. the last stats output) - their loops test THV_KILL and exit
* after the pass. RX/worker loops have no top KILL test and would do
* real work during teardown (a pcap-file pass dispatches into a pipeline
* being torn down), so they keep skipping.
*/
bool TmThreadsWaitForUnpause(ThreadVars *tv)
{
@@ -369,8 +376,10 @@ bool TmThreadsWaitForUnpause(ThreadVars *tv)
while (TmThreadsCheckFlag(tv, THV_PAUSE)) {
SleepUsec(100);
if (TmThreadsCheckFlag(tv, THV_KILL))
return false;
if (TmThreadsCheckFlag(tv, THV_KILL)) {
TmThreadsUnsetFlag(tv, THV_PAUSED);
return tv->type == TVT_MGMT;
}
}
TmThreadsUnsetFlag(tv, THV_PAUSED);
+5 -1
View File
@@ -298,8 +298,12 @@ bool SCTmThreadsSlotPacketLoopFinish(ThreadVars *tv);
* Check if a thread should wait to be unpaused and wait if so, or
* until the thread kill flag is set.
*
* \returns true if the thread was unpaused, false if killed.
* \returns true normally, and for mgmt threads killed while still in
* the initial pause (their main loop runs one final pass). Other thread
* types return false in that case and skip the main loop, as before.
*/
bool TmThreadsWaitForUnpause(ThreadVars *tv);
void TmThreadsRegisterTests(void);
#endif /* SURICATA_TM_THREADS_H */