Files
Max 1900670c18 perf(lists): compact wildcard caches and serialize group refreshes (#2289)
The wildcard cache stored each rule as trie nodes, about 86 bytes per
rule, so HaGeZi TIF alone held 150 MiB of heap. Rules are now grouped
by TLD and, within a TLD, sorted and concatenated into one string per
base length, the layout the exact cache already uses. A lookup splits
off the TLD and checks each suffix of the remainder from the broadest
parent down. A hash filter with two bits per rule in one 64-bit word
rejects most suffixes before the binary search. The broadest stored
parent still wins, a *.tld rule still covers the whole TLD, and the
reported rule string is unchanged. TIF drops to 24.5 MiB.

A refresh built every group in parallel, so a process with four large
lists peaked near 880 MiB RSS. Each ListCache now rebuilds one group at
a time and admits one refresh call at a time; downloads inside a group
keep blocking.loading.concurrency. A caller whose context ends while
waiting for its turn returns ctx.Err() instead of starting another
refresh afterwards. The old group stays active until its replacement
is ready. Peak RSS through a refresh drops to about 260 MiB and the
refresh takes about a third longer.

The exact, regex and wildcard caches build the logrus entry for a hit
only when debug logging is enabled, which removes the per-hit
allocations when it is off.
2026-10-09 22:15:05 +02:00
..