Skip to content

feat: better compression bandwidth - #198

Open
mterenzi wants to merge 18 commits into
farsightsec:nextfrom
mterenzi:realtime-compression
Open

feat: better compression bandwidth#198
mterenzi wants to merge 18 commits into
farsightsec:nextfrom
mterenzi:realtime-compression

Conversation

@mterenzi

Copy link
Copy Markdown

Change

nmsgtool -z compresses on the reader thread. While zlib runs, nothing drains the
socket. A single reader tops out before the kernel receive queue overflows and payloads are lost.
--zasync moves compression onto worker threads
and puts the writes back in order behind one committer thread.

Using it

--zasync n      up to n compressor threads (-1 chooses, 0 off — the default)
--zcull secs    give up a compressor idle this long (300; 0 never)
--zmin n        never cull below n compressors (1; 0 lets the pool empty)

Replace zlib with zlib-ng

default:              libz.so.1 => /usr/lib/x86_64-linux-gnu/libz.so.1
LD_LIBRARY_PATH=<ng>: libz.so.1 => .../zlib-ng-233/lib/libz.so.1

Related Confluence Page

Rafael Vanoni and others added 18 commits March 3, 2026 13:24
…ssage

it is compared using host byte order but previously was printed as is
(network byte order)
it had a parser error with opening and ending tag mismatch for replaceable
(roff was seen in rendered manpage)
Properly propagate the `nmsg_res_again` return of _input_nmsg_unpack_container
through _input_process_buffer_into_container.
@mterenzi mterenzi changed the title Better compression bandwidth feat: better compression bandwidth Aug 21, 2026
@kadealicious
kadealicious force-pushed the next branch 2 times, most recently from c2b10f5 to 8bf7036 Compare September 3, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants