Rust compression crates gain speed through better crate splitting
Splitting lz4rip into encode, decode, and core crates cut decoder regressions and lifted Silesia decompression 4% to 17%.
lz4rip's move to three crates is doing more than cleaning up the tree. The Rust LZ4 block and frame codec, derived from lz4_flex and built around aggressive skip acceleration and generational hash tables, now lives as lz4rip-encode, lz4rip-decode, and lz4rip-core so LLVM no longer has to optimize dead encoder code alongside the decoder hot path. That layout change is the kind of thing Rust systems programmers care about because crate boundaries can change generated code, not just source organization.
The payoff showed up in the latest release, which pushed lz4rip from v0.1.0 to v0.8.5. Decompression improved by 4% to 17% across the Silesia benchmark suite, and on 8 KB inputs the gap versus C LZ4 narrowed from 10% to 33% down to 2% to 21%. That is a serious shift in a corner of the ecosystem where the C implementation still sets a high bar, with the LZ4 project README describing throughput above 500 MB/s per core and scalable multi-core performance.
The same update package also included zrip 0.3.4, a pure Rust zstd codec aimed at transfer pipelines that need standard zstd frames at high speed. zrip supports Fast and DFast strategies, compression levels from -7 through 4, COVER and FastCOVER dictionary training, streaming encode and decode, and no_std support. In other words, this was not a single micro-optimization but a broader compression-tooling push from the same author.
The engineering details around lz4rip make the performance story easier to trust. The block codec has zero dependencies, the frame side uses one optional dependency, twox-hash, for xxHash32 checksums, and the project is backed by about 100 tests and 10 cargo-fuzz targets. The maintainer also marked the hot paths #[forbid(unsafe_code)] and kept the hash table for inputs under 64 KB at 4K×u16, or 8 KB, small enough to fit in L1d cache and still leave room for application data. That is the practical lesson here: in Rust compression code, crate splitting and micro-architecture choices can move throughput as much as a new loop body, and lz4rip's latest numbers show both levers working together.
This article was produced by Prism’s automated news system from verified source data, official records, and press releases, then run through automated quality and moderation checks before publishing. The system is built and supervised by the people who set the standards it runs under. Read our full AI policy.
Did this article answer your question?
