Entropy coders

This page is now superseded by new open-source library Finite State Entropy, which also includes a new version of huff0. Available at https://github.com/Cyan4973/FiniteStateEntropy


Entropy coders reduce a set of symbols according to their appearance probabilities. 
They are typically used as stage 2 of many compression programs, including zip, rar or 7zip. 

1) Range0 v0.7
New : small encoding & decoding speed improvements in extreme situations (badly or highly compressible data) 
This is a semi-static Range Coder implementation. Data is divided into blocks of 128KB. Each compressed block starts with a frequency count header.
Compression speed : 145-188 MB/s
Decoding speed : 77-148 MB/s

2) Huff0 v0.9
New : Improved multi-threading code (benchmark mode only)
This is a semi-static huffman coder implementation. Data is divided into blocks of 16KB. Each compressed block starts with a huffman tree header.
Benchmark mode allows testing multi-threaded compression and decompression.
Single Thread Compression speed : 190-285 MB/s
Single Thread Decoding speed : 184-280 MB/s

3) HuffX v0.1
New : Initial release, merge huff0 and range0, multi-threading available in benchmark mode
This is a mixed huffman and range coder, trying to provide the best of both worlds. Data is divided into blocks of 16KB.
Compression speed : 179-284 MB/s
Decoding speed : 141-271 MB/s

4) FSE : Finite State Entropy

Based on a completely new theory of entropy coding.
Delivered as an open source library at Github
(no benchmark number here yet).

Detailed performance assessment (single threaded):
Benchmark platform : Core 2 Duo E8400 (3GHz), Window Seven 32-bits


You can post comments on the dedicated forum.