tag:blogger.com,1999:blog-834134852788085492.post1641308962659345702..comments2024-03-02T07:59:30.808+01:00Comments on RealTime Data Compression: Huffman revisited part 5 : combining multi-streams with multi-symbolsCyanhttp://www.blogger.com/profile/02905407922640810117noreply@blogger.comBlogger1125tag:blogger.com,1999:blog-834134852788085492.post-81968589564342873622015-10-16T19:00:18.170+02:002015-10-16T19:00:18.170+02:00As always very nice insights, it is pleasure to re...As always very nice insights, it is pleasure to read your 'knowhows from the kitchen'.<br /><br />>With 4 separate segments being decoded in parallel, the design looks a lot like classical multi-threading, at micro-op level. Which would be a fair enough description.<br /><br />Exactly so! Feeding the CPU this way it won't be hungry.<br /><br />>... but that's not always the case (x86 32-bits mode notably).<br /><br />IMHO, Zstd should have one purely 64bit variant, wanting to be 32bit as well will inevitably hurt the beauty, I mean, inhere speed should trump versatility.<br /><br />>You may have noticed that measurements were produced on block sizes of 32 KB...<br /><br />If I am not wrong, with compression ratio of e.g. 2.7:1 the incoming (if text) is ~82KB, if so why not going with 256KB - one average ebook/novel is 600KB, the context of the text fits inthere.<br /><br />Two things interest me most:<br />- your "rigid format" is not to be thrown away, I would like to see such one calibrated for English texts, as far as I understand your latest 4 symbols 128KB fits, no?! My experience with 'rigid' i.e. dedicated/stripped etudes is that at the end they pay off not having junky code to slow them down with cycles and branching;<br />- I very much want to compile your latest Zstd with Intel v15.0 but it is hard for me to use make and what to compile, if you can provide a compile line I will be very glad to include this executable in my November/December decompression speed showdown. Or maybe I have to wait you to finalize it! This versatility surely could map onto a plethora of command line options.Anonymousnoreply@blogger.com