Page 2 of 2 FirstFirst 12
Results 31 to 33 of 33

Thread: libdeflate - a new optimized DEFLATE implementation

  1. #31
    Member
    Join Date
    Dec 2011
    Location
    Cambridge, UK
    Posts
    462
    Thanks
    147
    Thanked 158 Times in 106 Posts
    For what it's worth, we added this to the Samtools/Htslib packages for manipulating DNA sequencing files: https://github.com/samtools/htslib/pull/581

    The speed gains were very much worth it. We also fuzz tested with American Fuzzy Lop and found no issues. A big thumbs up from me.

    The only thing I can see lacking, which sadly I didn't get around to fixing, is a proper "make install" target.

  2. Thanks:

    Gonzalo (29th May 2018)

  3. #32
    Member
    Join Date
    Dec 2011
    Location
    Cambridge, UK
    Posts
    462
    Thanks
    147
    Thanked 158 Times in 106 Posts
    Quote Originally Posted by a902cd23 View Post
    Test of libdeflate-1.0-windows-x86_64-bin
    Code:
    1.787.785.811 mkv
    1.786.957.948 mkv.7z.22.zstd - 137 sec
    1.785.465.763 mkv.winrar.best.zip - 27 sec
    1.785.419.731 mkv.7z.deflate.mx.zip - 11.5 min
    1.785.400.203 mkv.1.gz - 37 sec
    1.785.221.871 mkv.7z.deflate64.mx.zip - 13 min
    1.784.581.959 mkv.12.gz - 80 sec
    1.784.439.182 mkv.m8.zcm093 - 9 min
    1.784.355.455 mkv.ect.10007.zip - 105 min
    1.784.044.948 mkv.b2047.bcm130 - 10 min
    It's an interesting data point, but I don't think comparing tools on uncompressible data is the normal use case.

    The "unstable" version of the Squash benchmark contains libdeflate, also against various other Deflate RFC compliant implementations: https://quixdb.github.io/squash-benchmark/unstable/

  4. #33
    Member
    Join Date
    Nov 2014
    Location
    Earth
    Posts
    38
    Thanks
    0
    Thanked 77 Times in 19 Posts
    Quote Originally Posted by JamesB View Post
    For what it's worth, we added this to the Samtools/Htslib packages for manipulating DNA sequencing files: https://github.com/samtools/htslib/pull/581

    The speed gains were very much worth it. We also fuzz tested with American Fuzzy Lop and found no issues. A big thumbs up from me.

    The only thing I can see lacking, which sadly I didn't get around to fixing, is a proper "make install" target.
    I've added "make install" and "make uninstall" targets.

  5. Thanks:

    JamesB (15th June 2018)

Page 2 of 2 FirstFirst 12

Similar Threads

  1. LZF - Optimized LZF compressor
    By encode in forum Data Compression
    Replies: 39
    Last Post: 28th March 2019, 20:49
  2. Intel's fast OSS deflate implementation
    By Bulat Ziganshin in forum Data Compression
    Replies: 16
    Last Post: 23rd May 2016, 18:13
  3. Optimized LZSS compressor
    By encode in forum Data Compression
    Replies: 11
    Last Post: 13th February 2014, 23:51
  4. I'm looking for the best free implementation of deflate
    By caveman in forum Data Compression
    Replies: 2
    Last Post: 22nd November 2010, 09:27
  5. 7zip >> Sfx optimized - 23,7 kb
    By Yuri Grille. in forum Data Compression
    Replies: 22
    Last Post: 12th April 2009, 22:33

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •