Results 1 to 4 of 4

Thread: Newpack compression algorithm from the author of Wavpack

  1. #1
    Member
    Join Date
    Feb 2018
    Location
    Singapore
    Posts
    6
    Thanks
    1
    Thanked 2 Times in 2 Posts

    Newpack compression algorithm from the author of Wavpack

    This is an experimental general-purpose lossless data compressor, the genesis of which was the "fast" DSD compression mode developed for WavPack. I discovered early on that several regular compressors (e.g., bzip2) do a surprisingly decent job on DSD (1-bit PCM) audio files. Conversely, after I developed the "fast" DSD mode of WavPack, I discovered that it could often do a decent job on other types of data (like text).

    This implementation scans each buffer of data to be compressed and creates a probability model for each byte which is based on the hash of some number of previous bytes in the stream. This model consists of a bitmask representing which hashes are actually present in the file and a symbol probability table for each hash that appeared. These two model components are then compressed recursively and sent to the output file, and then the actual data is encoded using the model with a range coder and appended to the end of the encoded block.
    Source: https://github.com/dbry/newpack

  2. Thanks:

    Shelwien (13th December 2020)

  3. #2
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    4,136
    Thanks
    320
    Thanked 1,397 Times in 802 Posts
    Need to compare it to a coder which has an unaligned bit model, eg. https://encode.su/threads/3323-nanoz...ll=1#post63524

  4. #3
    Member
    Join Date
    Oct 2009
    Location
    usa
    Posts
    62
    Thanks
    1
    Thanked 9 Times in 6 Posts
    Looks very interesting indeed. Was anyone able to build a binary for Win10 x64? I tried but did not succeed.

  5. #4
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    4,136
    Thanks
    320
    Thanked 1,397 Times in 802 Posts
    Builds with mingw without any problems.
    Also https://github.com/dbry/newpack/releases
    Attached Files Attached Files

Similar Threads

  1. My new compression algorithm
    By tefara in forum Random Compression
    Replies: 55
    Last Post: 12th June 2019, 21:45
  2. Anyone know which compression algorithm does this?
    By hjazz in forum Data Compression
    Replies: 8
    Last Post: 24th March 2014, 06:49
  3. New layer 0 - compression algorithm
    By abocut in forum Data Compression
    Replies: 5
    Last Post: 28th May 2010, 02:32
  4. The best algorithm for high compression
    By Wladmir in forum Data Compression
    Replies: 8
    Last Post: 18th April 2010, 15:54
  5. Looking for Thor's author Oscar Garcia
    By ovatsus in forum Data Compression
    Replies: 7
    Last Post: 8th May 2009, 13:02

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •