Results 1 to 19 of 19

Thread: Preprocessors and filters and Nanozip??

  1. #1
    Member kampaster's Avatar
    Join Date
    Apr 2010
    Location
    ->
    Posts
    59
    Thanks
    13
    Thanked 7 Times in 7 Posts

    Smile Preprocessors and filters and Nanozip??

    What preprocessors or filters uses Nanozip??
    Most of all me the audio compressors (Wav) interests.
    In advance thanks!

  2. #2
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,977
    Thanks
    296
    Thanked 1,304 Times in 740 Posts
    Well, Sami is fairly secretive, also he doesn't post at this forum, so your best bet is trying to ask him by email.
    But certainly there're some good english text filters (and likely word submodel for CM mode),
    executable filters (seems like improved version of bcj2) and record filters, and others.
    As to audio though, its not a preprocessor, but an audio model (likely similar to optimfrog),
    linear prediction + CM i think.

  3. #3
    Member Fu Siyuan's Avatar
    Join Date
    Apr 2009
    Location
    Mountain View, CA, US
    Posts
    176
    Thanks
    10
    Thanked 17 Times in 2 Posts
    Well, it's very interesting that Sami don't like others call it "filters". However, it is obviously what we called filters; It has a detector --- maybe by blocks. It can detect out x86, some multimedia data(some waves or bitmaps) , or recognized it as records (try Comp.Ratings Audio1 with tar), or english texts. Also nanozip firstly detect file types by header. Then it applys x86(I'm not sure if it's better than bcj2) to executable data\DICT to english\Special coder for records and special coder for audio and bitmaps. If you are only concentrate on wav compression, I think the nz isn't advanced much than that in FreeArc, which is open source.

    Hope this won't make Sami flamed.

  4. #4
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,977
    Thanks
    296
    Thanked 1,304 Times in 740 Posts
    > Well, it's very interesting that Sami don't like others call it "filters".

    Filters (preprocessors) and data-specific models are different, and while, for example, wav handler in rar is a preprocessor,
    the one in nz isn't.

    > Also nanozip firstly detect file types by header.

    Maybe, but I'm sure that there're also detectors by data type.

    > Then it applys x86(I'm not sure if it's better than bcj2)

    Well, I tested it before saying that and it wound up somewhat better.
    Anyway, its certainly better than E8 and worse than full disasm, and
    I don't think that he could implement E8 as a submodel for LZ/BWT
    (its possible for his CM though)

    > wav compression, I think the nz isn't advanced much than that in FreeArc, which is open source.

    And I think that in fact it is much more advanced. Afair TTA is based on a fairly simple predictor and
    bitfield coding... not even huffman, while nz wav model is comparable to most advanced lossless audio codecs
    (or am I wrong about fa using tta for that?).
    And as to the (maybe relatively small) difference in results, I guess it all depends on chosen samples.
    There's commonly a lot of noise in "original" audio, and noise isn't compressible.

  5. #5
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,572
    Thanks
    780
    Thanked 687 Times in 372 Posts
    >wav handler in rar is a preprocessor,

    afaik, it's model, something like shorten

    > wav compression, I think the nz isn't advanced much than that in FreeArc, which is open source.

    it's tta who is open-source and monkey audio is definitely better

  6. #6
    Member Surfer's Avatar
    Join Date
    Mar 2009
    Location
    oren
    Posts
    203
    Thanks
    18
    Thanked 7 Times in 1 Post
    Quote Originally Posted by Bulat Ziganshin View Post
    it's tta who is open-source and monkey audio is definitely better
    Wavpack have better compression ratio, faster speed and available under BSD license

  7. #7
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,977
    Thanks
    296
    Thanked 1,304 Times in 740 Posts
    @Bulat: as far as I can see in unrar DecodeAudio, there's a delta transform, but no specific entropy coding, and that's
    what I'd call a preprocessor.

  8. #8
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    688
    Thanks
    41
    Thanked 173 Times in 88 Posts
    Quote Originally Posted by Surfer View Post
    Wavpack have better compression ratio, faster speed and available under BSD license
    Better compression than what? Monkey's Audio? Its not true. I've done some tests myself and you can also look here:
    http://www.synthetic-soul.co.uk/comp...dedSize&Desc=0
    As for speed... Run WavPack with -hhx6

  9. #9
    Member Surfer's Avatar
    Join Date
    Mar 2009
    Location
    oren
    Posts
    203
    Thanks
    18
    Thanked 7 Times in 1 Post
    Quote Originally Posted by Skymmer View Post
    Better compression than what? Monkey's Audio?
    Better than TTA. Anyway APE have proprietary license.
    Last Updated: 21st December 2008
    Wavpack 4.60.1 released 1 December 2009.

  10. #10
    Member kampaster's Avatar
    Join Date
    Apr 2010
    Location
    ->
    Posts
    59
    Thanks
    13
    Thanked 7 Times in 7 Posts
    What discussion!
    The linear prediction + CM means.
    Means here it is necessary to drip more.
    (Where to get normally gathering source codes CM)
    And optimfrog is in source codes?
    And Saami answers a mail??
    And what for in winRAR a preprocessor? Delta??
    Really TTA so it is abrupt? He can see wav in a binary file? As can see nanozip ...

  11. #11
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    688
    Thanks
    41
    Thanked 173 Times in 88 Posts
    Quote Originally Posted by Surfer View Post
    Better than TTA. Anyway APE have proprietary license.
    Ah, now I see. From my point of view the best choice for including some lossless audio algo into archiver its TAK. With -p4m its quite close to MAC -c4000 while extremely fast on decoding. Unfortunately, TAK is not open-source and its author cleary stated on HydrogenAudio forum that probably it will never be. Very sad.

    Quote Originally Posted by kampaster View Post
    What discussion!
    And optimfrog is in source codes?
    And Saami answers a mail??
    And what for in winRAR a preprocessor? Delta??
    Really TTA so it is abrupt? He can see wav in a binary file? As can see nanozip ...
    There are no sources for OptimFrog but I have a suspicion that you can get it via direct communication with Florin Ghido and discussing your needs. Can be wrong here.
    Sami answers the letters. At least he replied to my letter with bug report.
    TTA has a simple code. Quite fast but anyway symmetric. No support for metadata in WAV files.

  12. #12
    Member kampaster's Avatar
    Join Date
    Apr 2010
    Location
    ->
    Posts
    59
    Thanks
    13
    Thanked 7 Times in 7 Posts
    And all the same I am surprised nanozip
    Here has carried out the small test:
    Here download a file http://files.gameworld.kz/b8asdfnpbr.html
    There OGG and Wav, in zip archive a demon of compression.
    Has compressed nanozip sfx for loading on a forum.
    Here results:
    Original part6.zip 25 968 kb
    part6.zip.arc 18 589 kb On packing 378mb on unpacking 314mb the RAM, parameter -max
    part6.rar 17 620 kb with included delta and 16 920 kb with disconnected delta
    nanozip: On packing everywhere exposed 64mb the RAM
    Compressor nz_lzhd [36 MB] 15 863 kb
    Compressor nz_lzhd_parallel [36 MB] 15 846 kb
    Compressor nz_lzhd_parallel_extra [36 MB] 15 836 kb SUPER - parity of speed + compression
    Compressor nz_lzhds [28 MB] 15 980 kb
    Compressor nz_lzhds_parallel [47 MB] 15 976 kb
    Compressor nz_lzhds_parallel_extra [47 MB] 15 980 kb
    Compressor nz_lzpf [29 MB] 16 564 kb
    Compressor nz_lzpf_large [68 MB] 16 765 kb
    Compressor nz_optimum1 [64 MB] 19 641 kb
    Compressor nz_optimum2 [81 MB] 18 532 kb
    I considered that nz_optimum2 should press better!
    Compressor nz_cm [98 MB] 15 613 kb
    That for filters uses nanozip???

  13. #13
    Member Fu Siyuan's Avatar
    Join Date
    Apr 2009
    Location
    Mountain View, CA, US
    Posts
    176
    Thanks
    10
    Thanked 17 Times in 2 Posts
    Why use .zip as original? If operate like this, it seems more you are testing their detectors, afaik FreeArc didn't detect by contents. Even nz can't detect out all multimedia data.

  14. #14
    Member kampaster's Avatar
    Join Date
    Apr 2010
    Location
    ->
    Posts
    59
    Thanks
    13
    Thanked 7 Times in 7 Posts
    Fu Siyuan
    But after all nanozip finds out the multimedia data in binary files, to me it is important!

  15. #15
    Member PAQer's Avatar
    Join Date
    Jan 2010
    Location
    Russia
    Posts
    22
    Thanks
    3
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by kampaster View Post
    And all the same I am surprised nanozip
    Here has carried out the small test:
    Here download a file http://files.gameworld.kz/b8asdfnpbr.html
    There OGG and Wav, in zip archive a demon of compression.
    Has compressed nanozip sfx for loading on a forum.
    Here results:
    Original part6.zip 25 968 kb
    part6.zip.arc 18 589 kb On packing 378mb on unpacking 314mb the RAM, parameter -max
    part6.rar 17 620 kb with included delta and 16 920 kb with disconnected delta
    nanozip: On packing everywhere exposed 64mb the RAM
    Compressor nz_lzhd [36 MB] 15 863 kb
    Compressor nz_lzhd_parallel [36 MB] 15 846 kb
    Compressor nz_lzhd_parallel_extra [36 MB] 15 836 kb SUPER - parity of speed + compression
    Compressor nz_lzhds [28 MB] 15 980 kb
    Compressor nz_lzhds_parallel [47 MB] 15 976 kb
    Compressor nz_lzhds_parallel_extra [47 MB] 15 980 kb
    Compressor nz_lzpf [29 MB] 16 564 kb
    Compressor nz_lzpf_large [68 MB] 16 765 kb
    Compressor nz_optimum1 [64 MB] 19 641 kb
    Compressor nz_optimum2 [81 MB] 18 532 kb
    I considered that nz_optimum2 should press better!
    Compressor nz_cm [98 MB] 15 613 kb
    That for filters uses nanozip???
    Soft: Nanozip 0.08 alpha | FreeARC 0.666 | TAK 2.0 final | OptimFROG 4.600ex

    Size in kbytes / Compressor / Decompression speed

    15 498 OptimFROG > 5mb/s
    15 612 NanoZIP > 780kb/s (nz_cm, 380mb)
    15 857 TAK > 60mb/s (p5max)
    15 955 NanoZIP > 46 mb/s (nz_lzhds_parallel_extra, 47 MB)
    16 069 FreeARC > 16 mb/s (tta:2*16)

    Same files, but TARed:
    14 962 OptimFROG
    15 319 TAK
    15 490 FreeARC
    15 614 NanoZIP (nz_cm, 380mb)
    15 922 NanoZIP (nz_lzhds_parallel_extra, 47 MB)

  16. #16
    Member kampaster's Avatar
    Join Date
    Apr 2010
    Location
    ->
    Posts
    59
    Thanks
    13
    Thanked 7 Times in 7 Posts
    PAQer
    Can fill in where be all compressors which used in the test?
    Thanks

  17. #17
    Member kampaster's Avatar
    Join Date
    Apr 2010
    Location
    ->
    Posts
    59
    Thanks
    13
    Thanked 7 Times in 7 Posts

  18. #18
    Member PAQer's Avatar
    Join Date
    Jan 2010
    Location
    Russia
    Posts
    22
    Thanks
    3
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by kampaster View Post
    PAQer
    Can fill in where be all compressors which used in the test?
    Thanks
    TAK & OptimFROG are usual lossless codecs like FLAC or WavPack. But no one bothers you using they for not typical tasks.

    OptimFROG's page and link to TAK

  19. #19
    Member kampaster's Avatar
    Join Date
    Apr 2010
    Location
    ->
    Posts
    59
    Thanks
    13
    Thanked 7 Times in 7 Posts
    PAQer Thanks only here the reference do not work for me=((
    http://www.losslessaudio.org/ NO

Similar Threads

  1. NanoZip - a new archiver, using bwt, lz, cm, etc...
    By Sami in forum Data Compression
    Replies: 305
    Last Post: 27th July 2020, 12:48
  2. Nanozip decompression data troubles
    By SvenBent in forum Data Compression
    Replies: 11
    Last Post: 12th January 2009, 22:25
  3. NanoZip huge efficiency issue
    By m^2 in forum Data Compression
    Replies: 9
    Last Post: 10th September 2008, 21:51
  4. enwik9 benchmark nanozip, bliz, m99, dark
    By Sami in forum Data Compression
    Replies: 6
    Last Post: 31st July 2008, 20:24
  5. Replies: 33
    Last Post: 24th October 2007, 12:39

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •