Results 1 to 11 of 11

Thread: Disassembled LZTurbo.exe 0.92...

  1. #1
    Member Raymond_NGhM's Avatar
    Join Date
    Oct 2008
    Location
    UK
    Posts
    51
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Here, you are Encoders...
    AND now... I say Welcome Decoders.

    Download LZTturbo092.7z

  2. #2
    Member
    Join Date
    Apr 2007
    Posts
    16
    Thanks
    0
    Thanked 0 Times in 0 Posts
    from tornado 0.3:

    LZ77_Coder.cpp:
    uint extra_dbits[32] = {4,4,5,5,5,6,6,7,7,8,8,9,9,10,10,11,11,12,12,13,13 ,14,14,15,15,16,17,18,19,21,23,30};

    from lzturbo asm as provided by raymond::

    L0044B640:

    db 04h;

    db 04h;

    db 05h;

    db 05h;

    db 05h;

    db 06h;

    db 06h;

    db 07h;

    db 07h;

    db 08h;

    db 08h;

    db 09h;

    db 09h;

    db 0Ah;

    db 0Ah;

    db 0Bh;

    db 0Bh;

    db 0Ch;

    db 0Ch;

    db 0Dh;

    db 0Dh;

    db 0Eh;

    db 0Eh;

    db 0Fh;

    db 0Fh;

    db 10h;

    db 11h;

    db 12h;

    db 13h;

    db 14h;

    db 16h;

    db 18h;

    the code look surprisingly similar, but also note:

    - the last 3 bytes are different

    - tornado uses int vs. lzturbo uses char as datatype

    - this buffer is nothing else than a representation of the lz77 deflate 64 distance tree... but all implementations i found start with value 0 not 4... so whatever source or paper both authors based their tree on, i guess its the same and would like to know which one.

    tired and lazy i give up...

    bulat: please can you tell me why the buffer starts with value 4?

  3. #3
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    Code:
    extra_lbits2: 
    zip: {0,0,0,0,0,0,0,0,1,1,1,1,2,2,2,2,3,3,3,3,4,4,4,4,5,5,5,5,0} 
    tor: {0,0,0,0,0,0,0,1,1,2,2,3,3,4,8,30}; 
    lzt: 00 00 01 01 02 02 03 03 04 04 05 05 06 07 08 18 
     
    extra_dbits: 
    zip: {0,0,0,0,1,1,2,2,3,3,4,4,5,5,6,6, 
          7,7,8,8,9,9,10,10,11,11,12,12,13,13} 
    tor: {4,4,5,5,5,6,6,7,7,8,8,9,9,10,10,11, 
          11,12,12,13,13,14,14,15,15,16,17,18,19,21,23,30} 
    lzt: 04 04 05 05 05 06 06 07 07 08 08 09 09 0A 0A 0B 
         0B 0C 0C 0D 0D 0E 0E 0F 0F 10 11 12 13 14 16 18

  4. #4
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    What does that mean? Is it similar as in tuned or more different?

  5. #5
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    - the last 3 bytes are different
    lzt compress data in 16mb blocks, so dists/lens are limited to 2^24. it's why he limited last values in both tables to 24, and made appropriate changes in two previous values. but note that he retained sizes of both tables - 16 and 32, respectively

    it's interesting to note that using lengths up to the size of entire buffer/window is a "smell" of my programs - arjz/tornado. all other programs i've seen, limits this to 258, 273 or smth like this (well, one more exception is uc2 forget at these days)

    - tornado uses int vs. lzturbo uses char as datatype
    he is smarter than me!

    - this buffer is nothing else than a representation of the lz77 deflate 64 distance tree... but all implementations i found start with value 0 not 4... so whatever source or paper both authors based their tree on, i guess its the same and would like to know which one.

    first, read lzxfmt from http://www.haskell.org/bz/lzh.7z

    i use combined dist/len code like lzx in order to make (de)compression faster. with 16 len codes and 32 dist codes there are total 512 codes for matches. this makes adding dist codes rather expensive and moreover you should add a lot of them to improve compression, like zip's {0,0,0,0,1,1,2,2,3,3}. this 10 more codes will increase total amount of match codes up to 672. moreover, unlike zip, tor uses repdist/repboth codes which significantly reduces amount of matches with small dist. moreover, unlike lzx, tor is not optimized strictly for executables/binary files. taking this all together, the resulting decision was to avoid these small codes. this slightly improves text compression and probably slightly decreases binary compression - experiment yourself

    it's interesting to note that lbits table in lzt was further optimized for text files. nevertheless, the 16*32 configuration remains untouched. lzx, for example, uses 8*50 codes

  6. #6
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    Ilya, CODE sections are absolutely unreadable now - the letters are very small

  7. #7
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,985
    Thanks
    377
    Thanked 353 Times in 141 Posts
    Quote Originally Posted by Bulat Ziganshin
    Ilya, CODE sections are absolutely unreadable now - the letters are very small
    FIXED!

  8. #8
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    thaks, Ilya. now it's perfect

  9. #9
    Member Raymond_NGhM's Avatar
    Join Date
    Oct 2008
    Location
    UK
    Posts
    51
    Thanks
    0
    Thanked 0 Times in 0 Posts
    WELL WELL WELL

    How do you think about it, Bulat Ziganshin...
    It compiled with GCC 4
    He think, ( my mean Hamid ) who is he, hah.
    and now LZTurbo is opensource.
    yes of course, of course 'Doctor Reactor'

    AH, one another thing, LZMA is best after RAR
    and igor need more and more to improve and improve IT.
    specially that,in adding multimedia filters for .7z

  10. #10
    Member Raymond_NGhM's Avatar
    Join Date
    Oct 2008
    Location
    UK
    Posts
    51
    Thanks
    0
    Thanked 0 Times in 0 Posts
    And Good luck my friends.

  11. #11
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    Quote Originally Posted by Black_Fox
    What does that mean? Is it similar as in tuned or more different?
    either:
    tor&lzt was written independent, but by accident
    1) both have the same 16*32 match coder configuration (as opposed to 8*50 used in the only other coder of this type)
    2) both start dist table with 4-bit codes (as opposed to every other coder that start with 0-bit codes)
    3) both use lengths up to the size of entire buffer

    third solution looks natural although not yet used anywhere except uc2. second solution is also improvement although independent develper may select to start table with 3-bit or 5-bit codes, this particular choice (4) is rather arbitrary. first solution seems rather arbitrary, although to some degree it is justified by using the same tables for bitcoder too

    or:
    dns may have started with tornado code and changed the following:
    1) last elements of len/dist tables due to 16mb limit of blocks used
    2) len codes to improve compression of enwik9 while decreasing compression of binary files


    btw, ive also found cabarc-like dist table in lzt executable:
    <div class=""jscript""><pre>00 00 00 00 01 01 02 02 │ 03 03 04 04 05 05 06 06
    07 07 08 08 09 09 0A 0A │ 0B 0B 0C 0C 0D 0D 0E 0E
    0F 0F 10 10 11 11 12 12 │ 12 12 12 12 12 12 12 12
    12 12 12 12 12 12 12 12 │ 12 12 12 12 12 12 12 12
    12 12 12 12 12 12 12 12 │ 12 12 12 12 12 12 12 12
    12 12 12 12 12 12 12 12 │ 12 12 12 12 12 12 12 12
    12 12 12 12 12[/code]



    probably, one table is used in -4* mode and another one in -5*. and -5* mode is slower because it encodes len and dist separately

    also, -59 speed seems rather close to 7zip -mx5

Similar Threads

  1. The worst case for LZTurbo?
    By m^2 in forum Data Compression
    Replies: 8
    Last Post: 16th April 2009, 01:24
  2. New Disassembled LZTurbo.exe 0.92...
    By Raymond_NGhM in forum Forum Archive
    Replies: 1
    Last Post: 19th April 2008, 09:58
  3. LZTURBO 0.9 parallel compressor
    By donotdisturb in forum Forum Archive
    Replies: 18
    Last Post: 6th March 2008, 01:23
  4. RASH - EXE-cryptor
    By encode in forum Forum Archive
    Replies: 21
    Last Post: 11th February 2008, 11:53
  5. LZTURBO 0.1 parallel compressor
    By donotdisturb in forum Forum Archive
    Replies: 5
    Last Post: 7th October 2007, 23:44

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •