Results 1 to 12 of 12

Thread: 4x4 bug?

  1. #1
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts

    4x4 bug?

    I found a strange thing happening - when I compress the same files multiple times, I got different output archives.
    It happens in modes 4..7, also with single threaded compression. They all decompress correctly.
    Is it a bug or just unusual, but expected behaviour?
    It might be related to this, that's the same test set, just tared.
    Standalone Tornado doesn't seem to have this issue.

    BTW, encode, could you drop or reduce search length word limit?
    4 characters is 1 too much.

  2. #2
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    408
    Thanks
    0
    Thanked 5 Times in 5 Posts
    You find something like this in most lz based compressors. It's absolut normal. I or someone else could give you a technical answer if you want to.

  3. #3
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Could you do this, please?
    I didn't mean that different modes work differently, but that each of them is non deterministic.

  4. #4
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    735
    Thanked 660 Times in 354 Posts
    i'm also intrigued )

  5. #5
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    408
    Thanks
    0
    Thanked 5 Times in 5 Posts
    Yes I understood what you mean. I'm not that into how most of the programs work but I am on the way of programming my own lz based compressor at the moment. I write my output into a unititialized buffer and at writing the buffer to the file there are most times some bits not used (at the last byte of the buffer) which could have any values but it does not matter because they won?t be interpreted at any time.
    I hope what I wrote is understandable

  6. #6
    Programmer toffer's Avatar
    Join Date
    May 2008
    Location
    Erfurt, Germany
    Posts
    587
    Thanks
    0
    Thanked 0 Times in 0 Posts
    I always thought the main reason is a uninitialized hash table (for LZ77 based stuff)?!

  7. #7
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by Simon Berger View Post
    Yes I understood what you mean. I'm not that into how most of the programs work but I am on the way of programming my own lz based compressor at the moment. I write my output into a unititialized buffer and at writing the buffer to the file there are most times some bits not used (at the last byte of the buffer) which could have any values but it does not matter because they won?t be interpreted at any time.
    I hope what I wrote is understandable
    But it shouldn't change the output size?
    There's like 0.1% variation with 2 threads. I run total of 15 tests ( 3*3 + 1*6 ) in this mode and only 1 result was repeated.
    With 1 thread it was 12 tests (4*3) and only 1 result was 1 byte off, so this is probably related to multithreading.

    And Bulat..grab the files and see what's the reason yourself.

  8. #8
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    735
    Thanked 660 Times in 354 Posts
    i've found bug which should lead to this problem. at the time when i wrote 4x4 i comletely firget that tornado has 1 global variable left- - it's used to detect binary tables in data (like delta filter does). i don't yet tested fixed 4x4 but hope that now it will compress binary data 1-2% better and show stable results. thank you for reporting the problem!

  9. #9
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by Bulat Ziganshin View Post
    i've found bug which should lead to this problem. at the time when i wrote 4x4 i comletely firget that tornado has 1 global variable left- - it's used to detect binary tables in data (like delta filter does). i don't yet tested fixed 4x4 but hope that now it will compress binary data 1-2% better and show stable results. thank you for reporting the problem!
    How about the single threaded mode?
    There was 1 result 1 byte different too. I can't find the logs, strange...possibly I lost them. It may be my mistake, but such assumptions are dangerous.
    I'll try to recreate the problem.

  10. #10
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    735
    Thanked 660 Times in 354 Posts
    well, i've already fixed bug with unitialized hash (exactly as Simon said, although it was unintentional) so we may expect that there were 2 bugs - one small and one worth 1% of compression. i've attached fixed 4x4, can you please test it thoroughly?
    Attached Files Attached Files

  11. #11
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    I did ~100 tests and everything seems OK.

  12. #12
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    735
    Thanked 660 Times in 354 Posts
    thanks!

Similar Threads

  1. FreeArc compression suite (4x4, Tornado, REP, Delta, Dict...)
    By Bulat Ziganshin in forum Data Compression
    Replies: 554
    Last Post: 26th September 2018, 02:41
  2. paq8pf(beta2) bug
    By Bulat Ziganshin in forum Data Compression
    Replies: 3
    Last Post: 30th October 2009, 01:42
  3. PAQ8 C++ precedence bug (or "-Wparentheses is annoying")
    By Rugxulo in forum Data Compression
    Replies: 13
    Last Post: 21st August 2009, 20:36
  4. Timer 3.01 bug
    By Rugxulo in forum The Off-Topic Lounge
    Replies: 7
    Last Post: 31st July 2009, 21:36
  5. 4x4 - multithreaded compressor
    By Bulat Ziganshin in forum Forum Archive
    Replies: 12
    Last Post: 19th April 2008, 17:25

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •