Page 3 of 85 FirstFirst 123451353 ... LastLast
Results 61 to 90 of 2532

Thread: zpaq updates

  1. #61
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    Nice improvement. Some results for bmp_j1 on the Kodak set.
    Code:
    zpaq108 cmax.cfg
     503,436 kodim01.bmp.zpaq (9 sec per file optimized)
     378,583 kodim02.bmp.zpaq
     327,005 kodim03.bmp.zpaq
     441,954 kodim04.bmp.zpaq
     640,768 kodim05.bmp.zpaq
     458,957 kodim06.bmp.zpaq
     385,302 kodim07.bmp.zpaq
     665,842 kodim08.bmp.zpaq
     392,373 kodim09.bmp.zpaq
     428,791 kodim10.bmp.zpaq
     465,125 kodim11.bmp.zpaq
     359,799 kodim12.bmp.zpaq
     649,429 kodim13.bmp.zpaq
     535,099 kodim14.bmp.zpaq
     427,761 kodim15.bmp.zpaq
     359,518 kodim16.bmp.zpaq
     420,052 kodim17.bmp.zpaq
     633,543 kodim18.bmp.zpaq
     472,418 kodim19.bmp.zpaq
     352,003 kodim20.bmp.zpaq
     470,359 kodim21.bmp.zpaq
     579,559 kodim22.bmp.zpaq
     464,091 kodim23.bmp.zpaq
     546,375 kodim24.bmp.zpaq
    11,358,142 bytes
    
    zpaq108 cbmp_j1.cfg
     395,186 kodim01.bmp.zpaq (8 sec per file optimized)
     320,798 kodim02.bmp.zpaq
     268,654 kodim03.bmp.zpaq
     348,623 kodim04.bmp.zpaq
     470,393 kodim05.bmp.zpaq
     366,516 kodim06.bmp.zpaq
     305,855 kodim07.bmp.zpaq
     462,274 kodim08.bmp.zpaq
     314,971 kodim09.bmp.zpaq
     331,017 kodim10.bmp.zpaq
     355,667 kodim11.bmp.zpaq
     299,517 kodim12.bmp.zpaq
     501,523 kodim13.bmp.zpaq
     420,000 kodim14.bmp.zpaq
     332,963 kodim15.bmp.zpaq
     301,618 kodim16.bmp.zpaq
     323,406 kodim17.bmp.zpaq
     475,853 kodim18.bmp.zpaq
     373,410 kodim19.bmp.zpaq
     287,555 kodim20.bmp.zpaq
     375,628 kodim21.bmp.zpaq
     432,041 kodim22.bmp.zpaq
     338,430 kodim23.bmp.zpaq
     393,155 kodim24.bmp.zpaq
    8,795,053 bytes
    
    paq8px_v60_turbo -6
     379,021 kodim01.bmp.paq8px (no color transform)
     311,804 kodim02.bmp.paq8px
     251,387 kodim03.bmp.paq8px
     327,269 kodim04.bmp.paq8px
     438,515 kodim05.bmp.paq8px
     349,815 kodim06.bmp.paq8px
     283,934 kodim07.bmp.paq8px
     433,936 kodim08.bmp.paq8px
     300,962 kodim09.bmp.paq8px
     310,515 kodim10.bmp.paq8px
     338,414 kodim11.bmp.paq8px
     281,905 kodim12.bmp.paq8px
     475,658 kodim13.bmp.paq8px
     394,315 kodim14.bmp.paq8px
     313,626 kodim15.bmp.paq8px
     286,418 kodim16.bmp.paq8px
     304,319 kodim17.bmp.paq8px
     444,424 kodim18.bmp.paq8px
     353,101 kodim19.bmp.paq8px
     276,842 kodim20.bmp.paq8px
     356,238 kodim21.bmp.paq8px
     405,036 kodim22.bmp.paq8px
     312,148 kodim23.bmp.paq8px
     368,399 kodim24.bmp.paq8px
    8,298,001 bytes
    
    paq8px_v64 -6
     338,569 kodim01.bmp.paq8px (25 sec per file)
     273,886 kodim02.bmp.paq8px
     230,970 kodim03.bmp.paq8px
     290,665 kodim04.bmp.paq8px
     393,021 kodim05.bmp.paq8px
     313,474 kodim06.bmp.paq8px
     256,819 kodim07.bmp.paq8px
     392,708 kodim08.bmp.paq8px
     271,395 kodim09.bmp.paq8px
     278,701 kodim10.bmp.paq8px
     303,628 kodim11.bmp.paq8px
     253,575 kodim12.bmp.paq8px
     432,576 kodim13.bmp.paq8px
     352,878 kodim14.bmp.paq8px
     283,793 kodim15.bmp.paq8px
     258,925 kodim16.bmp.paq8px
     278,275 kodim17.bmp.paq8px
     406,074 kodim18.bmp.paq8px
     319,608 kodim19.bmp.paq8px
     259,328 kodim20.bmp.paq8px
     323,466 kodim21.bmp.paq8px
     366,561 kodim22.bmp.paq8px
     284,988 kodim23.bmp.paq8px
     333,431 kodim24.bmp.paq8px
    7,497,314 bytes
    Compression time is about 21 sec. per file with zpaq 1.07 and 8 seconds after optimizing with zpaq 1.08. paq8px takes about 25 seconds per file. v60 does not include a color transform. v64 uses the transform (b,g,r) -> (g,g-r,g-b).

  2. #62
    Programmer Jan Ondrus's Avatar
    Join Date
    Sep 2008
    Location
    Rychnov nad Kněžnou, Czech Republic
    Posts
    279
    Thanks
    33
    Thanked 138 Times in 50 Posts
    bmp_j2.cfg
    - improved 24-bit image compression model and added (b,g,r) -> (g,g-r,g-b) color transformation (colorpre.cfg).

    rafale.bmp ... 4149414 bytes
    zpaq cmax.cfg ... 714263 bytes
    zpaq cbmp_j1.cfg ... 538228 bytes
    zpaq cbmp_j2.cfg ... 531900 bytes

    kodim01.bmp ... 1179702 bytes
    zpaq cmax.cfg ... 503436 bytes
    zpaq cbmp_j1.cfg ... 395186 bytes
    zpaq cbmp_j2.cfg ... 340242 bytes
    Attached Files Attached Files
    Last edited by Jan Ondrus; 12th October 2009 at 14:30.

  3. #63
    Programmer Jan Ondrus's Avatar
    Join Date
    Sep 2008
    Location
    Rychnov nad Kněžnou, Czech Republic
    Posts
    279
    Thanks
    33
    Thanked 138 Times in 50 Posts
    1) I can't detect end of file in HCOMP section (it is possible in PCOMP)
    2) I miss debug option for zpaq to write output of postprocessor (PCOMP) to file if comparison with unpreprocessed input FAILED.

    I solved this by adding this two lines before running postprocessor.

    Code:
    out=fopen("xxx", "wb");
          pz.output=out;

  4. #64
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    Very nice result. Compression is almost as good as paq8px_v64 and over twice as fast (11 sec. vs 25 sec. per file).

    Code:
    zpaq cbmp_j2.cfg
     340,242 kodim01.bmp.zpaq (11 sec per file)
     274,770 kodim02.bmp.zpaq
     233,212 kodim03.bmp.zpaq
     292,103 kodim04.bmp.zpaq
     393,855 kodim05.bmp.zpaq
     315,403 kodim06.bmp.zpaq
     257,922 kodim07.bmp.zpaq
     392,021 kodim08.bmp.zpaq
     272,943 kodim09.bmp.zpaq
     280,186 kodim10.bmp.zpaq
     304,658 kodim11.bmp.zpaq
     255,503 kodim12.bmp.zpaq
     433,220 kodim13.bmp.zpaq
     355,289 kodim14.bmp.zpaq
     283,121 kodim15.bmp.zpaq
     261,497 kodim16.bmp.zpaq
     279,832 kodim17.bmp.zpaq
     405,668 kodim18.bmp.zpaq
     321,464 kodim19.bmp.zpaq
     260,934 kodim20.bmp.zpaq
     325,393 kodim21.bmp.zpaq
     367,238 kodim22.bmp.zpaq
     286,420 kodim23.bmp.zpaq
     335,435 kodim24.bmp.zpaq
    7,528,329 bytes
    1. It is true you can't detect EOF in HCOMP. The model can only handle 8 bits per byte. A possible fix is to add info in the preprocessor, such as file size at beginning or special symbol at end.

    2. Another way to get the output of a failed preprocessor test:

    zpaq rcolorpre.cfg input tmp
    zpaq prbmp_j2.cfg tmp output

    A production compressor for BMP (with no config file or external preprocessor) would probably read the BMP file, get the image width, and modify the ZPAQL code appropriately. You can get a dump of the ZPAQL code as a C++ array of bytes from zpaqopt.h and use ZPAQL::read(Reader(array, size)) to load the code before compression. Before writing the header and compressing, modify the part of the code that depends on image width, and put each image in a separate block.

  5. #65
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,159
    Thanks
    707
    Thanked 462 Times in 357 Posts
    Hi!
    bmp_j2 is really a best performer. My scores:

    1.bmp = 600'054
    1.bmp/paq8px1 -7 = 264'193
    1.bmp/paq8px64 -7 = 236'363
    1.bmp/zpaq1.08 max = 358'276
    1.bmp/zpaq1.08 bmp_j2 = 235'691

    a.tif = 2'870'552
    a.tif/paq8px1 -8 = 864'257
    a.tif/paq8px64 -8 = 438'372
    a.tif/zpaq1.08 max = 1'005'564
    a.tif/zpaq1.08 bmp_j2 = FAILED (bmp_j1 = 790'261)

    b.tga = 2'872'818
    b.tga/paq8px1 -8 = 813'144
    b.tga/paq8px64 -8 = 402'625
    b.tga/zpaq1.08 max = 945'034
    b.tga/zpaq1.08 bmp_j2 = 616'520

    Of course, bmp_j2 doesn't have TIFF or TGA, recognition, then scores are worse than px64, however still much better than max config or px1.

    I've tried to test latest exe and bwt configs, but it need compiled bwtpre and exepre, which I haven't. Does anybody have compiled cersions of these files?

    Darek

  6. #66
    Programmer Jan Ondrus's Avatar
    Join Date
    Sep 2008
    Location
    Rychnov nad Kněžnou, Czech Republic
    Posts
    279
    Thanks
    33
    Thanked 138 Times in 50 Posts
    Quote Originally Posted by Darek View Post
    I've tried to test latest exe and bwt configs, but it need compiled bwtpre and exepre, which I haven't. Does anybody have compiled cersions of these files?
    This versions seems to contain compiled preprocessors:
    http://mattmahoney.net/dc/exe_j1.zip
    http://mattmahoney.net/dc/bwt_j3.zip

    bmp_j3.cfg - improved 24-bit compression model

    rafale.bmp ... 4149414 bytes
    zpaq cmax.cfg ... 714263 bytes
    zpaq cbmp_j1.cfg ... 538228 bytes
    zpaq cbmp_j2.cfg ... 531900 bytes
    zpaq cbmp_j3.cfg ... 523693 bytes

    kodim01.bmp ... 1179702 bytes
    zpaq cmax.cfg ... 503436 bytes
    zpaq cbmp_j1.cfg ... 395186 bytes
    zpaq cbmp_j2.cfg ... 340242 bytes
    zpaq cbmp_j3.cfg ... 338204 bytes

    Should be as good as paq8px_v64 or slightly better on average.

    EDIT:
    rafale.bmp with special transformation trick:
    Code:
          g^=(g&4)?3:0;
          r^=(r&8)?7:0;
          b^=(b&8)?7:0;
    -> 507521 bytes
    Attached Files Attached Files
    Last edited by Jan Ondrus; 13th October 2009 at 15:17.

  7. #67
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,159
    Thanks
    707
    Thanked 462 Times in 357 Posts
    Jan,
    many thanks for compilations.
    Darek

  8. #68
    Programmer Jan Ondrus's Avatar
    Join Date
    Sep 2008
    Location
    Rychnov nad Kněžnou, Czech Republic
    Posts
    279
    Thanks
    33
    Thanked 138 Times in 50 Posts
    Kodak set (http://r0k.us/graphics/kodak/) - result:
    Code:
    zpaq108 cmax.cfg ... 11358142 bytes
    zpaq cbmp_j1.cfg ... 8795053 bytes
    paq8px_v60       ... 8298001 bytes
    zpaq cbmp_j2.cfg ... 7528329 bytes
    zpaq cbmp_j3.cfg ... 7500633 bytes
    paq8px_v64       ... 7497314 bytes
    zpaq cbmp_j4.cfg ... 7485249 bytes
    Attached Files Attached Files

  9. #69
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    Some results with zpaq 1.08, which I just uploaded to http://mattmahoney.net/dc/

    Code:
    zpaq108 ocbmp_j4.cfg
     338,151 kodim01.bmp.zpaq (10-11 sec each)
     272,029 kodim02.bmp.zpaq
     230,431 kodim03.bmp.zpaq
     291,064 kodim04.bmp.zpaq
     393,110 kodim05.bmp.zpaq
     314,011 kodim06.bmp.zpaq
     255,350 kodim07.bmp.zpaq
     390,582 kodim08.bmp.zpaq
     270,386 kodim09.bmp.zpaq
     278,175 kodim10.bmp.zpaq
     303,601 kodim11.bmp.zpaq
     253,364 kodim12.bmp.zpaq
     431,880 kodim13.bmp.zpaq
     353,718 kodim14.bmp.zpaq
     281,349 kodim15.bmp.zpaq
     258,687 kodim16.bmp.zpaq
     277,247 kodim17.bmp.zpaq
     404,235 kodim18.bmp.zpaq
     319,912 kodim19.bmp.zpaq
     259,391 kodim20.bmp.zpaq
     323,942 kodim21.bmp.zpaq
     365,658 kodim22.bmp.zpaq
     285,507 kodim23.bmp.zpaq
     333,469 kodim24.bmp.zpaq
    7,485,249 bytes
    This compares to 7,497,314 bytes and about 25 seconds per file for paq8px_v64 -6. The "o" modifier searches for optimized code (compiled COMP and ZPAQL) and creates and compiles it as needed. This adds about 2 seconds to the first file to run the g++ compiler and put the optimized version of ZPAQ in %TEMP%. The rest will find the .exe already there and run it. This compares to 26 seconds for zpaq 1.07 or if you don't use the "o" modifier in v1.08. Decompress with "ox".

    To install, you need to configure zpaqmake.bat to find zpaq.cpp (or zpaq.o) and zpaq.h and call your C++ compiler with appropriate options, then put the script and zpaq.exe somewhere in your PATH. The program name will be passed as an argument. It should be compiled with -DNDEBUG -DOPT. An example is included.

    Optimized programs are created by replacing the interpreters in ZPAQL::run(), Predictor:: predict() and Predictor::update() with compiled code. -DOPT causes zpaq.cpp to strip out unneeded features (like "l" and "r" commands), and to link to this code that it just wrote. When decompressing, ZPAQ calls its optimized version of itself once for each block in the archive. For compressing, it only calls itself once because ZPAQ adds one block at a time. The name of the new program is based on the SHA1 hash of the COMP, HCOMP, and PCOMP code from the config file or block header, plus the name of the preprocessor. The decompresser doesn't check the header before decompressing. It just assumes there will be no hash collisions, so there is a 2^-160 probability of failing

    You can delete the contents of %TEMP% any time you want and ZPAQL will create new files as needed.

    I did not test in Linux yet but I designed with it in mind. ZPAQ will look in $TMPDIR (normally /tmp) first, then $TEMP, then in the current directory. You would need to write a shell script equivalent to zpaqmake.bat. The supplied script was tested with g++, Borland and Mars compilers.
    Last edited by Matt Mahoney; 15th October 2009 at 06:32.

  10. #70
    Programmer Jan Ondrus's Avatar
    Join Date
    Sep 2008
    Location
    Rychnov nad Kněžnou, Czech Republic
    Posts
    279
    Thanks
    33
    Thanked 138 Times in 50 Posts
    bwt_slowmode1
    - slow mode (using 1.25x block-size memory) inverse bwt rewritten to ZPAQL.
    Attached Files Attached Files

  11. #71
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    I continue to be amazed at your work I tested on enwik9. Decompression with optimized code in BWT slow mode is faster than unoptimized code in fast mode. http://mattmahoney.net/dc/text.html#1493

    Code:
                      Compression                          Compressed size      Decompresser  Total size   Time (ns/byte)
    Program            Options                           enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note
    -------            -------                         ----------  -----------  -----------  -----------  ----- -----  --- --- ----
    zpaq 1.07          cbwt_j2.cfg,18                  20,756,888  174,171,969     13,421 x  174,185,390   5593  4347 1443 BWT 26
    zpaq 1.08          ocbwt_slowmodel_1GB_block.cfg   20,756,996  163,565,006     29,153 x  163,594,159   7957  3875 1443 BWT 26
    Last edited by Matt Mahoney; 16th October 2009 at 02:23.

  12. #72
    Programmer Jan Ondrus's Avatar
    Join Date
    Sep 2008
    Location
    Rychnov nad Kněžnou, Czech Republic
    Posts
    279
    Thanks
    33
    Thanked 138 Times in 50 Posts
    Quote Originally Posted by Matt Mahoney View Post
    You could even write a JPEG compressor that undid the Huffman coded DCT coefficients and append a ZPAQL program to redo the Huffman coding. Any existing ZPAQ decompressor could then read your files even though the compression algorithm hadn't even been written yet
    Done.
    It has no special jpeg model, only transformation for decoding/coding Huffman.

    a10.jpg ... 842468 bytes
    zpaq.exe cjpg_test1.cfg ... 745710 bytes (-11.5%)

    EDIT:
    attachment removed - please download fixed version
    Last edited by Jan Ondrus; 19th October 2009 at 12:49.

  13. #73
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    The preprocessor test failed on the 3 act-jpeg test images.
    http://compression.ca/act/act-jpeg.html

  14. #74
    Programmer Jan Ondrus's Avatar
    Join Date
    Sep 2008
    Location
    Rychnov nad Kněžnou, Czech Republic
    Posts
    279
    Thanks
    33
    Thanked 138 Times in 50 Posts
    Quote Originally Posted by Matt Mahoney View Post
    The preprocessor test failed on the 3 act-jpeg test images.
    http://compression.ca/act/act-jpeg.html
    There was bug in read_header() procedure:
    (r[2] > 192 || r[2] < 255) --> (r[2] > 192 && r[2] < 255)

    Here is fixed version test1 + version test2 with basic jpeg model.

    DSCN3974.jpg - 1114198 bytes
    zpaq cjpg_test1.cfg - 989232 bytes
    zpaq cjpg_test2.cfg - 916368 bytes
    Last edited by Jan Ondrus; 20th October 2009 at 16:19.

  15. #75
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    409
    Thanks
    0
    Thanked 5 Times in 5 Posts
    You are unbelievable!

  16. #76
    Member
    Join Date
    Aug 2009
    Location
    Bari
    Posts
    74
    Thanks
    1
    Thanked 1 Time in 1 Post
    W-O-N-D-E-R-F-U-L!

  17. #77
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    Unfortunately jpg_test1.cfg and jpg_test2.cfg fail on 47 of 100 files in the Picasa data set (100 random JPEGS I got from Picasa).

    http://mattmahoney.net/tmp/picasa.zip

    results are shown below with paq8px_v64 -6 for comparison. Files with no zpaq1 or zpaq2 compression failed the preprocessor test.

    Code:
    08/25/2007  04:04 PM           320,403 0607087.JPG
    10/19/2009  02:58 PM           257,829 0607087.JPG.paq8px
    10/19/2009  02:40 PM           286,821 0607087.JPG.zpaq1
    10/19/2009  02:48 PM           269,639 0607087.JPG.zpaq2
    08/25/2007  03:51 PM            26,513 070330_23310001.jpg
    10/19/2009  02:58 PM            21,741 070330_23310001.jpg.paq8px
    10/19/2009  02:40 PM            25,019 070330_23310001.jpg.zpaq1
    10/19/2009  02:48 PM            24,210 070330_23310001.jpg.zpaq2
    08/25/2007  04:03 PM           113,463 070730_Arcachon_22.jpg
    10/19/2009  02:58 PM            92,361 070730_Arcachon_22.jpg.paq8px
    08/25/2007  04:04 PM            85,686 070806_Main_06.jpg
    10/19/2009  02:58 PM            67,946 070806_Main_06.jpg.paq8px
    08/25/2007  04:03 PM         1,029,908 1010177_IMG.JPG
    10/19/2009  02:59 PM           745,430 1010177_IMG.JPG.paq8px
    10/19/2009  02:40 PM           895,150 1010177_IMG.JPG.zpaq1
    10/19/2009  02:49 PM           798,347 1010177_IMG.JPG.zpaq2
    08/25/2007  03:54 PM           601,832 110_1063.JPG
    10/19/2009  02:59 PM           454,413 110_1063.JPG.paq8px
    08/25/2007  03:56 PM           278,467 1180924891597.jpg
    10/19/2009  02:59 PM           226,190 1180924891597.jpg.paq8px
    10/19/2009  02:40 PM           254,910 1180924891597.jpg.zpaq1
    10/19/2009  02:49 PM           241,356 1180924891597.jpg.zpaq2
    08/25/2007  03:53 PM            60,435 1st_Haiti_Trip_012.jpg
    10/19/2009  02:59 PM            50,578 1st_Haiti_Trip_012.jpg.paq8px
    10/19/2009  02:40 PM            57,154 1st_Haiti_Trip_012.jpg.zpaq1
    10/19/2009  02:49 PM            53,971 1st_Haiti_Trip_012.jpg.zpaq2
    08/25/2007  04:03 PM           102,378 20070324_271.jpg
    10/19/2009  03:00 PM            67,373 20070324_271.jpg.paq8px
    10/19/2009  02:40 PM            80,213 20070324_271.jpg.zpaq1
    10/19/2009  02:49 PM            81,798 20070324_271.jpg.zpaq2
    08/25/2007  04:00 PM           117,272 2007398.jpg
    10/19/2009  03:00 PM            82,849 2007398.jpg.paq8px
    10/19/2009  02:40 PM            99,894 2007398.jpg.zpaq1
    10/19/2009  02:49 PM            91,401 2007398.jpg.zpaq2
    08/25/2007  04:02 PM            56,446 4215626041.jpg
    10/19/2009  03:00 PM            55,432 4215626041.jpg.paq8px
    08/25/2007  04:04 PM            40,595 49.jpg
    10/19/2009  03:00 PM            33,494 49.jpg.paq8px
    10/19/2009  02:41 PM            37,862 49.jpg.zpaq1
    10/19/2009  02:49 PM            37,274 49.jpg.zpaq2
    08/25/2007  03:44 PM            11,709 5.jpg
    10/19/2009  03:00 PM             9,960 5.jpg.paq8px
    10/19/2009  02:41 PM            12,021 5.jpg.zpaq1
    10/19/2009  02:49 PM            12,167 5.jpg.zpaq2
    08/25/2007  03:48 PM            47,558 53500006.jpg
    10/19/2009  03:00 PM            40,601 53500006.jpg.paq8px
    10/19/2009  02:41 PM            45,351 53500006.jpg.zpaq1
    10/19/2009  02:49 PM            44,154 53500006.jpg.zpaq2
    08/25/2007  04:05 PM           163,423 AkhaSwingFest_SIMANN_10.JPG
    10/19/2009  03:00 PM           126,264 AkhaSwingFest_SIMANN_10.JPG.paq8px
    08/25/2007  04:04 PM            89,780 AkhaSwingFest_SIMANN_9.JPG
    10/19/2009  03:00 PM            65,877 AkhaSwingFest_SIMANN_9.JPG.paq8px
    08/25/2007  03:51 PM           437,170 Alfonso_in_primo_piano.JPG
    10/19/2009  03:00 PM           328,463 Alfonso_in_primo_piano.JPG.paq8px
    10/19/2009  02:41 PM           378,216 Alfonso_in_primo_piano.JPG.zpaq1
    10/19/2009  02:49 PM           355,727 Alfonso_in_primo_piano.JPG.zpaq2
    08/25/2007  04:08 PM           262,135 ART_Fusion_13.JPG
    10/19/2009  03:01 PM           205,806 ART_Fusion_13.JPG.paq8px
    10/19/2009  02:41 PM           235,271 ART_Fusion_13.JPG.zpaq1
    10/19/2009  02:50 PM           225,450 ART_Fusion_13.JPG.zpaq2
    08/25/2007  04:07 PM           229,862 ART_Fusion_62.JPG
    10/19/2009  03:01 PM           177,545 ART_Fusion_62.JPG.paq8px
    10/19/2009  02:41 PM           204,539 ART_Fusion_62.JPG.zpaq1
    10/19/2009  02:50 PM           196,312 ART_Fusion_62.JPG.zpaq2
    08/25/2007  04:04 PM            24,369 Attachment60639.jpg
    10/19/2009  03:01 PM            19,552 Attachment60639.jpg.paq8px
    10/19/2009  02:41 PM            22,973 Attachment60639.jpg.zpaq1
    10/19/2009  02:50 PM            22,904 Attachment60639.jpg.zpaq2
    08/25/2007  04:04 PM            25,507 Attachment76429.jpg
    10/19/2009  03:01 PM            20,015 Attachment76429.jpg.paq8px
    10/19/2009  02:41 PM            24,237 Attachment76429.jpg.zpaq1
    10/19/2009  02:50 PM            23,900 Attachment76429.jpg.zpaq2
    08/25/2007  04:01 PM           317,386 august_18_2007_enso_karate_new_buffalo_beach_training_102.JPG
    10/19/2009  03:01 PM           255,785 august_18_2007_enso_karate_new_buffalo_beach_training_102.JPG.paq8px
    08/25/2007  03:51 PM            69,720 blogbanner.JPG
    10/19/2009  03:01 PM            53,656 blogbanner.JPG.paq8px
    10/19/2009  02:41 PM            61,174 blogbanner.JPG.zpaq1
    10/19/2009  02:50 PM            57,957 blogbanner.JPG.zpaq2
    08/25/2007  03:49 PM           310,795 buster_002.jpg
    10/19/2009  03:01 PM           251,134 buster_002.jpg.paq8px
    08/25/2007  03:48 PM           314,548 catnKeys200519.jpg
    10/19/2009  03:02 PM           243,375 catnKeys200519.jpg.paq8px
    08/25/2007  03:49 PM           232,330 catnKeys2005205.jpg
    10/19/2009  03:02 PM           174,622 catnKeys2005205.jpg.paq8px
    10/19/2009  02:41 PM           214,353 catnKeys2005205.jpg.zpaq1
    10/19/2009  02:50 PM           201,067 catnKeys2005205.jpg.zpaq2
    08/25/2007  03:47 PM           306,185 CatShotAccident.jpg
    10/19/2009  03:02 PM           246,723 CatShotAccident.jpg.paq8px
    10/19/2009  02:41 PM           278,183 CatShotAccident.jpg.zpaq1
    10/19/2009  02:50 PM           262,670 CatShotAccident.jpg.zpaq2
    08/25/2007  03:45 PM           121,575 catt_002.jpg
    10/19/2009  03:02 PM            93,760 catt_002.jpg.paq8px
    08/25/2007  04:04 PM            25,320 CAUDEC8F.jpg
    10/19/2009  03:02 PM            20,190 CAUDEC8F.jpg.paq8px
    10/19/2009  02:41 PM            24,409 CAUDEC8F.jpg.zpaq1
    10/19/2009  02:50 PM            24,108 CAUDEC8F.jpg.zpaq2
    08/25/2007  04:05 PM           126,693 chiangrai_068.jpg
    10/19/2009  03:02 PM           100,961 chiangrai_068.jpg.paq8px
    08/25/2007  04:06 PM           120,481 chiangrai_097.jpg
    10/19/2009  03:02 PM            93,553 chiangrai_097.jpg.paq8px
    08/25/2007  03:54 PM            29,556 CIMG04825_f.jpg
    10/19/2009  03:03 PM            20,751 CIMG04825_f.jpg.paq8px
    10/19/2009  02:41 PM            25,415 CIMG04825_f.jpg.zpaq1
    10/19/2009  02:50 PM            24,300 CIMG04825_f.jpg.zpaq2
    08/25/2007  03:54 PM           295,206 CIMG4779.JPG
    10/19/2009  03:03 PM           235,010 CIMG4779.JPG.paq8px
    08/25/2007  04:01 PM           207,380 Cold_Sunset.JPG
    10/19/2009  03:03 PM           159,782 Cold_Sunset.JPG.paq8px
    08/25/2007  03:46 PM            16,845 Copy_253_of_EXIF_Fuji1500_400x315_17K.JPG
    10/19/2009  03:03 PM            13,459 Copy_253_of_EXIF_Fuji1500_400x315_17K.JPG.paq8px
    10/19/2009  02:41 PM            15,706 Copy_253_of_EXIF_Fuji1500_400x315_17K.JPG.zpaq1
    10/19/2009  02:50 PM            16,308 Copy_253_of_EXIF_Fuji1500_400x315_17K.JPG.zpaq2
    08/25/2007  04:03 PM            41,013 co_022.jpg
    10/19/2009  03:03 PM            31,623 co_022.jpg.paq8px
    08/25/2007  04:06 PM           100,298 dad.jpg
    10/19/2009  03:03 PM            74,638 dad.jpg.paq8px
    08/25/2007  04:01 PM           285,292 day_at_the_beach_034.JPG
    10/19/2009  03:03 PM           222,527 day_at_the_beach_034.JPG.paq8px
    10/19/2009  02:41 PM           257,616 day_at_the_beach_034.JPG.zpaq1
    10/19/2009  02:50 PM           241,047 day_at_the_beach_034.JPG.zpaq2
    08/25/2007  03:44 PM            37,440 dfl_0060.jpg
    10/19/2009  03:03 PM            31,515 dfl_0060.jpg.paq8px
    10/19/2009  02:41 PM            35,985 dfl_0060.jpg.zpaq1
    10/19/2009  02:50 PM            35,091 dfl_0060.jpg.zpaq2
    08/25/2007  03:43 PM            59,016 dfl_0505.jpg
    10/19/2009  03:03 PM            50,343 dfl_0505.jpg.paq8px
    10/19/2009  02:42 PM            55,627 dfl_0505.jpg.zpaq1
    10/19/2009  02:50 PM            53,893 dfl_0505.jpg.zpaq2
    08/25/2007  04:05 PM           624,114 DilsbergNeckarsteinach_065.JPG
    10/19/2009  03:04 PM           514,435 DilsbergNeckarsteinach_065.JPG.paq8px
    10/19/2009  02:42 PM           563,391 DilsbergNeckarsteinach_065.JPG.zpaq1
    10/19/2009  02:51 PM           529,904 DilsbergNeckarsteinach_065.JPG.zpaq2
    08/25/2007  04:01 PM           387,635 DSC02753.JPG
    10/19/2009  03:04 PM           315,064 DSC02753.JPG.paq8px
    08/25/2007  04:00 PM           272,931 DSC02837.JPG
    10/19/2009  03:04 PM           222,984 DSC02837.JPG.paq8px
    08/25/2007  04:04 PM            36,819 DSC06086.jpg
    10/19/2009  03:04 PM            28,830 DSC06086.jpg.paq8px
    10/19/2009  02:42 PM            32,692 DSC06086.jpg.zpaq1
    10/19/2009  02:51 PM            32,591 DSC06086.jpg.zpaq2
    08/25/2007  03:51 PM           134,143 DSCF1348.jpg
    10/19/2009  03:04 PM           110,607 DSCF1348.jpg.paq8px
    10/19/2009  02:42 PM           124,180 DSCF1348.jpg.zpaq1
    10/19/2009  02:51 PM           118,629 DSCF1348.jpg.zpaq2
    08/25/2007  04:00 PM            46,673 DSCF1980.jpg
    10/19/2009  03:04 PM            39,182 DSCF1980.jpg.paq8px
    10/19/2009  02:42 PM            43,724 DSCF1980.jpg.zpaq1
    10/19/2009  02:51 PM            42,173 DSCF1980.jpg.zpaq2
    08/25/2007  03:58 PM           710,719 DSCN0230.JPG
    10/19/2009  03:05 PM           564,387 DSCN0230.JPG.paq8px
    10/19/2009  02:42 PM           628,455 DSCN0230.JPG.zpaq1
    10/19/2009  02:51 PM           588,323 DSCN0230.JPG.zpaq2
    08/25/2007  04:08 PM           353,399 DSCN0400.JPG
    10/19/2009  03:05 PM           285,487 DSCN0400.JPG.paq8px
    08/25/2007  03:55 PM           819,747 DSCN1654.JPG
    10/19/2009  03:06 PM           618,875 DSCN1654.JPG.paq8px
    10/19/2009  02:42 PM           736,550 DSCN1654.JPG.zpaq1
    10/19/2009  02:52 PM           682,288 DSCN1654.JPG.zpaq2
    08/25/2007  03:57 PM            46,556 DSCN1681.JPG
    10/19/2009  03:06 PM            38,941 DSCN1681.JPG.paq8px
    10/19/2009  02:43 PM            44,254 DSCN1681.JPG.zpaq1
    10/19/2009  02:52 PM            42,236 DSCN1681.JPG.zpaq2
    08/25/2007  04:04 PM           576,480 DSCN1937.JPG
    10/19/2009  03:06 PM           416,984 DSCN1937.JPG.paq8px
    08/25/2007  03:57 PM            46,182 DSCN1948.JPG
    10/19/2009  03:06 PM            37,957 DSCN1948.JPG.paq8px
    10/19/2009  02:43 PM            42,948 DSCN1948.JPG.zpaq1
    10/19/2009  02:52 PM            41,267 DSCN1948.JPG.zpaq2
    08/25/2007  03:57 PM           891,520 DSCN2865.JPG
    10/19/2009  03:07 PM           674,004 DSCN2865.JPG.paq8px
    08/25/2007  04:05 PM           231,252 DSC_0020.JPG
    10/19/2009  03:07 PM           185,419 DSC_0020.JPG.paq8px
    08/25/2007  04:06 PM           143,762 DSC_1825.jpg
    10/19/2009  03:07 PM           104,474 DSC_1825.jpg.paq8px
    08/25/2007  03:48 PM           666,543 DSC_3717.jpg
    10/19/2009  03:08 PM           553,880 DSC_3717.jpg.paq8px
    10/19/2009  02:43 PM           602,551 DSC_3717.jpg.zpaq1
    10/19/2009  02:52 PM           573,262 DSC_3717.jpg.zpaq2
    08/25/2007  04:07 PM           152,060 EZEGNAX21.jpg
    10/19/2009  03:08 PM           144,950 EZEGNAX21.jpg.paq8px
    08/25/2007  03:51 PM           222,128 grey_002.jpg
    10/19/2009  03:08 PM           176,579 grey_002.jpg.paq8px
    08/25/2007  04:07 PM         1,369,536 Hassam_GalleryPlayer.jpg
    10/19/2009  03:09 PM         1,187,962 Hassam_GalleryPlayer.jpg.paq8px
    08/25/2007  04:01 PM           218,834 HPIM2867.JPG
    10/19/2009  03:09 PM           169,804 HPIM2867.JPG.paq8px
    08/25/2007  04:08 PM            33,726 ILC_056.jpg
    10/19/2009  03:09 PM            25,756 ILC_056.jpg.paq8px
    08/25/2007  04:08 PM            40,811 ILC_072.jpg
    10/19/2009  03:10 PM            30,481 ILC_072.jpg.paq8px
    08/25/2007  03:54 PM           124,864 Image00006.jpg
    10/19/2009  03:10 PM            81,835 Image00006.jpg.paq8px
    08/25/2007  04:07 PM            45,322 image005.jpg
    10/19/2009  03:10 PM            37,459 image005.jpg.paq8px
    10/19/2009  02:43 PM            42,625 image005.jpg.zpaq1
    10/19/2009  02:52 PM            41,343 image005.jpg.zpaq2
    08/25/2007  03:52 PM         2,230,194 IMG_0010.JPG
    10/19/2009  03:11 PM         1,683,525 IMG_0010.JPG.paq8px
    08/25/2007  04:02 PM           212,403 IMG_0189.JPG
    10/19/2009  03:11 PM           168,631 IMG_0189.JPG.paq8px
    10/19/2009  02:43 PM           197,069 IMG_0189.JPG.zpaq1
    10/19/2009  02:52 PM           187,237 IMG_0189.JPG.zpaq2
    08/25/2007  04:03 PM            44,643 IMG_0349.JPG
    10/19/2009  03:11 PM            37,010 IMG_0349.JPG.paq8px
    10/19/2009  02:43 PM            42,796 IMG_0349.JPG.zpaq1
    10/19/2009  02:53 PM            41,276 IMG_0349.JPG.zpaq2
    08/25/2007  04:05 PM           288,511 IMG_1296.JPG
    10/19/2009  03:12 PM           225,612 IMG_1296.JPG.paq8px
    10/19/2009  02:43 PM           259,762 IMG_1296.JPG.zpaq1
    10/19/2009  02:53 PM           247,793 IMG_1296.JPG.zpaq2
    08/25/2007  03:45 PM            22,216 IMG_2549.jpg
    10/19/2009  03:12 PM            18,349 IMG_2549.jpg.paq8px
    10/19/2009  02:43 PM            21,659 IMG_2549.jpg.zpaq1
    10/19/2009  02:53 PM            21,287 IMG_2549.jpg.zpaq2
    08/25/2007  03:58 PM           139,946 IMG_3766.JPG
    10/19/2009  03:12 PM           113,355 IMG_3766.JPG.paq8px
    10/19/2009  02:43 PM           127,832 IMG_3766.JPG.zpaq1
    10/19/2009  02:53 PM           121,987 IMG_3766.JPG.zpaq2
    08/25/2007  03:51 PM           239,908 IMG_3969.JPG
    10/19/2009  03:12 PM           189,964 IMG_3969.JPG.paq8px
    08/25/2007  04:03 PM            82,320 JamaicaCraft_04.jpg
    10/19/2009  03:12 PM            67,984 JamaicaCraft_04.jpg.paq8px
    10/19/2009  02:43 PM            76,252 JamaicaCraft_04.jpg.zpaq1
    10/19/2009  02:53 PM            73,404 JamaicaCraft_04.jpg.zpaq2
    08/25/2007  04:05 PM            35,833 jugs424px.jpg
    10/19/2009  03:12 PM            28,976 jugs424px.jpg.paq8px
    10/19/2009  02:43 PM            35,232 jugs424px.jpg.zpaq1
    10/19/2009  02:53 PM            32,761 jugs424px.jpg.zpaq2
    08/25/2007  03:48 PM           199,101 kittycat_015.JPG
    10/19/2009  03:12 PM           159,608 kittycat_015.JPG.paq8px
    10/19/2009  02:44 PM           180,039 kittycat_015.JPG.zpaq1
    10/19/2009  02:53 PM           172,243 kittycat_015.JPG.zpaq2
    08/25/2007  04:05 PM           207,893 kohphangan4_014.jpg
    10/19/2009  03:12 PM           170,150 kohphangan4_014.jpg.paq8px
    08/25/2007  04:05 PM           105,874 kohphangan_missingshots_029.jpg
    10/19/2009  03:12 PM            81,357 kohphangan_missingshots_029.jpg.paq8px
    08/25/2007  04:06 PM           119,408 kohphangan_missingshots_033.jpg
    10/19/2009  03:13 PM            92,913 kohphangan_missingshots_033.jpg.paq8px
    08/25/2007  04:01 PM         2,065,498 March_2007_014.jpg
    10/19/2009  03:14 PM         1,673,043 March_2007_014.jpg.paq8px
    08/25/2007  04:01 PM            19,571 Marian_skotti.jpg
    10/19/2009  03:14 PM            15,403 Marian_skotti.jpg.paq8px
    10/19/2009  02:44 PM            18,751 Marian_skotti.jpg.zpaq1
    10/19/2009  02:53 PM            17,898 Marian_skotti.jpg.zpaq2
    08/25/2007  03:54 PM           374,250 mountains_july_2007_022.JPG
    10/19/2009  03:14 PM           303,937 mountains_july_2007_022.JPG.paq8px
    08/25/2007  03:46 PM            41,114 Mu_007.jpg
    10/19/2009  03:14 PM            31,768 Mu_007.jpg.paq8px
    08/25/2007  04:06 PM           239,129 Neridas_21st_015.jpg
    10/19/2009  03:14 PM           195,304 Neridas_21st_015.jpg.paq8px
    08/25/2007  04:07 PM           172,078 Newtown_street_art_2.jpg
    10/19/2009  03:14 PM           125,609 Newtown_street_art_2.jpg.paq8px
    10/19/2009  02:44 PM           154,540 Newtown_street_art_2.jpg.zpaq1
    10/19/2009  02:53 PM           136,108 Newtown_street_art_2.jpg.zpaq2
    08/25/2007  04:05 PM           123,938 newyork2007_15.jpg
    10/19/2009  03:15 PM            98,128 newyork2007_15.jpg.paq8px
    08/25/2007  04:00 PM           359,270 OB_62607_019a.jpg
    10/19/2009  03:15 PM           290,547 OB_62607_019a.jpg.paq8px
    10/19/2009  02:44 PM           325,536 OB_62607_019a.jpg.zpaq1
    10/19/2009  02:53 PM           307,037 OB_62607_019a.jpg.zpaq2
    08/25/2007  04:06 PM           162,467 Overworked.jpg
    10/19/2009  03:15 PM           119,451 Overworked.jpg.paq8px
    08/25/2007  04:01 PM            35,519 P1090197.JPG
    10/19/2009  03:15 PM            29,422 P1090197.JPG.paq8px
    10/19/2009  02:44 PM            34,167 P1090197.JPG.zpaq1
    10/19/2009  02:53 PM            32,385 P1090197.JPG.zpaq2
    08/25/2007  03:56 PM           182,985 PICT0947.JPG
    10/19/2009  03:15 PM           142,534 PICT0947.JPG.paq8px
    08/25/2007  03:58 PM           912,241 Reflection_GalleryPlayer.jpg
    10/19/2009  03:16 PM           720,418 Reflection_GalleryPlayer.jpg.paq8px
    08/25/2007  04:07 PM           124,076 Scene_from_the_Book_of_Portals_in_the_Tomb_of_Ramesses_I.jpg
    10/19/2009  03:16 PM            99,982 Scene_from_the_Book_of_Portals_in_the_Tomb_of_Ramesses_I.jpg.paq8px
    10/19/2009  02:44 PM           115,396 Scene_from_the_Book_of_Portals_in_the_Tomb_of_Ramesses_I.jpg.zpaq1
    10/19/2009  02:53 PM           110,654 Scene_from_the_Book_of_Portals_in_the_Tomb_of_Ramesses_I.jpg.zpaq2
    08/25/2007  03:46 PM           125,075 shadz_017.jpg
    10/19/2009  03:16 PM            94,594 shadz_017.jpg.paq8px
    08/25/2007  04:06 PM            11,927 ShowLetterj.jpg
    10/19/2009  03:16 PM             9,547 ShowLetterj.jpg.paq8px
    10/19/2009  02:44 PM            11,909 ShowLetterj.jpg.zpaq1
    10/19/2009  02:53 PM            12,168 ShowLetterj.jpg.zpaq2
    08/25/2007  03:53 PM            20,317 SUN04MB039001[2].jpg
    10/19/2009  03:16 PM            16,022 SUN04MB039001[2].jpg.paq8px
    10/19/2009  02:44 PM            19,534 SUN04MB039001[2].jpg.zpaq1
    10/19/2009  02:53 PM            18,966 SUN04MB039001[2].jpg.zpaq2
    08/25/2007  04:03 PM            42,781 Sunset_16.jpg
    10/19/2009  03:16 PM            33,764 Sunset_16.jpg.paq8px
    10/19/2009  02:44 PM            41,466 Sunset_16.jpg.zpaq1
    10/19/2009  02:54 PM            37,664 Sunset_16.jpg.zpaq2
    08/25/2007  03:57 PM            42,926 Terry_disturbing_tthe_peace_on_Beinn_na_Lap.JPG
    10/19/2009  03:16 PM            35,314 Terry_disturbing_tthe_peace_on_Beinn_na_Lap.JPG.paq8px
    10/19/2009  02:44 PM            39,973 Terry_disturbing_tthe_peace_on_Beinn_na_Lap.JPG.zpaq1
    10/19/2009  02:54 PM            39,012 Terry_disturbing_tthe_peace_on_Beinn_na_Lap.JPG.zpaq2
    08/25/2007  04:08 PM             2,852 The_Letter.jpg
    10/19/2009  03:16 PM             2,579 The_Letter.jpg.paq8px
    10/19/2009  02:44 PM             3,783 The_Letter.jpg.zpaq1
    10/19/2009  02:54 PM             4,293 The_Letter.jpg.zpaq2
    08/25/2007  04:05 PM           185,938 victoria_veckham_shirt_sheer_5_big.jpg
    10/19/2009  03:16 PM           120,415 victoria_veckham_shirt_sheer_5_big.jpg.paq8px
    08/25/2007  03:47 PM           824,722 Wildwood_200615.JPG
    10/19/2009  03:17 PM           642,257 Wildwood_200615.JPG.paq8px
    08/25/2007  04:07 PM            18,167 wolk23.jpg
    10/19/2009  03:17 PM            14,652 wolk23.jpg.paq8px
    10/19/2009  02:44 PM            17,746 wolk23.jpg.zpaq1
    10/19/2009  02:54 PM            17,277 wolk23.jpg.zpaq2
    08/25/2007  03:51 PM           426,089 XXXA4I0020.JPG
    10/19/2009  03:17 PM           266,366 XXXA4I0020.JPG.paq8px
                 306 File(s)     61,918,306 bytes

  18. #78
    Programmer Jan Ondrus's Avatar
    Join Date
    Sep 2008
    Location
    Rychnov nad Kněžnou, Czech Republic
    Posts
    279
    Thanks
    33
    Thanked 138 Times in 50 Posts
    Fixed another bug in reading header (APP and COM field were not properly skipped).
    Attached Files Attached Files

  19. #79
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    613
    Thanks
    252
    Thanked 241 Times in 120 Posts
    Quote Originally Posted by Jan Ondrus View Post
    Fixed another bug in reading header (APP and COM field were not properly skipped).
    I tried the fixed config files on the 47 files that failed before with ZPAQ v1.07. There are only 7 files left that still fail the preprocessing test, 2 of them (4215626041.jpg and EZEGNAX21.jpg) even gave a Windows "send bug report" dialog.

    As a sidenote, a little ZPAQ usability request: It would be nice to have the size of the preprocessed file displayed, too (at the moment only the size of the original file is shown).

    Results for jpg_test1.cfg:
    Code:
    20.10.2009  17:42           105.347 070730_Arcachon_22.jpg.zpaq1
    20.10.2009  17:42            79.447 070806_Main_06.jpg.zpaq1
    20.10.2009  17:45           532.444 110_1063.JPG.zpaq1
    20.10.2009  17:46           143.711 AkhaSwingFest_SIMANN_10.JPG.zpaq1
    20.10.2009  17:46            76.092 AkhaSwingFest_SIMANN_9.JPG.zpaq1
    20.10.2009  17:47           284.295 august_18_2007_enso_karate_new_buffalo_beach_training_102.JPG.zpaq1
    20.10.2009  17:49           284.107 buster_002.jpg.zpaq1
    20.10.2009  17:50           105.291 catt_002.jpg.zpaq1
    20.10.2009  17:51           111.987 chiangrai_068.jpg.zpaq1
    20.10.2009  17:51           107.116 chiangrai_097.jpg.zpaq1
    20.10.2009  17:53           266.075 CIMG4779.JPG.zpaq1
    20.10.2009  17:53           186.254 Cold_Sunset.JPG.zpaq1
    20.10.2009  17:54            38.000 co_022.jpg.zpaq1
    20.10.2009  17:54            87.531 dad.jpg.zpaq1
    20.10.2009  17:56           355.686 DSC02753.JPG.zpaq1
    20.10.2009  17:57           248.303 DSC02837.JPG.zpaq1
    20.10.2009  17:59           321.909 DSCN0400.JPG.zpaq1
    20.10.2009  18:02           513.078 DSCN1937.JPG.zpaq1
    20.10.2009  18:07           806.967 DSCN2865.JPG.zpaq1
    20.10.2009  18:08           210.743 DSC_0020.JPG.zpaq1
    20.10.2009  18:10           204.674 grey_002.jpg.zpaq1
    20.10.2009  18:15           194.497 HPIM2867.JPG.zpaq1
    20.10.2009  18:15            30.813 ILC_056.jpg.zpaq1
    20.10.2009  18:16            36.062 ILC_072.jpg.zpaq1
    20.10.2009  18:28         1.971.935 IMG_0010.JPG.zpaq1
    20.10.2009  18:29           216.766 IMG_3969.JPG.zpaq1
    20.10.2009  18:30           188.736 kohphangan4_014.jpg.zpaq1
    20.10.2009  18:31            94.012 kohphangan_missingshots_029.jpg.zpaq1
    20.10.2009  18:32           105.535 kohphangan_missingshots_033.jpg.zpaq1
    20.10.2009  18:43         1.832.378 March_2007_014.jpg.zpaq1
    20.10.2009  18:46           333.925 mountains_july_2007_022.JPG.zpaq1
    20.10.2009  18:46            36.955 Mu_007.jpg.zpaq1
    20.10.2009  18:47           219.911 Neridas_21st_015.jpg.zpaq1
    20.10.2009  18:48           110.558 newyork2007_15.jpg.zpaq1
    20.10.2009  18:48           137.271 Overworked.jpg.zpaq1
    20.10.2009  18:50           161.425 PICT0947.JPG.zpaq1
    20.10.2009  18:54           792.275 Reflection_GalleryPlayer.jpg.zpaq1
    20.10.2009  18:54           106.280 shadz_017.jpg.zpaq1
    20.10.2009  18:59           755.370 Wildwood_200615.JPG.zpaq1
    20.10.2009  19:02           352.116 XXXA4I0020.JPG.zpaq1
    
    Total size: 12.745.877 bytes
    Results for jpg_test2.cfg:

    Code:
    20.10.2009  17:40           100.029 070730_Arcachon_22.jpg.zpaq2
    20.10.2009  17:41            76.653 070806_Main_06.jpg.zpaq2
    20.10.2009  16:00           498.477 110_1063.JPG.zpaq2
    20.10.2009  16:03           136.365 AkhaSwingFest_SIMANN_10.JPG.zpaq2
    20.10.2009  16:04            73.607 AkhaSwingFest_SIMANN_9.JPG.zpaq2
    20.10.2009  16:05           268.815 august_18_2007_enso_karate_new_buffalo_beach_training_102.JPG.zpaq2
    20.10.2009  16:07           273.048 buster_002.jpg.zpaq2
    20.10.2009  16:09            98.768 catt_002.jpg.zpaq2
    20.10.2009  16:09           108.730 chiangrai_068.jpg.zpaq2
    20.10.2009  16:10           103.817 chiangrai_097.jpg.zpaq2
    20.10.2009  16:12           250.471 CIMG4779.JPG.zpaq2
    20.10.2009  16:13           175.720 Cold_Sunset.JPG.zpaq2
    20.10.2009  16:14            35.616 co_022.jpg.zpaq2
    20.10.2009  16:14            85.514 dad.jpg.zpaq2
    20.10.2009  16:16           333.052 DSC02753.JPG.zpaq2
    20.10.2009  16:18           237.455 DSC02837.JPG.zpaq2
    20.10.2009  16:19           308.017 DSCN0400.JPG.zpaq2
    20.10.2009  16:22           488.314 DSCN1937.JPG.zpaq2
    20.10.2009  16:27           758.052 DSCN2865.JPG.zpaq2
    20.10.2009  16:28           202.709 DSC_0020.JPG.zpaq2
    20.10.2009  16:30           195.194 grey_002.jpg.zpaq2
    20.10.2009  16:36           184.023 HPIM2867.JPG.zpaq2
    20.10.2009  16:36            29.855 ILC_056.jpg.zpaq2
    20.10.2009  16:36            34.496 ILC_072.jpg.zpaq2
    20.10.2009  16:50         1.851.151 IMG_0010.JPG.zpaq2
    20.10.2009  16:52           205.369 IMG_3969.JPG.zpaq2
    20.10.2009  16:53           180.864 kohphangan4_014.jpg.zpaq2
    20.10.2009  16:54            90.045 kohphangan_missingshots_029.jpg.zpaq2
    20.10.2009  16:54           102.129 kohphangan_missingshots_033.jpg.zpaq2
    20.10.2009  17:08         1.727.077 March_2007_014.jpg.zpaq2
    20.10.2009  17:10           317.865 mountains_july_2007_022.JPG.zpaq2
    20.10.2009  17:11            35.585 Mu_007.jpg.zpaq2
    20.10.2009  17:12           210.072 Neridas_21st_015.jpg.zpaq2
    20.10.2009  17:13           106.886 newyork2007_15.jpg.zpaq2
    20.10.2009  17:16           142.404 Overworked.jpg.zpaq2
    20.10.2009  17:19           156.408 PICT0947.JPG.zpaq2
    20.10.2009  17:30           774.184 Reflection_GalleryPlayer.jpg.zpaq2
    20.10.2009  17:30            99.059 shadz_017.jpg.zpaq2
    20.10.2009  17:36           691.133 Wildwood_200615.JPG.zpaq2
    20.10.2009  17:38           332.643 XXXA4I0020.JPG.zpaq2
    
    Total size: 12.079.671 bytes
    The 7 files that failed are:

    Code:
    4215626041.jpg
    catnKeys200519.jpg
    DSC_1825.jpg
    EZEGNAX21.jpg
    Hassam_GalleryPlayer.jpg
    Image00006.jpg
    victoria_veckham_shirt_sheer_5_big.jpg
    Last edited by schnaader; 20th October 2009 at 20:15.
    http://schnaader.info
    Damn kids. They're all alike.

  20. #80
    Programmer Jan Ondrus's Avatar
    Join Date
    Sep 2008
    Location
    Rychnov nad Kněžnou, Czech Republic
    Posts
    279
    Thanks
    33
    Thanked 138 Times in 50 Posts
    Quote Originally Posted by schnaader View Post
    The 7 files that failed are:

    Code:
    4215626041.jpg
    catnKeys200519.jpg
    DSC_1825.jpg
    EZEGNAX21.jpg
    Hassam_GalleryPlayer.jpg
    Image00006.jpg
    victoria_veckham_shirt_sheer_5_big.jpg
    1.)
    - 4215626041.jpg
    - EZEGNAX21.jpg
    are JPEG progressive which is not supported. I fixed preprocessor so that it won't crash on this files but preprocessing will fail.
    2.)
    - catnKeys200519.jpg
    - DSC_1825.jpg
    - Hassam_GalleryPlayer.jpg
    - Image00006.jpg
    - victoria_veckham_shirt_sheer_5_big.jpg
    failed due to bug : missing stuffed byte before RST code wasn't restored by ZPAQL code.
    Attached Files Attached Files

  21. #81
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    Some test results on the Picasa set. For comparison purposes I added the uncompressed sizes of the 2 progressive mode JPEGs that jpg_test1/2 won't compress.

    Code:
    25,778,926 picasa original size of 100 files
    25,559,350 stuffit 11 fast (18 sec)
    25,415,338 zip -9 (3 sec)
    23,114,270 zpaq ocjpg_test1.cfg (600 sec, fails on 4215626041.jpg, EZEGNAX21.jpg)
    21,903,634 zpaq ocjpg_test2.cfg (840 sec, fails on 4215626041.jpg, EZEGNAX21.jpg)
    20,951,499 packjpg 2.2 (62 sec)
    20,627,622 paq8o -6 (916 sec)
    20,438,131 paq8fthis3 -6 (738 sec)
    20,292,713 paq8o6 -6 (1080 sec)
    20,263,436 paq8o7 -6 (1089 sec)
    20,244,413 paq8o8 -6 (1160 sec)
    20,207,952 paq8px_v64 -6 (1140 sec)
    19,819,783 stuffit 11 max (50 sec)
    19,828,626 stuffit 12 --jpeg-no-thumbnails (58 sec)
    In other news I'm working on a Linux/Mac port for zpaq 1.09. It mostly works as-is on a PowerPC except that Linux doesn't like "$" in the temporary file names so preprocessing fails. I'm also fixing a bunch of minor bugs mostly in the display and cleaning up compiler warnings.

  22. #82
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    zpaq 1.09 is posted to http://mattmahoney.net/dc/

    Mainly this is a Linux port and some mostly cosmetic bug fixes. The port involved changing the temporary file names (no $), flushing stdout when showing compression progress, and fixing a crash when neither $TEMP nor $TMPDIR is set. $TMPDIR is no longer used. All temporary files including preprocessing now go in %TEMP% in Windows or $TEMP in Linux. If you want them in /tmp then

    setenv TEMP /tmp

    or else they go in the current directory. %TEMP% is normally set in Windows to a temp directory.

    Other changes:

    q (quiet) option - no display during compression or decompression except for errors.

    Compression progress shows preprocessed size if different, so you know when it will be finished.

    Display is updated once per second regardless of speed.

    Program no longer uses fseek() and ftell() anywhere, so file sizes should display correctly over 2 GB (not tested).

    Optimized decompressers check the block header and don't assume it matches the compiled ZPAQL code.

    Documentation in zpaq.cpp is updated to reflect these changes.

    BTW, nice job on moving the forum to a reliable website.

  23. #83
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    http://mattmahoney.net/dc/

    I posted Jan Ondrus's JPEG model and added a few test results for his other models. The BMP model would be ranked #1 on maximumcompression.com. Too bad that site appears dead Also reorganized a bit and made zpaqsfx a separate download.

    I also tested BWT (fast mode) with and without optimization on enwik9. Times below are with optimization (oc, ox). bwt_j2.cfg,18 without optimization (c, x) is 5593, 4347. Optimizing makes decompression 2.35 times faster.

    Code:
      Config                          enwik8      enwik9       oc    ox   Mem
      ------                        ----------  -----------   ----  ----  ----
      mid.cfg                       20,941,558   180,279,221  3521  3652   111
      bwt_j2.cfg,18                 20,756,883   174,171,965  4529  1847  1838  (256 MB block)
      bwt_slowmodel_1GB_block.cfg   20,756,996   163,565,006  7957  3875  1443  (1 GB block)
    More testing on zpaq 1.09 with large files (3 GB, 5 GB) shows everything works, but lzppre (preprocessor for min.cfg) fails above 4 GB. zpaq detects this and refuses to compress, just like it is supposed to. Haven't fixed yet.

  24. #84
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    zpaq v1.10 is posted to http://mattmahoney.net/dc/

    This is a minor bug fix for g++ 4.4.1. v1.09 failed during compression if a preprocessor was used (attempting to read from a closed input file). Otherwise there is no change in speed or compression or in any of the commands. The code in v1.09 should not have worked but apparently did anyway (in 3 different compilers). Another user found the bug when testing in Ubuntu and sent me a patch.

  25. #85
    Member
    Join Date
    Mar 2009
    Location
    Prague, CZ
    Posts
    62
    Thanks
    32
    Thanked 7 Times in 7 Posts
    Hi,
    thank you, nice archiver, or at least promising - especially considering the future archive compatibility. Do you plan to release a version that will be able to use more than 2GB RAM ? It seems current config files can be set so, but not the 32bit program. I think that having more than 2GB RAM is not / or at least will not be / so rare anymore, and so the archiver and users can gain smaller files easily sometimes.
    Well, in general, I'm surprised how few archivers support more than 2GB RAM, good exception beeing 7-zip (and even 7-zip only for lzma algo, not for ppm), nanozip has a bug, ash is experimental, 64bit WinRK doesn't allow to go beyond 2GB model either, ... But I'm not a good/active programmer, so maybe I don't uderstand it just because I don't know that changing the archiver or algorythm in such a way is not easy at all. Is this the reason?
    regards
    Michal

  26. #86
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    You should be able to compile ZPAQ for 64 bits and write models that use more than 2 GB memory although each component is limited to 1 GB in most cases. This is a limitation of using 32 bit arithmetic for memory allocation and computing array indexes.

    Normally most memory would be used by the ICM and ISSE components. These map a context to a 1-byte bit history using hash tables indexed by by a 32 bit context hash computed every 4 bits of input. Each table entry is an array of 15 bit histories (for up to 3 bits of additional context) and an 8 bit checksum. The context hash is computed as a 32 bit number in the HCOMP section of the ZPAQL code. The low bits (up to 26 for 1 GB) index the table and the high 8 bits are used as a hash confirmation. This means for the largest size table there is a 2 bit overlap so there is an effective 6 bit checksum which means 1/64 chance of an undetected hash collision for each of the 3 entries in the hash search. Using larger tables would mean more false matches which would hurt compression.

    A CM does not have this problem because it is indexed directly and hash collisions (always undetected) have a much smaller effect. It also has a 32 bit index and 4 bytes per entry so in theory it could be as large as 16 GB. However, using a large CM table for high order contexts usually gives worse compression than an ICM. A CM is better suited for low orders for stationary sources.

  27. #87
    Member
    Join Date
    Feb 2010
    Location
    Nordic
    Posts
    200
    Thanks
    41
    Thanked 36 Times in 12 Posts
    Quote Originally Posted by Matt Mahoney View Post
    a crash when neither $TEMP nor $TMPDIR is set.
    tmpfile() is the appropriate function and it works on both Windows and Linux.

  28. #88
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    Actually tmpfile() fails in Windows Vista/g++ on the C: drive if you don't have administrator privileges. This is a problem in all recent PAQ versions and BBB in slow mode. tmpfile() tries to create a temporary file at the root of the current drive. I have to go to D: to run PAQ. tmpfile() works under XP. I haven't tried in Win7.

    tmpnam() has the same problem, since it generates a file name starting with \

    ZPAQ would be a lot simpler if I could use tmpfile() instead of writing my own code to work around a bug in g++. It is hard to make this thread safe (so I didn't).

  29. #89
    Member
    Join Date
    Feb 2010
    Location
    Nordic
    Posts
    200
    Thanks
    41
    Thanked 36 Times in 12 Posts
    crikey I didn't know that;

    GetTempFileName() might be better, you can tell it to do it in the current directory

    still, it shocks me that tmpfile() doesn't work on vista, but it seems you're all over it

  30. #90
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    Yes, that would work but I would like to avoid OS dependency since it is Windows only. It would be better to not use temporary files at all. In zpaq I use it to check that running the input through the preprocessor and postprocessor restores the original file by comparing the SHA-1 checksum. This could be done in 2 passes without a temp file, or in 1 pass but you wouldn't find out until the file was already compressed, or could be skipped entirely if you trust that the code is correct. (You would find out when you decompressed and got a checksum error).

Page 3 of 85 FirstFirst 123451353 ... LastLast

Similar Threads

  1. ZPAQ self extracting archives
    By Matt Mahoney in forum Data Compression
    Replies: 31
    Last Post: 17th April 2014, 03:39
  2. ZPAQ 1.05 preview
    By Matt Mahoney in forum Data Compression
    Replies: 11
    Last Post: 30th September 2009, 04:26
  3. zpaq 1.02 update
    By Matt Mahoney in forum Data Compression
    Replies: 11
    Last Post: 10th July 2009, 00:55
  4. Metacompressor.com benchmark updates
    By Sportman in forum Data Compression
    Replies: 79
    Last Post: 22nd April 2009, 03:24
  5. ZPAQ pre-release
    By Matt Mahoney in forum Data Compression
    Replies: 54
    Last Post: 23rd March 2009, 02:17

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •