Results 1 to 4 of 4

Thread: the zero redunduncy in zpaq normal jpg compression

  1. #1
    Member toi007's Avatar
    Join Date
    Jun 2011
    Location
    Lisbon
    Posts
    35
    Thanks
    0
    Thanked 0 Times in 0 Posts

    the zero redunduncy in zpaq normal jpg compression

    Hi people this is just a theory....
    if i compress zpaq -m4 c 001.jpg.zpaq 001.jpg i get like > 1194181 bytes
    if i use a relaxor of bits puting a zero bit between each bit of jpg i get zpaq -m4 c 001.jpg.dat.zpaq 001.jpg.dat > 1190184 bytes
    its like 4 kb better in 1 megabyte
    heres some samples and the relaxor (put-one-zero)
    http://www.jpg2bmp.com/JPEG/put-one-zero.zip

    it contains 10 samples the blitz basic fast program to convert to dat file and vise versa
    and the zpaq to compress the dat files

    its nothing relevant so its off topic

  2. #2
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 795 Times in 488 Posts
    Actually I think this is on topic. I did some more experiments with 001.jpg. I wrote a program to split each byte into groups of 4, 2, or 1 bits in MSB to LSB order.

    Code:
    1,248,679 001.jpg (original)
    2,497,366 001.jpg.dat (interleaved bits using put-one-zero.exe)
    2,497,358 out4 (4 bits per byte in high, low nibble order)
    4,994,716 out2 (2 bits per byte)
    9,989,432 out1 (1 bit per byte)
    
    // Code to create out4, out2, out1 from 001.jpg
    #include <stdio.h>
    int main() {
      FILE* in=fopen("001.jpg", "rb");
      FILE* out4=fopen("out4", "wb");
      FILE* out2=fopen("out2", "wb");
      FILE* out1=fopen("out1", "wb");
      int c;
      while ((c=getc(in))!=EOF) {
        fprintf(out4, "%c%c", c>>4, c&15);
        fprintf(out2, "%c%c%c%c", c>>6&3, c>>4&3, c>>2&3, c&3);
        fprintf(out1, "%c%c%c%c%c%c%c%c", c>>7&1, c>>6&1, c>>5&1, c>>4&1,
                                          c>>3&1, c>>2&1, c>>1&1, c&1);
      }
      return 0;
    }
    Then I compressed with several programs and ordered by size:

    Code:
    zpaq -m1: original file compresses best.
    1,210,940 001.jpg-m1.zpaq
    1,224,279 out4-m1.zpaq
    1,225,548 001.jpg.dat-m1.zpaq
    1,320,724 out2-m1.zpaq
    1,608,673 out1-m1.zpaq
    
    zpaq -m2: 1 bit per byte compresses best.
    1,188,265 out1-m2.zpaq
    1,194,348 out2-m2.zpaq
    1,200,993 out4-m2.zpaq
    1,201,008 001.jpg.dat-m2.zpaq
    1,209,990 001.jpg-m2.zpaq
    
    zpaq -m3: bits interleaved with 0 compresses best.
    1,190,750 001.jpg.dat-m3.zpaq
    1,190,815 out4-m3.zpaq
    1,202,213 001.jpg-m3.zpaq
    1,225,491 out2-m3.zpaq
    1,241,802 out1-m3.zpaq
    
    zpaq -m4: bits interleaved with 0 compresses best.
    1,190,185 001.jpg.dat-m4.zpaq
    1,190,527 out4-m4.zpaq
    1,194,181 001.jpg-m4.zpaq
    1,206,570 out2-m4.zpaq
    1,232,146 out1-m4.zpaq
    
    7zip: original compresses best.
    1,235,547 001.jpg.7z
    1,282,511 out4.7z
    1,282,614 out2.7z
    1,282,774 001.jpg.dat.7z
    1,315,042 out1.7z
    
    zip: original compresses best.
    1,234,307 001.jpg.zip
    1,372,361 out4.zip
    1,372,662 001.jpg.dat.zip
    1,435,215 out2.zip
    1,567,542 out1.zip
    
    nanozip: 1 bit per byte compresses best.
    1,199,699 out1.nz
    1,204,839 out2.nz
    1,209,057 001.jpg.dat.nz
    1,209,080 out4.nz
    1,212,267 001.jpg.nz
    
    bsc: 1 bit per byte compresses best.
    1,187,569 out1.bsc
    1,195,593 out2.bsc
    1,200,769 out4.bsc
    1,200,824 001.jpg.dat.bsc
    1,204,831 001.jpg.bsc
    
    ppmonstr: 4 bits per byte compresses best.
    1,198,188 out4.pmm
    1,198,518 001.jpg.dat.pmm
    1,200,243 out2.pmm
    1,204,931 001.jpg.pmm
    1,208,440 out1.pmm
    I think the explanation is that context boundaries are more likely to fall on Huffman code boundaries in the JPEG file. Of course, parsing the Huffman codes would be best. That's what jpg_test2.cfg does.

    992,424 001.jpg-mjpg_test2.zpaq

    BTW the put-one-zero program doesn't restore the file exactly. The last byte is restored as 00 instead of D9.

  3. #3
    Member toi007's Avatar
    Join Date
    Jun 2011
    Location
    Lisbon
    Posts
    35
    Thanks
    0
    Thanked 0 Times in 0 Posts
    its interesting the fact that zpaq -m2 one bit per bite get good results as bsc one bit per bite witch i dont know what it is
    the 2 better benchmarks
    can you make a dos file of your encoder decoder that is the correct version of it mine fails last byte (im not a professional programmer)
    i will try to test in several compressors includind that bsc
    and in several files that i as a designer and photographer addicted to nature use in several jpgs

  4. #4
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 795 Times in 488 Posts
    zpaq -m2 and bsc both use BWT and use a block size large enough to contain the whole input. One bit per byte is theoretically best because Huffman code boundaries always fall on byte boundaries. But that only works for compressors with enough memory.

    These 2 programs should be equivalent to your bit interleave. There's no error checking on the arguments to keep it simple.

    Code:
    // Put a 0 before each bit of input
    #include <stdio.h>
    int main(int argc, char** argv) {
      FILE* in=fopen(argv[1], "rb");
      FILE* out=fopen(argv[2], "wb");
      for (int c; (c=getc(in))!=EOF;) {
        putc(c>>1&64|c>>2&16|c>>3&4|c>>4&1, out);
        putc(c<<3&64|c<<2&16|c<<1&4|c&1, out);
      }
    }
    
    // Inverse of above program
    #include <stdio.h>
    int main(int argc, char** argv) {
      FILE* in=fopen(argv[1], "rb");
      FILE* out=fopen(argv[2], "wb");
      for (int c1, c2; (c1=getc(in))!=EOF && (c2=getc(in))!=EOF;)
        putc(c1<<1&128|c1<<2&64|c1<<3&32|c1<<4&16|c2>>3&8|c2>>2&4|c2>>1&2|c2&1, out);
    }
    Last edited by Matt Mahoney; 16th August 2012 at 23:42.

Similar Threads

  1. lossy simplifier of a png bmp jpg
    By toi007 in forum The Off-Topic Lounge
    Replies: 7
    Last Post: 7th July 2012, 00:44
  2. Replies: 3
    Last Post: 15th November 2011, 20:41
  3. ZPAQ 1.05 preview
    By Matt Mahoney in forum Data Compression
    Replies: 11
    Last Post: 30th September 2009, 04:26
  4. Winzip v12.0 with JPG recompression & 7z support
    By maadjordan in forum Data Compression
    Replies: 3
    Last Post: 12th September 2008, 23:58
  5. JPG COMPRESSION
    By Bulat Ziganshin in forum Forum Archive
    Replies: 4
    Last Post: 8th May 2007, 14:49

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •