Quo Vadis JPEG - Another update (3)

[thorfdbg]

Hi folks, just released a new version of the experimental(!) jpeg coder demonstrating the ideas our new JPEG standard will provide. The sources are here: https://github.com/thorfdbg/libjpeg Only minor modifications were made in this release. The Huffman coder now fills undefined bits at the end of the entropy coded segment by ones, and potentially inserts a stuffed zero byte. Note that the standard defines this only in an informative note, so it is likely not required. The insertion of a stuffed zero byte at this position might also be unnecessary as a parser should identify the first 0xff byte ahead of the marker as "fill byte". Residual coding has been reworked and uses now a simple progressive scan with the special DC case removed. The performance does not differ significantly, and it makes integration into legacy software or hardware easier. Default Huffman tables are now also provided for progressive and lossless scans, thus optimizing the Huffman tables just for these scan types is no longer necessary. Residual or refinement scans still require this argument, though. Usage of the codec does not differ from previous releases, though codestreams using the residual coding mode are not compatible to those generated by 0.5. Note that this is still experimental software and the codestream might still change until the ISO committee decides on the technology. Finally, a couple of workarounds for earlier releases of the g++ compiler have been added.

[load]

a win32 compile here.

http://ompldr.org/vZnY1NA

[thorfdbg]

a win32 compile here. http://ompldr.org/vZnY1NA

Thanks!

[Fallon]

I suggest to use mediafire for small legal files, or a google account. Or others.
ompldr.org is known to have spread malicious stuff like the ransom virus.
(Sorry load).

[load]

hm, i couldn't find any evidence.

https://www.virustotal.com/url/1093f...ac5c/analysis/
http://scanurl.net/?u=ompldr.org&ues...is+URL#results

[Fallon]

But there's a warning if you try to open the site in Opera browser.
https://sites.google.com/site/fallon...2012-10-14.png
Okay, I probably should have left it at that.

[Alexander Rhatushnyak]

Guess there's a bug in this version.
It fails to decompress well the attached image
(after it's converted from png back to ppm).
Most likely because of the header processing code.

jpeg.exe -p -c -a fuji15.ppm fuji15.ari
jpeg.exe fuji15.ari fuji15.dec
fc/b fuji15.ppm fuji15.dec |more
Comparing files fuji15.ppm and FUJI15.DEC
0000000F: 0B 01
00000010: 01 03
00000011: 03 09
00000012: 09 02
00000013: 02 03
00000014: 03 0A
00000015: 0A 02
00000016: 02 03
00000017: 03 0B
00000018: 0B 01
00000019: 01 03
0000001A: 03 0B
0000001B: 0B 02
0000001C: 02 03
0000001D: 03 0B
0000001E: 0B 02
0000001F: 02 03
00000020: 03 0C
...

[thorfdbg]

Guess there's a bug in this version. It fails to decompress well the attached image (after it's converted from png back to ppm). Most likely because of the header processing code.

Thanks, could very well be. Could you please sent the image to thor at math dot tu dash berlin dot de?

[thorfdbg]

Guess there's a bug in this version. It fails to decompress well the attached image

Thanks, found and fixed. Interestingly, actually not a bug of the core library. This is a side effect of writing a primitive PPM reader in a rush and not reading fscanf() properly. The latter gobbled up any character at the start of the image that is a valid white space character (bummer!) and thus shifted the image and the components within the image.

[thorfdbg]

Please find a new release at its usual place https://github.com/thorfdbg/libjpeg/ Besides the fix already mentioned above, I also found that the support for grey-scale images was broken, which was again not a library but a front-end problem. I also fixed a bug in the hierarchical coder which, depending on the image height, sometimes computed the residual signal incorrectly.

[caveman]

Please find a new release at its usual place https://github.com/thorfdbg/libjpeg/ Besides the fix already mentioned above, I also found that the support for grey-scale images was broken, which was again not a library but a front-end problem. I also fixed a bug in the hierarchical coder which, depending on the image height, sometimes computed the residual signal incorrectly.

Just a stupid question, does your lib write all the DHT (Huffman tables) and DQT (Quantization tables) in a single marker (Adobe Photoshop does it this way and it saves a few bytes) or does it only enclose one table per marker (IJG libjpg does it this way)?

Sample files attached.

IJG libjpeg (4 Huffman tables in 4 different DHT markers):

Offset 0x00b1 Marker 0xffc4 DHT Define Huffman Table(s) length variable 0x1c
JPEG_DHT_Parameters:
TableClass = 0
HuffmanTableIdentifier = 0
nHuffmanCodesOfLength 0 = 1
ValueOfHuffmanCode 0 = 0
nHuffmanCodesOfLength 1 = 0
nHuffmanCodesOfLength 2 = 0
nHuffmanCodesOfLength 3 = 7
ValueOfHuffmanCode 0 = 1
ValueOfHuffmanCode 1 = 2
ValueOfHuffmanCode 2 = 3
ValueOfHuffmanCode 3 = 5
ValueOfHuffmanCode 4 = 6
ValueOfHuffmanCode 5 = 7
ValueOfHuffmanCode 6 = 8
nHuffmanCodesOfLength 4 = 1
ValueOfHuffmanCode 0 = 4
nHuffmanCodesOfLength 5 = 0
nHuffmanCodesOfLength 6 = 0
nHuffmanCodesOfLength 7 = 0
nHuffmanCodesOfLength 8 = 0
nHuffmanCodesOfLength 9 = 0
nHuffmanCodesOfLength 10 = 0
nHuffmanCodesOfLength 11 = 0
nHuffmanCodesOfLength 12 = 0
nHuffmanCodesOfLength 13 = 0
nHuffmanCodesOfLength 14 = 0
nHuffmanCodesOfLength 15 = 0

Offset 0x00cf Marker 0xffc4 DHT Define Huffman Table(s) length variable 0x5b
JPEG_DHT_Parameters:
TableClass = 1
HuffmanTableIdentifier = 0
nHuffmanCodesOfLength 0 = 0
nHuffmanCodesOfLength 1 = 0
nHuffmanCodesOfLength 2 = 5
ValueOfHuffmanCode 0 = 0
ValueOfHuffmanCode 1 = 1
ValueOfHuffmanCode 2 = 2
ValueOfHuffmanCode 3 = 3
ValueOfHuffmanCode 4 = 4
nHuffmanCodesOfLength 3 = 2
ValueOfHuffmanCode 0 = 5
ValueOfHuffmanCode 1 = 18
nHuffmanCodesOfLength 4 = 2
ValueOfHuffmanCode 0 = 6
ValueOfHuffmanCode 1 = 17
nHuffmanCodesOfLength 5 = 3
ValueOfHuffmanCode 0 = 7
ValueOfHuffmanCode 1 = 19
ValueOfHuffmanCode 2 = 34
nHuffmanCodesOfLength 6 = 8
ValueOfHuffmanCode 0 = 8
ValueOfHuffmanCode 1 = 20
ValueOfHuffmanCode 2 = 33
ValueOfHuffmanCode 3 = 49
ValueOfHuffmanCode 4 = 50
ValueOfHuffmanCode 5 = 65
ValueOfHuffmanCode 6 = 81
ValueOfHuffmanCode 7 = 180
nHuffmanCodesOfLength 7 = 13
ValueOfHuffmanCode 0 = 21
ValueOfHuffmanCode 1 = 24
ValueOfHuffmanCode 2 = 35
ValueOfHuffmanCode 3 = 40
ValueOfHuffmanCode 4 = 55
ValueOfHuffmanCode 5 = 66
ValueOfHuffmanCode 6 = 82
ValueOfHuffmanCode 7 = 85
ValueOfHuffmanCode 8 = 97
ValueOfHuffmanCode 9 = 117
ValueOfHuffmanCode 10 = 148
ValueOfHuffmanCode 11 = 161
ValueOfHuffmanCode 12 = 212
nHuffmanCodesOfLength 8 = 7
ValueOfHuffmanCode 0 = 36
ValueOfHuffmanCode 1 = 51
ValueOfHuffmanCode 2 = 98
ValueOfHuffmanCode 3 = 113
ValueOfHuffmanCode 4 = 114
ValueOfHuffmanCode 5 = 145
ValueOfHuffmanCode 6 = 182
nHuffmanCodesOfLength 9 = 6
ValueOfHuffmanCode 0 = 22
ValueOfHuffmanCode 1 = 23
ValueOfHuffmanCode 2 = 129
ValueOfHuffmanCode 3 = 146
ValueOfHuffmanCode 4 = 162
ValueOfHuffmanCode 5 = 210
nHuffmanCodesOfLength 10 = 9
ValueOfHuffmanCode 0 = 52
ValueOfHuffmanCode 1 = 54
ValueOfHuffmanCode 2 = 83
ValueOfHuffmanCode 3 = 99
ValueOfHuffmanCode 4 = 130
ValueOfHuffmanCode 5 = 163
ValueOfHuffmanCode 6 = 164
ValueOfHuffmanCode 7 = 177
ValueOfHuffmanCode 8 = 178
nHuffmanCodesOfLength 11 = 10
ValueOfHuffmanCode 0 = 37
ValueOfHuffmanCode 1 = 38
ValueOfHuffmanCode 2 = 39
ValueOfHuffmanCode 3 = 101
ValueOfHuffmanCode 4 = 102
ValueOfHuffmanCode 5 = 118
ValueOfHuffmanCode 6 = 132
ValueOfHuffmanCode 7 = 150
ValueOfHuffmanCode 8 = 194
ValueOfHuffmanCode 9 = 213
nHuffmanCodesOfLength 12 = 7
ValueOfHuffmanCode 0 = 67
ValueOfHuffmanCode 1 = 69
ValueOfHuffmanCode 2 = 116
ValueOfHuffmanCode 3 = 147
ValueOfHuffmanCode 4 = 166
ValueOfHuffmanCode 5 = 179
ValueOfHuffmanCode 6 = 226
nHuffmanCodesOfLength 13 = 0
nHuffmanCodesOfLength 14 = 0
nHuffmanCodesOfLength 15 = 0

Offset 0x012c Marker 0xffc4 DHT Define Huffman Table(s) length variable 0x1c
JPEG_DHT_Parameters:
TableClass = 0
HuffmanTableIdentifier = 1
nHuffmanCodesOfLength 0 = 1
ValueOfHuffmanCode 0 = 0
nHuffmanCodesOfLength 1 = 0
nHuffmanCodesOfLength 2 = 2
ValueOfHuffmanCode 0 = 1
ValueOfHuffmanCode 1 = 2
nHuffmanCodesOfLength 3 = 3
ValueOfHuffmanCode 0 = 3
ValueOfHuffmanCode 1 = 4
ValueOfHuffmanCode 2 = 5
nHuffmanCodesOfLength 4 = 1
ValueOfHuffmanCode 0 = 6
nHuffmanCodesOfLength 5 = 1
ValueOfHuffmanCode 0 = 7
nHuffmanCodesOfLength 6 = 1
ValueOfHuffmanCode 0 = 8
nHuffmanCodesOfLength 7 = 0
nHuffmanCodesOfLength 8 = 0
nHuffmanCodesOfLength 9 = 0
nHuffmanCodesOfLength 10 = 0
nHuffmanCodesOfLength 11 = 0
nHuffmanCodesOfLength 12 = 0
nHuffmanCodesOfLength 13 = 0
nHuffmanCodesOfLength 14 = 0
nHuffmanCodesOfLength 15 = 0

Offset 0x014a Marker 0xffc4 DHT Define Huffman Table(s) length variable 0x46
JPEG_DHT_Parameters:
TableClass = 1
HuffmanTableIdentifier = 1
nHuffmanCodesOfLength 0 = 0
nHuffmanCodesOfLength 1 = 2
ValueOfHuffmanCode 0 = 0
ValueOfHuffmanCode 1 = 1
nHuffmanCodesOfLength 2 = 1
ValueOfHuffmanCode 0 = 2
nHuffmanCodesOfLength 3 = 2
ValueOfHuffmanCode 0 = 3
ValueOfHuffmanCode 1 = 17
nHuffmanCodesOfLength 4 = 2
ValueOfHuffmanCode 0 = 4
ValueOfHuffmanCode 1 = 33
nHuffmanCodesOfLength 5 = 5
ValueOfHuffmanCode 0 = 5
ValueOfHuffmanCode 1 = 18
ValueOfHuffmanCode 2 = 19
ValueOfHuffmanCode 3 = 49
ValueOfHuffmanCode 4 = 240
nHuffmanCodesOfLength 6 = 7
ValueOfHuffmanCode 0 = 6
ValueOfHuffmanCode 1 = 21
ValueOfHuffmanCode 2 = 34
ValueOfHuffmanCode 3 = 50
ValueOfHuffmanCode 4 = 65
ValueOfHuffmanCode 5 = 81
ValueOfHuffmanCode 6 = 209
nHuffmanCodesOfLength 7 = 8
ValueOfHuffmanCode 0 = 20
ValueOfHuffmanCode 1 = 113
ValueOfHuffmanCode 2 = 129
ValueOfHuffmanCode 3 = 145
ValueOfHuffmanCode 4 = 161
ValueOfHuffmanCode 5 = 177
ValueOfHuffmanCode 6 = 193
ValueOfHuffmanCode 7 = 241
nHuffmanCodesOfLength 8 = 7
ValueOfHuffmanCode 0 = 7
ValueOfHuffmanCode 1 = 51
ValueOfHuffmanCode 2 = 66
ValueOfHuffmanCode 3 = 82
ValueOfHuffmanCode 4 = 97
ValueOfHuffmanCode 5 = 130
ValueOfHuffmanCode 6 = 194
nHuffmanCodesOfLength 9 = 5
ValueOfHuffmanCode 0 = 35
ValueOfHuffmanCode 1 = 36
ValueOfHuffmanCode 2 = 114
ValueOfHuffmanCode 3 = 162
ValueOfHuffmanCode 4 = 210
nHuffmanCodesOfLength 10 = 8
ValueOfHuffmanCode 0 = 22
ValueOfHuffmanCode 1 = 52
ValueOfHuffmanCode 2 = 98
ValueOfHuffmanCode 3 = 99
ValueOfHuffmanCode 4 = 115
ValueOfHuffmanCode 5 = 146
ValueOfHuffmanCode 6 = 225
ValueOfHuffmanCode 7 = 242
nHuffmanCodesOfLength 11 = 3
ValueOfHuffmanCode 0 = 178
ValueOfHuffmanCode 1 = 211
ValueOfHuffmanCode 2 = 226
nHuffmanCodesOfLength 12 = 1
ValueOfHuffmanCode 0 = 227
nHuffmanCodesOfLength 13 = 0
nHuffmanCodesOfLength 14 = 0
nHuffmanCodesOfLength 15 = 0



Photoshop (4 Huffman tables in single DHT marker):

Offset 0x00d8 Marker 0xffc4 DHT Define Huffman Table(s) length variable 0xd3
JPEG_DHT_Parameters:
TableClass = 0
HuffmanTableIdentifier = 0
nHuffmanCodesOfLength 0 = 1
ValueOfHuffmanCode 0 = 0
nHuffmanCodesOfLength 1 = 0
nHuffmanCodesOfLength 2 = 0
nHuffmanCodesOfLength 3 = 7
ValueOfHuffmanCode 0 = 1
ValueOfHuffmanCode 1 = 2
ValueOfHuffmanCode 2 = 3
ValueOfHuffmanCode 3 = 5
ValueOfHuffmanCode 4 = 6
ValueOfHuffmanCode 5 = 7
ValueOfHuffmanCode 6 = 8
nHuffmanCodesOfLength 4 = 1
ValueOfHuffmanCode 0 = 4
nHuffmanCodesOfLength 5 = 0
nHuffmanCodesOfLength 6 = 0
nHuffmanCodesOfLength 7 = 0
nHuffmanCodesOfLength 8 = 0
nHuffmanCodesOfLength 9 = 0
nHuffmanCodesOfLength 10 = 0
nHuffmanCodesOfLength 11 = 0
nHuffmanCodesOfLength 12 = 0
nHuffmanCodesOfLength 13 = 0
nHuffmanCodesOfLength 14 = 0
nHuffmanCodesOfLength 15 = 0
TableClass = 0
HuffmanTableIdentifier = 1
nHuffmanCodesOfLength 0 = 1
ValueOfHuffmanCode 0 = 0
nHuffmanCodesOfLength 1 = 0
nHuffmanCodesOfLength 2 = 2
ValueOfHuffmanCode 0 = 1
ValueOfHuffmanCode 1 = 2
nHuffmanCodesOfLength 3 = 3
ValueOfHuffmanCode 0 = 3
ValueOfHuffmanCode 1 = 4
ValueOfHuffmanCode 2 = 5
nHuffmanCodesOfLength 4 = 1
ValueOfHuffmanCode 0 = 6
nHuffmanCodesOfLength 5 = 1
ValueOfHuffmanCode 0 = 7
nHuffmanCodesOfLength 6 = 1
ValueOfHuffmanCode 0 = 8
nHuffmanCodesOfLength 7 = 0
nHuffmanCodesOfLength 8 = 0
nHuffmanCodesOfLength 9 = 0
nHuffmanCodesOfLength 10 = 0
nHuffmanCodesOfLength 11 = 0
nHuffmanCodesOfLength 12 = 0
nHuffmanCodesOfLength 13 = 0
nHuffmanCodesOfLength 14 = 0
nHuffmanCodesOfLength 15 = 0
TableClass = 1
HuffmanTableIdentifier = 0
nHuffmanCodesOfLength 0 = 0
nHuffmanCodesOfLength 1 = 0
nHuffmanCodesOfLength 2 = 5
ValueOfHuffmanCode 0 = 0
ValueOfHuffmanCode 1 = 1
ValueOfHuffmanCode 2 = 2
ValueOfHuffmanCode 3 = 3
ValueOfHuffmanCode 4 = 4
nHuffmanCodesOfLength 3 = 2
ValueOfHuffmanCode 0 = 18
ValueOfHuffmanCode 1 = 5
nHuffmanCodesOfLength 4 = 2
ValueOfHuffmanCode 0 = 17
ValueOfHuffmanCode 1 = 6
nHuffmanCodesOfLength 5 = 3
ValueOfHuffmanCode 0 = 34
ValueOfHuffmanCode 1 = 19
ValueOfHuffmanCode 2 = 7
nHuffmanCodesOfLength 6 = 8
ValueOfHuffmanCode 0 = 33
ValueOfHuffmanCode 1 = 49
ValueOfHuffmanCode 2 = 65
ValueOfHuffmanCode 3 = 81
ValueOfHuffmanCode 4 = 50
ValueOfHuffmanCode 5 = 20
ValueOfHuffmanCode 6 = 180
ValueOfHuffmanCode 7 = 8
nHuffmanCodesOfLength 7 = 14
ValueOfHuffmanCode 0 = 97
ValueOfHuffmanCode 1 = 161
ValueOfHuffmanCode 2 = 66
ValueOfHuffmanCode 3 = 82
ValueOfHuffmanCode 4 = 98
ValueOfHuffmanCode 5 = 35
ValueOfHuffmanCode 6 = 148
ValueOfHuffmanCode 7 = 212
ValueOfHuffmanCode 8 = 21
ValueOfHuffmanCode 9 = 85
ValueOfHuffmanCode 10 = 117
ValueOfHuffmanCode 11 = 55
ValueOfHuffmanCode 12 = 24
ValueOfHuffmanCode 13 = 40
nHuffmanCodesOfLength 8 = 6
ValueOfHuffmanCode 0 = 113
ValueOfHuffmanCode 1 = 145
ValueOfHuffmanCode 2 = 114
ValueOfHuffmanCode 3 = 51
ValueOfHuffmanCode 4 = 36
ValueOfHuffmanCode 5 = 182
nHuffmanCodesOfLength 9 = 4
ValueOfHuffmanCode 0 = 129
ValueOfHuffmanCode 1 = 146
ValueOfHuffmanCode 2 = 162
ValueOfHuffmanCode 3 = 210
nHuffmanCodesOfLength 10 = 9
ValueOfHuffmanCode 0 = 177
ValueOfHuffmanCode 1 = 130
ValueOfHuffmanCode 2 = 178
ValueOfHuffmanCode 3 = 83
ValueOfHuffmanCode 4 = 99
ValueOfHuffmanCode 5 = 164
ValueOfHuffmanCode 6 = 22
ValueOfHuffmanCode 7 = 54
ValueOfHuffmanCode 8 = 23
nHuffmanCodesOfLength 11 = 9
ValueOfHuffmanCode 0 = 194
ValueOfHuffmanCode 1 = 163
ValueOfHuffmanCode 2 = 52
ValueOfHuffmanCode 3 = 132
ValueOfHuffmanCode 4 = 101
ValueOfHuffmanCode 5 = 213
ValueOfHuffmanCode 6 = 102
ValueOfHuffmanCode 7 = 118
ValueOfHuffmanCode 8 = 150
nHuffmanCodesOfLength 12 = 9
ValueOfHuffmanCode 0 = 226
ValueOfHuffmanCode 1 = 67
ValueOfHuffmanCode 2 = 147
ValueOfHuffmanCode 3 = 179
ValueOfHuffmanCode 4 = 116
ValueOfHuffmanCode 5 = 37
ValueOfHuffmanCode 6 = 69
ValueOfHuffmanCode 7 = 38
ValueOfHuffmanCode 8 = 166
nHuffmanCodesOfLength 13 = 1
ValueOfHuffmanCode 0 = 39
nHuffmanCodesOfLength 14 = 0
nHuffmanCodesOfLength 15 = 0
TableClass = 1
HuffmanTableIdentifier = 1
nHuffmanCodesOfLength 0 = 0
nHuffmanCodesOfLength 1 = 2
ValueOfHuffmanCode 0 = 0
ValueOfHuffmanCode 1 = 1
nHuffmanCodesOfLength 2 = 1
ValueOfHuffmanCode 0 = 2
nHuffmanCodesOfLength 3 = 2
ValueOfHuffmanCode 0 = 17
ValueOfHuffmanCode 1 = 3
nHuffmanCodesOfLength 4 = 2
ValueOfHuffmanCode 0 = 33
ValueOfHuffmanCode 1 = 4
nHuffmanCodesOfLength 5 = 5
ValueOfHuffmanCode 0 = 240
ValueOfHuffmanCode 1 = 49
ValueOfHuffmanCode 2 = 18
ValueOfHuffmanCode 3 = 19
ValueOfHuffmanCode 4 = 5
nHuffmanCodesOfLength 6 = 7
ValueOfHuffmanCode 0 = 65
ValueOfHuffmanCode 1 = 81
ValueOfHuffmanCode 2 = 209
ValueOfHuffmanCode 3 = 34
ValueOfHuffmanCode 4 = 50
ValueOfHuffmanCode 5 = 21
ValueOfHuffmanCode 6 = 6
nHuffmanCodesOfLength 7 = 8
ValueOfHuffmanCode 0 = 113
ValueOfHuffmanCode 1 = 129
ValueOfHuffmanCode 2 = 145
ValueOfHuffmanCode 3 = 161
ValueOfHuffmanCode 4 = 177
ValueOfHuffmanCode 5 = 193
ValueOfHuffmanCode 6 = 241
ValueOfHuffmanCode 7 = 20
nHuffmanCodesOfLength 8 = 7
ValueOfHuffmanCode 0 = 97
ValueOfHuffmanCode 1 = 66
ValueOfHuffmanCode 2 = 82
ValueOfHuffmanCode 3 = 130
ValueOfHuffmanCode 4 = 194
ValueOfHuffmanCode 5 = 51
ValueOfHuffmanCode 6 = 7
nHuffmanCodesOfLength 9 = 5
ValueOfHuffmanCode 0 = 114
ValueOfHuffmanCode 1 = 162
ValueOfHuffmanCode 2 = 210
ValueOfHuffmanCode 3 = 35
ValueOfHuffmanCode 4 = 36
nHuffmanCodesOfLength 10 = 8
ValueOfHuffmanCode 0 = 225
ValueOfHuffmanCode 1 = 98
ValueOfHuffmanCode 2 = 146
ValueOfHuffmanCode 3 = 242
ValueOfHuffmanCode 4 = 99
ValueOfHuffmanCode 5 = 115
ValueOfHuffmanCode 6 = 52
ValueOfHuffmanCode 7 = 22
nHuffmanCodesOfLength 11 = 3
ValueOfHuffmanCode 0 = 178
ValueOfHuffmanCode 1 = 226
ValueOfHuffmanCode 2 = 211
nHuffmanCodesOfLength 12 = 1
ValueOfHuffmanCode 0 = 227
nHuffmanCodesOfLength 13 = 0
nHuffmanCodesOfLength 14 = 0
nHuffmanCodesOfLength 15 = 0

[thorfdbg]

Just a stupid question, does your lib write all the DHT (Huffman tables) and DQT (Quantization tables) in a single marker (Adobe Photoshop does it this way and it saves a few bytes) or does it only enclose one table per marker (IJG libjpg does it this way)? Sample files attached.

They go all into one marker, i.e. one marker for all tables, one marker for all quantization matrices. For optimal compression, you should of course use the optimized Huffman option.

[caveman]

They go all into one marker, i.e. one marker for all tables, one marker for all quantization matrices. For optimal compression, you should of course use the optimized Huffman option.

Interesting!
And do you know if trying different permutations of the Huffman values of the same size (they are usually sorted in growing order, I don't think this is mandatory) could hinder the occurrence of FF and subsequent 00 bytes in the compressed data? (this is done to avoid false marker detection)

[load]

windows compile:

http://rghost.net/41704129

[thorfdbg]

Interesting! And do you know if trying different permutations of the Huffman values of the same size (they are usually sorted in growing order, I don't think this is mandatory) could hinder the occurrence of FF and subsequent 00 bytes in the compressed data? (this is done to avoid false marker detection)

Actually, I haven't thought about this yet. False markers and bitstuffing may appear as soon as you have a code with more than eight consecutive ones, which - due to the way how the huffman tables are built - appear automatically as soon as you have codes longer than eight bit. One could probably avoid such codes by assigning the corresponding bit sequences to codewords that are very rare or do not appear at all, but that also enlarges the code of the other sequences. Actually, this sounds like a nice optimization problem I haven't thought trough yet. However, please note that not all bits are huffman coded, and ff-bytes can also appear due to the raw bits inserted into the bitstream, thus, this won't avoid byte-stuffing in all cases.

[thorfdbg]

windows compile: http://rghost.net/41704129

Thanks!

[thorfdbg]

Just uploaded a new release to git:
https://github.com/thorfdbg/libjpeg
This improves the performance of the AC coding modes by using separate contexts for chroma/luma. It also fixes the workaround for the JPEG LS part-2 color transformation which did the overflow check incorrectly.
The repository also includes a revised version of the ISO working draft. Note, however, that this is an early WD, the software does not conform to this WD, nor has the WD been decided upon.
The JPEG online test at
http://jpegonline.rus.uni-stuttgart.de/index.py
has been improved to also measure mean multiscale SSIM as one of the metrics. Have patience, though, measuring SSIM takes more time than PSNR or peak error.

[load]

Win32 compile:

http://www.sendmyway.com/rs9cv8lum5le/libjpeg.rar.html
http://ge.tt/3AHupoS
https://netload.in/dateijfzX1CQ49o/libjpeg.rar.htm
http://www.load.to/SfW7DLKWnh/libjpeg.rar
http://www.gamefront.com/files/22703238/libjpeg.rar
http://rghost.net/41980355

[Alexander Rhatushnyak]

Tested compression quality and bit-exactness on LPCB files.

2,010,200,240 bytes   jpeg -c -p         %1.ppm %1.qvj   http://imagecompression.info/gralic/list-qvj.txt
1,440,226,595 bytes   jpeg -c -p -a      %1.ppm %1.qva   http://imagecompression.info/gralic/list-qva.txt
1,290,129,169 bytes   jpeg -c -ls 0      %1.ppm %1.qv0   http://imagecompression.info/gralic/list-qv0.txt
1,293,859,667 bytes   jpeg -c -ls 1      %1.ppm %1.qv1   http://imagecompression.info/gralic/list-qv1.txt
1,299,798,722 bytes   jpeg -c -ls 2      %1.ppm %1.qv2   http://imagecompression.info/gralic/list-qv2.txt
1,210,043,117 bytes   jpeg -c -ls 0 -cls %1.ppm %1.qc0   http://imagecompression.info/gralic/list-qc0.txt
1,217,125,694 bytes   jpeg -c -ls 1 -cls %1.ppm %1.qc1   http://imagecompression.info/gralic/list-qc1.txt
1,222,696,140 bytes   jpeg -c -ls 2 -cls %1.ppm %1.qc2   http://imagecompression.info/gralic/list-qc2.txt

In the latter three cases the decompressed images aren't bit-exact in many cases, e.g. image from http://www.imagecompression.info/gralic/PIA13862ppm.rar :

000335a5: 00 FF
// skipped 2239 lines, all ending with 00 FF
0018f677: 00 FF

Are there other sets of options for lossless compression?

[thorfdbg]

Tested compression quality and bit-exactness on LPCB files. In the latter three cases the decompressed images aren't bit-exact in many cases, e.g. image from http://www.imagecompression.info/gralic/PIA13862ppm.rar : 000335a5: 00 FF // skipped 2239 lines, all ending with 00 FF 0018f677: 00 FF Are there other sets of options for lossless compression?

Fixed. I misplaced a > where I should have a >=, only in the blue component. All other components are fine.
Other options for lossless: "-r -h" uses residual coding, use "-q" to select a "base quality" which will be visible to legacy decoders. Useful values for -q depend on the image. For complex images, q can be as low as 60, for simple images, go higher. I would pick 90 as base layer quality. This does not require "-c", also works with YCbCr. This is a new mode in the JPEG extensions.
You can also use hierarchical coding for lossless, but only in RGB space, and only with this coder (simply because the DCT isn't standardized in the old standard). For this, you can use the original DCT image as base layer, or a downscaled version of the DCT image as base image. Use -q to select a base quality, -y 0 for a same-resolution base layer, and -y 1 for a downscaled image as base layer. Thus:
-h -y 0 -c -q 90
or
-h -y 1 -c -q 90
Again, only for this code, not lossless in general. "-r" for residual coding works best.

[thorfdbg]

Tested compression quality and bit-exactness on LPCB files. In the latter three cases the decompressed images aren't bit-exact in many cases, e.g. image from http://www.imagecompression.info/gralic/PIA13862ppm.rar : 000335a5: 00 FF // skipped 2239 lines, all ending with 00 FF 0018f677: 00 FF Are there other sets of options for lossless compression?

Fixed. I misplaced a > where I should have a >=, only in the blue component. All other components are fine.
Other options for lossless: "-r -h" uses residual coding, use "-q" to select a "base quality" which will be visible to legacy decoders. Useful values for -q depend on the image. For complex images, q can be as low as 60, for simple images, go higher. I would pick 90 as base layer quality. This does not require "-c", also works with YCbCr. This is a new mode in the JPEG extensions.
You can also use hierarchical coding for lossless, but only in RGB space, and only with this coder (simply because the DCT isn't standardized in the old standard). For this, you can use the original DCT image as base layer, or a downscaled version of the DCT image as base image. Use -q to select a base quality, -y 0 for a same-resolution base layer, and -y 1 for a downscaled image as base layer. Thus:
-h -y 0 -c -q 90
or
-h -y 1 -c -q 90
Again, only for this code, not lossless in general. "-r" for residual coding works best. Again, this *is* backwards compatible, though legacy decoders will not be able to decode lossless, but only to the base-layer quality you specified. "-y" "should" work with legacy decoders theoretically, except that there is no legacy decoder I'm aware that implemented hierarchical.

[Alexander Rhatushnyak]

Added to LPCB.

Win32 compile:

How was it built, and is the Linux compile ~8 times slower than CharLS too?

[Alexander Rhatushnyak]

Okay, will try these five next time:
jpeg -c -ls 0 -cls
jpeg -c -ls 1 -cls
jpeg -c -ls 2 -cls
jpeg -h -y 0 -c -q 90
jpeg -h -y 1 -c -q 90

-r isn't needed for the latter two variants, is it?

This time only the following five were added to LPCB:
jpeg -c -p
jpeg -c -p -a
jpeg -c -ls 0
jpeg -c -ls 1
jpeg -c -ls 2

[thorfdbg]

Okay, will try these five next time: jpeg -c -ls 0 -cls jpeg -c -ls 1 -cls jpeg -c -ls 2 -cls jpeg -h -y 0 -c -q 90 jpeg -h -y 1 -c -q 90 -r isn't needed for the latter two variants, is it?

For none of them. Residual coding is based on the "still lossy" modes - not required for modes that are already lossless. Thus, simply "jpeg -h -r -q XX" or "jpeg -a -r -q XX", where the latter (ac-coding of legacy and residual data) will likely not end up in the specs.

[thorfdbg]

New version on git. Really only a single misplaced > instead of >=. Bummer!

[thorfdbg]

Added to LPCB. How was it built, and is the Linux compile ~8 times slower than CharLS too?

Wouldn't surprise me. For example, the Golomb decoder in the current version is "clear", but "outright stupid", and it would be obvious how to improve its performance: It currently reads one bit at a time, counting bits. It's really not necessary to do it this way. However, speed wasn't quite the issue here. Speed might come later.

[load]

Win32 compile:

http://www.mirrorcreator.com/files/0...jpeg.rar_links

[thorfdbg]

Just released a tiny fix - apparently, the code picked up the wrong Huffman AC coding table in the progressive mode due to a stupid typo that was completely unnoticed. Fixed that.

[load]

ok, here's the win32 compile. maybe it works.

http://www.mirrorcreator.com/files/P...jpeg.rar_links