paq8px

[Jan Ondrus]

I added a validation that the directory isn't on offset 8 then the probability for image block starting there is good.

Don't understand what is problem with directory on offset 8?

bswap(buf0)!=8

Why you added this check?

[Simon Berger]

Is much more fun to integrate new things into paq. Excellent work Jan
The tga detection excludes files with an image id cause the leading byte would be unusable for identification.
8-bit tga's should be easy but not often used?!?

What's with a 32-bit image model? See them almost as often as 24-bit.

[osmanturan]

8-bit tga's should be easy but not often used?!?

Mostly used in games as alpha map for blending or as height map for per-pixel shading or terrain construction (they are all greyscale).

[Simon Berger]

Don't understand what is problem with directory on offset 8?

This detection only works if the image data starts at offset 8 or am I wrong? On implementations other then LibTiff/IrfanView it often starts at 8. Those can immediately be discarded.

[Jan Ondrus]

This detection only works if the image data starts at offset 8 or am I wrong? On implementations other then LibTiff/IrfanView it often starts at 8. Those can immediately be discarded.

Actually it reads offset from directory. It reads value marked with tag StripOffsets (273) = pointer to list of offsets. Than seeks to that position and reads first value = pointer to first strip.
But it needs to have all image parts concatenated (in the right order and without data between)

if (tag==273) tifofs=tagval;

fseek(in, start+tiff+tifofs-3, SEEK_SET);

[Simon Berger]

You are right I missed this part then this surely is a bad check and should be removed in the next version.

[Darek]

@Simon, thanks for information. My testbed tiffs should be one of kind you mention and due to this aren't recognised.
TGA 24bit compression works great and gives one of the best scores for my testfile:
b.tga original: 2'872'818
after paq8px v9 -6: 823'158
after paq9px v10 -6: 457'020
winrk3.03b max, 1.7GB: 442'700
Darek

[Simon Berger]

paq8px_v11

Added tga (8-bit) UNTESTED - please verify. Didn't find any 8-bit tga file
Fixed tiff reading of offset stored by value (partcount=1)
Fixed now the tiff stream won't be read if offsets stored in 16-bit (also not worth to implement)
Removed bad tiff directory check

@Darek
My answer wasn't right sorry. I missed some "features" of Jans implementation.
There are some possibilities it could fail if you could give me the file I will have a look. I fixed one thing it could be in this version so please check again.
Only to be sure. The tiff file is uncompressed?

[Darek]

@Simon,
in my testbed I use three tiff files, first is 24 bit and not compressed internally, second is 8 bit, either uncompressed, and third 8 bit compressed LZW. Paq isn't recognise none of them, but I didn't test v.11 yet. If it wouldn't work I'll send you my file.
Darek

[Darek]

@Simon
in attached files are two tga examples - 8 bit tga from my testbed (but it's compressed rle), and the same picture as 8 bit tga uncompressed (I think ).
Darek

[Simon Berger]

Re-uploaded v11 which works now with one tested 8-bit tga file.
Had some troubles parsing the tga header.

@Darek
Could you try if your tiff files are working with my TiffPrecompression?
Tiffprecompression->paq8px_v9 or v10.
Thanks for the tga files. I also get it to work with a 8-bit Tiff files which I converted. But nowhere found something to create one.

EDIT:
Works without problems.
Absolutely better to have such a picture without any compression or am I wrong?

[Darek]

@Simon,
unfortunately tiffprecompresson didn't run on my machine. I've tried all versions. My system return information that this program is "not properly installed" - strange....
Darek

[LovePimple]

Thanks Simon!

Compiled...

EDIT: Attachment "paq8px_v11.zip" removed.

[Darek]

@Simon,
here you are some examples of tiffs (c - 8 bit uncompressed, e - 8 bit compressed lzw). Both are in similar style (sorry) but it becomes from the same album of Boris Vallejo, who was my favourite artists 12 years ago when I was creating this testbed.
Darek

[Simon Berger]

It's exactly the problem I mentioned. This is fixed in v11 release. Many bad implementations of the Tiff standard only use 1 image block or read them like one block.

For TiffPrecompression you have to install the VC++ 2008 Redistributable package. I linked it at the first post.
It's really bad that so many haven't installed it yet. I have to use mingw next time.

@LovePimple
Thank you for all your compiled versions.

[Darek]

Thanks for LovePimple for compilation!

@Simon
Version 11 properly recognised all uncompressed tiff and tga files. Thanks for enchancements, which are very impressive!
Scores for -6 option:
A.TIF - org.: 2'870'552, v9: 874'429, v11: 498'354 (43% of improvement)
B.TGA - org.: 2'872'818, v9: 823'158, v11: 457'020 (44% of improvement)
C.TIF - org.: 896'094, v9: 334'985, v11: 308'102 (8% of improvement)
Darek

[Rugxulo]

For TiffPrecompression you have to install the VC++ 2008 Redistributable package. I linked it at the first post.
It's really bad that so many haven't installed it yet. I have to use mingw next time.

While I still don't know why nobody has fixed MinGW to not use MSVCRT's broken tmpfile(), at least somebody unofficially compiled a MinGW-hosted GCC 4.4.0 !! (I still prefer DJGPP, OpenWatcom, and Cygwin, though, for various reasons.)

http://www.tdragon.net/recentgcc/

[Shelwien]

@Simon Berger:
You can just compile stuff statically, without dynamic libraries (/MT switch).
Or, like I do, you can use libraries from VC6, which only require msvcrt.dll.

[Jan Ondrus]

paq8px_v12
-fixed bug in TGA detection
-some code changed in BMP,RGB and TIFF detection

m1l10002.tga (24-bit) - not detected by v11
M1.TGA (8-bit) - detected by v11 but with wrong data offset

m1l10002.tga - 196626 bytes
M1.TGA - 5544 bytes
m1l10002.tga.paq8px_v11 - 66983 bytes
M1.TGA.paq8px_v11 - 1760 bytes
m1l10002.tga.paq8px_v12 - 58377 bytes
M1.TGA.paq8px_v12 - 1686 bytes

[LovePimple]

Thanks Jan!

Compiled...

EDIT: Attachment "paq8px_v12.zip" removed.

[Black_Fox]

...at least somebody unofficially compiled a MinGW-hosted GCC 4.4.0 !!
http://www.tdragon.net/recentgcc/

Thanks! I'll finally get an update after a year

[Simon Berger]

@Jan
The tga detection works for both of my test files, but you still worked a new mistake in:
image format 3 is monochrome and won't work.


I have no new ideas at the moment so won't change it myself.

[Jan Ondrus]

@Jan
The tga detection works for both of my test files, but you still worked a new mistake in:
image format 3 is monochrome and won't work.

Yes, you are right.
a) fixed - monochrome TGA works again
b) some code modified in TIFF detection

WhiteFlare1.tga - Monochrome TGA (4140 bytes)
WhiteFlare1.tga.paq8px_v11 - 1005 bytes
WhiteFlare1.tga.paq8px_v12 - 1270 bytes
WhiteFlare1.tga.paq8px_v13 - 1005 bytes

[LovePimple]

Fantastic, I'm looking forward to the next release. Thanks Jan!

Compiled...

EDIT: Attachment "paq8px_v13.zip" removed.

[Simon Berger]

Jan could you create a image 32bit model? I have many tga files using this. It's Alpha|Red|Green|Blue. The paq core is still over my head.
This could also be used to compress special 32-bit bmps.

[Jan Ondrus]

Jan could you create a image 32bit model? I have many tga files using this. It's Alpha|Red|Green|Blue. The paq core is still over my head.
This could also be used to compress special 32-bit bmps.

I am not author of image models and don't know how to write one too.

[Simon Berger]

Graphic formats still in my mind, to add:

1) PSD - would be nice to add. But I still stuck on fully understanding the file format. Size values don't make sense to me on many places. Also where what picture data will be saved. If someone knows more it would be fine.
Second problem is that most images are saved with RLE compression (same as in TIFF). But this could maybe also be done and at least with something like TiffPrecompress.

2) DDS - Like it. But it's good (lossless compressed). Also paq still compresses it well. A first attempt to convert it to a bmp showed no better result.

[osmanturan]

2) DDS - Like it. But it's good (lossless compressed). Also paq still compresses it well. A first attempt to convert it to a bmp showed no better result.

You are not completely right in here. Because, DDS files may have lossy compression. It's actually a container format as DirectDraw textures. They are coded in native graphics accelerator pixel storage. So, they have several pixel format including lossy. DirectX library has built-in loading capabalities of them (for both DirectDraw and Direct3D). Also, in OpenGL; S3TC, DXT1-5 etc like lossy compressed textures can be directly uploaded to graphics accelerator too (of course, if hardware support them - though only some ancient GPUs don't support them). So, that's why they're mainly used in games. Note that, DDS files can even store cube map textures (set of 6 equal size textures for sky mapping, environment mapping etc) and volume textures (3D textures). As a summary, DDS files are not easy to handle. Because, GPU world advance quickly. But, still mostly DXT1-5 lossy compressed textures and cubemaps are used in many games. On the other hand, by pixel shading becoming popular in many games, ATI's 3Dc compression for normalmaps could be popular in near future (personally I'm not satisfied with it's current popularity until now).

Another thing you should know about DDS files is that mipmapping. GPUs needs multi-resolution representation of uploaded textures for eliminating aliasing problems on rasterized surfaces. And in here mipmapping is used. Mipmapping also reduces required transfer rate between texture memory and GPU. Now, let's assume you want to upload a texture which is 512x512 pixels. If you want to enable mipmapping, you should upload also 256x256, 128x128, ... , 1x1 copies too (todays GPUs have automatic mipmapping feature on uploads via SGIS_generate_mipmap extension). As you could easily guessed, a typical DDS file can store all mipmap levels

[Jan Ondrus]

-do same thing like for images also for audio (wav model)

-uncompressed audio 8-bit/16-bit mono/stereo is now compressed in separate AUDIO blocks
-wave file is divided into HDR and AUDIO block (images into HDR and IMAGEx)
-extra information (like width for images) saved in compressed stream is BitsPerSample+Channels (possible values are 9, 10, 17, 1
-found and fixed problem for images blocks - metadata (=size of block, image width) were compressed using special models (image models)

[LovePimple]

Thanks Jan!

Compiled...

EDIT: Attachment "paq8px_v14.zip" removed.