QUAD 1.05a is here!

**encode** · 21st February 2007, 22:45

Okay, a new version of QUAD has been released! This version is not about super compression, but fast decompression speed. I think this version is pretty close to the final release which I'm planning to release at Source Forge. However, maybe I'll change something in both directions – compression and/or speed. Anyway, enjoy!

Link:
quad-1.05a.zip (55 KB)

**thometal** · 21st February 2007, 22:56

coool thx

**LovePimple** · 21st February 2007, 23:27

Thanks Ilia!

**Black_Fox1** · 21st February 2007, 23:30

Thanks!

EDIT: 13,478,664 bytes in total!
Details about speeds in next table

**LovePimple** · 22nd February 2007, 01:43

Quick test...

Normal mode:

A10.jpg > 853,522
AcroRd32.exe > 1,503,699
english.dic > 914,252
FlashMX.pdf > 3,832,072
FP.log > 713,280
MSO97.dll > 1,914,337
ohs.doc > 848,925
rafale.bmp > 1,037,418
vcfiu.hlp > 708,539
world95.txt > 627,165

Total = 12,953,209 bytes

Max mode:

A10.jpg > 853,522
AcroRd32.exe > 1,498,051
english.dic > 823,666
FlashMX.pdf > 3,825,201
FP.log > 632,630
MSO97.dll > 1,909,522
ohs.doc > 843,827
rafale.bmp > 1,013,743
vcfiu.hlp > 684,776
world95.txt > 600,075

Total = 12,685,013 bytes

**encode** · 22nd February 2007, 16:54

2 Black_Fox

By the way, I'm very curious about decompression speed of QUAD v1.02a at your test. Currently I'm still thinking about PPM-1 (order-1) with Full Exclusion. QUAD v1.02 uses PPM-1 with no Full Exclusion trick. However, decompression speed must be about the same.

2 All

Note that I also tried to use a pure PPM-2 (order-2) with Full Exclusion – this variant doesn't work, since LZ 'eats' all contexts needed for PPM. For example, PPM cannot be used with baseline LZ77 – since pure LZ destroys all contexts needed for context coding. ROLZ keeps low order context untouched, however, the max order is order-1 or tricky order-2 (which can be described also as an enhanced order-1). Therefore, ROLZ with increased MIN_MATCH_LEN can be useful with high order context coders (Malcolm Taylor implementations).

Some tech details (assume PPM-2X is my enhanced variant of PPM-1):

PPM-1 uses 256 KB
PPM-2X uses 1 MB

Memory usage ~32 MB + 256 KB or 1 MB – i.e. no difference if at all.

I will keep Full Exclusion, since this really helps with binary and already compressed files a la A10.jpg. The speed is not really affected; however, for PPM-2X speed impact is much more noticeable than for PPM-1.

Another question is rescale rate – or speed of model adaptation. With PPM-2X I've found value four is the best, however A10.jpg inflates (~834 KB – anyway, this value is not criminal, I guess). With an order-1 this rescale rate is more important thing, with RR of four we can even slightly compress A10.jpg, however such RR leads poor binary compression.

So anyways, I will run some additional decompression speed tests to find the most efficient solution.

However, I'm looking forward for benchmarking results. By the way, what do you think about compression ratio and decompression speed of QUAD? Is it works for you?

**LovePimple** · 22nd February 2007, 17:33

Quick speed test...

Test file: "mintest.tar" - Contains: html, txt, gif, bmp, exe, dll, etc
Decompressed size: 112 MB (117,768,192 bytes)

Quad v1.05a

Compressed size: 21.9 MB (22,989,829 bytes)

Compression speed: 00:02:41.125

Decompression speed: 00:01:05.343

Quad v1.04a

Compressed size: 19.3 MB (20,274,568 bytes)

Compression speed: 00:03:22.344

Decompression speed: 00:02:02.078

CCM v 1.0.6a2

Compressed size: 21.2 MB (22,289,356 bytes)

Compression speed: 00:02:49.250

Decompression speed: 00:02:44.734

CCM_extra v1.0.6a2

Compressed size: 19.4 MB (20,396,131 bytes)

Compression speed: 00:03:06.109

Decompression speed: 00:03:09.812

**encode** · 22nd February 2007, 17:51

QUAD is good! By the way, soon a very first BETA version will be available! You may ask - so what? This one will be OpenSource...

**encode** · 22nd February 2007, 18:02

http://sourceforge.net/projects/quad/

**LovePimple** · 22nd February 2007, 18:27

Originally Posted by encode

This one will be OpenSource..

**Christian** · 22nd February 2007, 18:36

You announced multi-threading. Is it implemented in this version? On my side it doesnt seem to work.

Originally Posted by encode

By the way, what do you think about compression ratio and decompression speed of QUAD? Is it works for you?

I think the compromise youve found is a really good one! Congratulations to you! Btw. good move to make it OS!

Regards, Christian

**encode** · 22nd February 2007, 18:51

Originally Posted by Christian

You announced multi-threading. Is it implemented in this version? On my side it doesnt seem to work.

The most work with multi-threading are done with my CM. This version of QUAD uses PPM. However, it has some speed improvements over older ROLZ-based versions. Since Im going OpenSource I simplified some parts of code, including I eliminated some experimental MT features of my ROLZ. However, all in process...

**Black_Fox1** · 22nd February 2007, 23:05

Originally Posted by encode

By the way, Im very curious about decompression speed of QUAD v1.02a at your test.

Im off for weekend, will test it ASAP.

**Black_Fox1** · 27th February 2007, 00:55

QUAD 1.02a decompression speed is 2574kB/s on my test

**encode** · 27th February 2007, 14:35

A few notes:
+ QUAD 1.07BETA must be faster than 1.05a
+ At your test (a set of single files) QUAD must be slower. Why?
A brief explanation:
QUAD has a few models:
1. order-2 with 10-bit context
2. order-0 with excluded symbols
3. order-0
4. Stationary model for match indexes
In this explanation I'll omit the last model and will keep all eyes on decoder.

Principal scheme of PPM:
If order-2 context exists then
Decode symbol via order-2 context

If symbol == Escape Symbol then
Generate a new counter that based on order-0 counter, but has no symbols that has order-2

Decode symbol via new counter with excluded symbols

Add symbol to order-0 counter
Add escape symbol to order-2 counter
Add symbol to order-2 counter

Else
Add symbol to order-2 counter
End if
Else
Decode symbol via order-0 counter

Add symbol to order-0 counter
Add escape symbol to order-2 counter
Add symbol to order-2 counter
End if

The "bottleneck" of this code is the process of generating a new counter with excluded symbols. At start of decoding relatively at each symbol coded we invoke such routine, after some data decoded and model gets more statistics, we use order-0 with excluded symbols more rarely – since symbols in order-2 contexts are already seen. At some point of decoding decoder stars using just order-2. So, if we decompressing many files, each time we start decompressing decoder uses many statistics from order-0 with excluded symbols – as a result noticeable speed impact.

To be honest, such trick as Full Exclusion is incompatible with speed. However, here I use a Small PPM, and on large enough files the speed impact is not so huge.

Note that in upcoming QUAD 1.08BETA I will improve some parts of code, including PPM encoder and decoder, since in 1.07BETA, some parts of PPM, to be honest, inefficiently implemented. So, expecting a new speed improvements...

**Fallon** · 27th February 2007, 23:31

Looking forward to the next version.

**encode** · 4th March 2007, 23:22

By the way, next QUAD 1.08 (a stable release) has a higher compression! Along with a higher compression/decompression speed, an additional error checking and many additional improvements...

As a proof, download the 'world95.txt' compressed to 596,582 bytes:
world95.txt.quad

Thread: QUAD 1.05a is here!

Thread Tools

Search Thread

Display

Similar Threads

Quad 1.05a overview

Posting Permissions