Compression speed benchmark

[Sportman]

I measured compression speeds for some engines. Test system quad core QX9650 4,2Ghz+ 45nm, 2GB DDR3-1600+, 3 x 32GB SSD disk RAID0 (280MB/s read/write, 0,1ms) and OS 64bit Windows 2008 server. As input I used one big text file and time limit round 5 minutes for each compressor.

Input:
1.552.758.784 bytes

Output:

size bytes, ratio, hh:mm:ss (compressor version level)

791,106,298 bytes 1.96x 00:00:17 (tornado 0.3 -1)
726,315,671 bytes 2.14x 00:00:15 (lzop 1.01 -1)
724,024,574 bytes 2.14x 00:00:15 (thor 0.96 e1)
694,267,304 bytes 2.24x 00:00:15 (quicklz 1.4.0 b7 1)
642,443,977 bytes 2.42x 00:00:28 (brieflz)
620,349,828 bytes 2.50x 00:00:13 (lzturbo -11)
619,945,577 bytes 2.50x 00:00:16 (quicklz 1.4.0 b7 2)
606,425,265 bytes 2.56x 00:00:50 (quicklz 1.4.0 b7 3)
601,618,370 bytes 2.58x 00:00:11 (slug 1.1b)
598,106,700 bytes 2.60x 00:13:23 (lzw 0.2)
587,980,752 bytes 2.64x 00:00:16 (thor 0.96 e2)
573,256,474 bytes 2.71x 00:00:18 (tornado 0.3 -2)
571,336,015 bytes 2.72x 00:00:38 (rar 3.71 zip fastest)
565,694,630 bytes 2.74x 00:03:34 (lzop 1.01 -9)
539,971,642 bytes 2.88x 00:00:50 (rar 3.71 zip fast)
520,129,208 bytes 2.99x 00:00:20 (thor 0.96 e3)
518,948,547 bytes 2.99x 00:00:20 (arc 0.4 -1)
518,948,306 bytes 2.99x 00:00:20 (tornado 0.3 -3)
513,166,014 bytes 3.03x 00:00:55 (7za 4.57 zip 1)
513.165.918 bytes 3.03x 00:00:56 (7za 4.57 gzip 1)
505,509,998 bytes 3.07x 00:00:39 (rar 3.71 -1)
494,177,370 bytes 3.14x 00:01:37 (rar 3.71 zip normal)
494,121,095 bytes 3.14x 00:02:01 (rar 3.71 zip good)
494,116,089 bytes 3.14x 00:02:10 (rar 3.71 zip max)
493,674,645 bytes 3.15x 00:02:14 (cabarc 1.0 default)
488,112,115 bytes 3.18x 00:00:52 (7za 4.57 bzip2 1)
482,307,196 bytes 3.22x 00:00:59 (thor 0.96 e5)
480,395,200 bytes 3.23x 00:00:15 (lzturbo 0.9 -51)
478,473,307 bytes 3.25x 00:01:58 (mcomp 1.0 -d64f)
477,168,430 bytes 3.25x 00:03:00 (7za 4.57 zip 5)
477,168,334 bytes 3.25x 00:03:01 (7za 4.57 gzip 5)
467,415,045 bytes 3.32x 00:01:44 (7za 4.57 7z 1)
456,133,689 bytes 3.26x 00:00:35 (flashzip 0.02)
447,968,986 bytes 3.47x 00:00:28 (tornado 0.3 -4)
446,796,676 bytes 3.48x 00:08:11 (lzxq 0.2)
445,295,202 bytes 3.49x 00:00:54 (sr2)
439,925,432 bytes 3.53x 00:00:35 (thor 0.96 e4)
436,142,695 bytes 3.56x 00:00:14 (lzturbo 0.9 -52)
424,431,052 bytes 3.66x 00:01:13 (7za 4.57 bzip 5)
415,584,164 bytes 3.74x 00:00:16 (lzturbo 0.9 -53)
391,134,937 bytes 3.97x 00:00:41 (tornado 0.3 -5)
390,940,115 bytes 3.80x 00:01:18 (ring v1.2)
390,548,501 bytes 3.98x 00:00:30 (lzturbo 0.9 -54)
381,395,174 bytes 4.07x 00:00:41 (lzturbo 0.9 -55)
373,998,196 bytes 4.15x 00:00:58 (lzturbo 0.9 -56)
368,221,758 bytes 4.22x 00:01:26 (lzturbo 0.9 -57)
366,424,863 bytes 4.24x 00:02:38 (rar 3.71 -2)
363,984,157 bytes 4.27x 00:02:15 (lzturbo 0.9 -5
361,782,952 bytes 4.29x 00:00:49 (tornado 0.3 -6)
356,995,472 bytes 4.35x 00:02:52 (lzpm 0.15 1)
356,757,580 bytes 4.35x 00:03:58 (rar 3.71 -3)
354,955,940 bytes 4.37x 00:01:21 (arc 0.4 -m2)
350,655,035 bytes 4.43x 00:02:42 (sbc 0.97 -m1)
338,651,096 bytes 4.58x 00:03:07 (sbc 0.97 -m2)
334,725,101 bytes 4.64x 00:06:25 (lzpm 0.15 2)
334,050,397 bytes 4.65x 00:01:08 (tornado 0.3 -7)
324,846,207 bytes 4.78x 00:04:01 (rar 3.71 -4)
324,328,498 bytes 4.79x 00:04:42 (rar 3.71 -5)
322,372,806 bytes 4.82x 00:03:41 (lzturbo 0.9 -59)
313,161,387 bytes 4.96x 00:01:39 (tornado 0.3 -
297,948,640 bytes 5.21x 00:03:25 (sbc 0.97 -m3)
295,739,455 bytes 5.25x 00:04:45 (mcomp 1.0 -ff)
294,980,095 bytes 5.26x 00:05:49 (ccm 1.3 0)
293,323,778 bytes 5.29x 00:02:46 (tornado 0.3 -9)
293,161,091 bytes 5,30x 00:05:31 (mcomp 1.0 -f3f)
285,264,394 bytes 5.44x 00:08:15 (7za 4.57 7z 5)
281,340,239 bytes 5.52x 00:03:58 (arc 0.4 -m3)
277,663,310 bytes 5.59x 00:04:19 (tornado 0.3 -10)
266,715,072 bytes 5.82x 00:09:03 (tornado 0.3 -11)
263,048,551 bytes 5.90x 00:07:49 (durilca light 0.5 default)
261,024,407 bytes 5.95x 00:04:13 (arc 0.4 -m4)
255,807,105 bytes 6.07x 00:04:30 (arc 0.4 -m5)
236,817,364 bytes 6.56x 00:04:32 (arc 0.4 -m6)
232,225,864 bytes 6.69x 00:04:54 (arc 0.4 -m7)

[Bulat Ziganshin]

test where tornado outperforms 7zip doesn't look serious. it's prettu obvious that your data contains too much repetitions at longs distances (and tornado/fa with their gigantic dictionaries is the only programs that found these repetitions)

[Sportman]

I wrote a very simple compressor to compress this type of data what build up a keyword dictionary; 839.968.787 bytes 1.85x 00:14:28 (kwc 0.0.3.0). It find 15,931,475 unique keywords (length 6 bytes) from total 258,793,131 keywords (length 6 bytes). So yes it has repeatings but that has every big real world text file.

[nimdamsk]

Here are results filtered with Pareto frontier with Mbps added:

601,618,370 bytes 2.58x 00:00:11 134,62 Mbps (slug 1.1b) 
436,142,695 bytes 3.56x 00:00:14 105,77 Mbps (lzturbo 0.9 -52) 
415,584,164 bytes 3.74x 00:00:16 92,55 Mbps  (lzturbo 0.9 -53) 
390,548,501 bytes 3.98x 00:00:30 49,36 Mbps  (lzturbo 0.9 -54) 
381,395,174 bytes 4.07x 00:00:41 36,12 Mbps  (lzturbo 0.9 -55) 
361,782,952 bytes 4.29x 00:00:49 30,22 Mbps  (tornado 0.3 -6) 
334,050,397 bytes 4.65x 00:01:08 21,78 Mbps  (tornado 0.3 -7) 
313,161,387 bytes 4.96x 00:01:39 14,96 Mbps  (tornado 0.3 -8) 
293,323,778 bytes 5.29x 00:02:46 8,92 Mbps   (tornado 0.3 -9) 
281,340,239 bytes 5.52x 00:03:58 6,22 Mbps   (arc 0.4 -m3) 
261,024,407 bytes 5.95x 00:04:13 5,85 Mbps   (arc 0.4 -m4) 
255,807,105 bytes 6.07x 00:04:30 5,48 Mbps   (arc 0.4 -m5) 
236,817,364 bytes 6.56x 00:04:32 5,44 Mbps   (arc 0.4 -m6) 
232,225,864 bytes 6.69x 00:04:54 5 Mbps      (arc 0.4 -m7)

[Nania Francesco]

Please , if possible , test Winturtle 1.6.0! Hi

[SvenBent]

I'm more interested in decompression time

[Black_Fox1]

Well, ARC is symmetric, SLUG is always fast and Tornado is highly assymetric (by looking at LTCB I guess 4-7x faster decompression - depending on -# param)

[Bulat Ziganshin]

Tornado is highly assymetric

tornado here will decompress in ~20 seconds

anyway, these results by no means are representative

So yes it has repeatings but that has every big real world text file.

the question is AMOUNT of this repetition. all-spaces file is also has repeatings, but its a bit too much repetitions. tornado outperforming lzturbo/lzma, arc significantly outperforming ccm are definitely not usual situations - ive seen and done a lot of tests

my own results on primarily text data:

<div class=""jscript""><pre>ACE 2.04 -d4096 -s
-m1 -d64 4.652 1.047 3.981
-m5 5.563 0.545 4.129
SBC 0.970 -of
-m1 -b5 6.038 1.442 3.664
-m2 -b15 6.215 0.989 3.130
-m3 -b63 6.461 0.572 2.220
RAR 3.70 -md4096 -s
-m1 4.111 3.944 18.849
-m2 5.463 1.180 21.774
-m3 5.672 0.830 21.696
-m5 -mcd- 6.272 0.757 1.858
-m5 6.315 0.759 1.863
-m5 -mc14:128t 6.568 0.673 1.453
Squeez 5.2
-uxx1 6.313 0.548 2.323
-uxx5 6.353 0.289 2.322
-uxx9 6.354 0.217 2.322
7-zip 4.52
-mx1 4.865 2.192 13.747
-mx3 5.328 1.736 14.883
-mx5 6.560 0.439 16.882
-mx7 6.819 0.353 17.110
-mx9 -md=32m 6.842 0.355 16.988
ARC 0.40
-m1x 4.164 7.469 21.085
-m2x 5.128 4.678 18.575
-m3x 6.372 1.728 11.461
-m4x 7.120 0.747 11.848
-m5x 7.209 0.615 11.940
-m6x 7.213 0.613 11.966

-m2 6.144 3.324 3.723
-m2r 6.148 2.914 3.581
-m3r 6.557 1.654 3.464
-m3 7.043 1.306 2.044
-m4 7.419 0.995 1.923
-m5 7.561 0.864 1.718
-m6 7.609 0.810 1.585

-m6 -mcd- 7.566 0.828 1.585
-m5p 8.345 0.344 0.469
-m6p 8.494 0.307 0.411
ARC externals
-mdul0 7.166 1.010 1.086
-mdul 7.117 0.229 1.007
-mccm 7.842 0.392
-mccmx 8.046 0.296
-mlpaq 8.660 0.095
-mdur 8.464 0.065
-muda 8.943 0.019
UHARC 0.6 -md32768
-mz 5.611 2.662 2.552
-m1 6.145 0.417 4.618
-m2 6.448 0.295 4.958
-m3 6.582 0.254 5.058
-mx 7.404 0.364 0.402
</pre>[/QUOTE]


here first column is compression ratio (1:N), second - c.speed, mb/sec, third - decompression speed


[QUOTE]<div class=""quoting"">Quoting: nimdamsk</div>Here are results filtered with Pareto frontier with Mbps added:</div>
can you add p.f. with lzturbo results multiplied by 4?

[nimdamsk]

can you add p.f. with lzturbo results multiplied by 4?

Sure Thus lzturbo is out of PF. Even if lzturbo is similar to tornado, except using multiple cores, decompression speed of both archivers should be the same.

<div class=""jscript""><pre>601,618,370 bytes 2.58x 00:00:11 134,62 Mbps (slug 1.1b)
436,142,695 bytes 3.56x 00:00:56 26,44 Mbps (lzturbo 0.9 -52)*4
415,584,164 bytes 3.74x 00:00:64 23,14 Mbps (lzturbo 0.9 -53)*4
390,548,501 bytes 3.98x 00:00:120 12,34 Mbps (lzturbo 0.9 -54)*4
381,395,174 bytes 4.07x 00:00:164 9,03 Mbps (lzturbo 0.9 -55)*4
361,782,952 bytes 4.29x 00:00:49 30,22 Mbps (tornado 0.3 -6)
334,050,397 bytes 4.65x 00:01:08 21,78 Mbps (tornado 0.3 -7)
313,161,387 bytes 4.96x 00:01:39 14,96 Mbps (tornado 0.3 -
293,323,778 bytes 5.29x 00:02:46 8,92 Mbps (tornado 0.3 -9)
281,340,239 bytes 5.52x 00:03:58 6,22 Mbps (arc 0.4 -m3)
261,024,407 bytes 5.95x 00:04:13 5,85 Mbps (arc 0.4 -m4)
255,807,105 bytes 6.07x 00:04:30 5,48 Mbps (arc 0.4 -m5)
236,817,364 bytes 6.56x 00:04:32 5,44 Mbps (arc 0.4 -m6)
232,225,864 bytes 6.69x 00:04:54 5 Mbps (arc 0.4 -m7)
[/code]

[Sportman]

Under Windows 2008 server 64bit I installed with Hyper-V a virtual Windows 2008 server 32bit core only version (to get a simulair size as last text benchmark) and logged in once. The virtual disk created I used as input to test all programs again for compression speed at the same hardware. Some lzturbo results are missing because errors or crash during compression (maybe to less memory available). I added winturtle what scored 303,958,161 bytes 5.11x 00:01:32 (winturtle 1.6.0b) at the text file.

Input:
1,489,400,832 bytes

Output:

size bytes, ratio, hh:mm:ss (compressor version level)

831,814,903 bytes 1.79x 00:00:15 (tornado 0.3 -1)
768,161,732 bytes 1.94x 00:00:16 (quicklz 1.4.0 b7 1)
753,891,792 bytes 1.98x 00:06:30 (lzw 0.2)
745,446,424 bytes 2.00x 00:00:19 (lzop 1.01 -1)
738,273,233 bytes 2.02x 00:00:21 (brieflz)
710,091,298 bytes 2.10x 00:00:13 (thor 0.96 e1)
688,118,239 bytes 2.16x 00:00:16 (quicklz 1.4.0 b7 2)
680,858,790 bytes 2.19x 00:00:11 (slug 1.1b)
670,786,076 bytes 2.22x 00:00:14 (thor 0.96 e2)
649,742,230 bytes 2.29x 00:00:47 (quicklz 1.4.0 b7 3)
646,055,708 bytes 2.31x 00:00:12 (lzturbo -11)
637,926,485 bytes 2.33x 00:04:23 (lzop 1.01 -9)
632,169,343 bytes 2.36x 00:00:40 (rar 3.71 zip fastest)
613,495,011 bytes 2.43x 00:00:47 (rar 3.71 zip fast)
601,092,170 bytes 2.48x 00:00:19 (tornado 0.3 -2)
595,691,408 bytes 2.50x 00:00:43 (7za 4.57 zip 1)
595,691,318 bytes 2.50x 00:00:43 (7za 4.57 gzip 1)
589,189,472 bytes 2.53x 00:01:17 (rar 3.71 zip normal)
587,140,526 bytes 2.54x 00:02:09 (rar 3.71 zip good)
586,896,424 bytes 2.54x 00:02:59 (rar 3.71 zip max)
586,226,715 bytes 2.54x 00:02:39 (cabarc 1.0 default)
584,047,711 bytes 2.55x 00:01:33 (mcomp 1.0 -d64f)
572,380,727 bytes 2.60x 00:02:19 (7za 4.57 zip 5)
572,380,637 bytes 2.60x 00:02:30 (7za 4.57 gzip 5)
569,328,884 bytes 2.62x 00:00:48 (7za 4.57 bzip2 1)
567,291,160 bytes 2.63x 00:00:46 (thor 0.96 e5)
538,826,265 bytes 2.76x 00:00:21 (arc 0.4 -1)
538,826,030 bytes 2.76x 00:00:20 (tornado 0.3 -3)
515,656,085 bytes 2.89x 00:00:56 (sr2)
513,191,748 bytes 2.90x 00:01:38 (7za 4.57 7z 1)
501,195,648 bytes 2.07x 00:00:14 (thor 0.96 e3)
498,784,541 bytes 2.99x 00:01:10 (7za 4.57 bzip 5)
489,538,813 bytes 3.04x 00:02:53 (sbc 0.97 -m1)
481,363,647 bytes 3.10x 00:00:39 (rar 3.71 -1)
462,569,128 bytes 3.22x 00:00:38 (flashzip 0.02)
455,525,927 bytes 3.27x 00:06:53 (lzxq 0.2)
440,985,058 bytes 3.38x 00:01:45 (rings v1.2)
389,759,379 bytes 3.82x 00:00:50 (lzturbo 0.9 -54)
389,596,530 bytes 3.82x 00:00:19 (tornado 0.3 -4)
388,195,728 bytes 3.84x 00:00:29 (thor 0.96 e4)
387,333,145 bytes 3.85x 00:00:58 (lzturbo 0.9 -55)
360,859,944 bytes 4.13x 00:13:23 (durilca light 0.5 default)
354,737,867 bytes 4.20x 00:02:52 (sbc 0.97 -m2)
354,650,498 bytes 4.20x 00:01:48 (lzpm 0.15 1)
352,941,919 bytes 4.22x 00:02:43 (rar 3.71 -2)
352,723,630 bytes 4.22x 00:22:32 (lzturbo 0.9 -59)
349,715,271 bytes 4.26x 00:02:36 (lzpm 0.15 2)
341,669,940 bytes 4.36x 00:19:55 (lzpm 0.15 9)
332,963,696 bytes 4.47x 00:00:35 (arc 0.4 -m2)
332,963,461 bytes 4.47x 00:00:33 (tornado 0.3 -5)
329,071,086 bytes 4.53x 00:02:49 (rar 3.71 -3)
327,799,448 bytes 4.54x 00:03:02 (rar 3.71 -4)
326,301,547 bytes 4.56x 00:04:17 (rar 3.71 -5)
318,514,886 bytes 4.68x 00:01:36 (winturtle 1.6.0b)
304,811,555 bytes 4.89x 00:00:41 (tornado 0.3 -6)
296,997,201 bytes 5.01x 00:00:58 (tornado 0.3 -7)
294,206,636 bytes 5.06x 00:05:09 (mcomp 1.0 -ff)
289,722,661 bytes 5.14x 00:01:25 (tornado 0.3 -
287,744,894 bytes 5.18x 00:05:18 (ccm 1.3 0)
285,990,844 bytes 5.21x 00:01:51 (arc 0.4 -m3)
282,633,636 bytes 5.27x 00:02:11 (tornado 0.3 -9)
281,130,137 bytes 5.30x 00:04:16 (sbc 0.97 -m3)
278,867,672 bytes 5.34x 00:03:22 (tornado 0.3 -10)
275,726,299 bytes 5.40x 00:07:52 (tornado 0.3 -11)
273,436,840 bytes 5.45x 00:13:25 (tornado 0.3 -12)
269,655,647 bytes 5.52x 00:04:05 (mcomp 1.0 -f3f)
260,483,319 bytes 5.72x 00:03:24 (7za 4.57 7z 5)
257,950,551 bytes 5.77x 00:03:26 (arc 0.4 -m4)
249,870,590 bytes 5.96x 00:04:39 (arc 0.4 -m5)
246,251,355 bytes 6.05x 00:05:04 (arc 0.4 -m6)
241,521,886 bytes 6.17x 00:05:47 (arc 0.4 -m7)

[Bulat Ziganshin]

can you add p.f. with lzturbo results multiplied by 4?
Sure

you just multiplied lzturbo results by 4 i mean that single-core PF will include _other_ algorithms, either thor or tornado

although anyway its better to ask Sportman to repeat his tests with lzturbo -p1 and wider variety of compression settings - including -12, -43 and so on

[nimdamsk]

you just multiplied lzturbo results by 4

OK, ive got the idea
<div class=""jscript""><pre>601,618,370 bytes 2.58x 00:00:11 134,62 Mbps (slug 1.1b)
587,980,752 bytes 2.64x 00:00:16 92,55 Mbps (thor 0.96 e2)
573,256,474 bytes 2.71x 00:00:18 82,27 Mbps (tornado 0.3 -2)
518,948,306 bytes 2.99x 00:00:20 74,04 Mbps (tornado 0.3 -3)
447,968,986 bytes 3.47x 00:00:28 52,89 Mbps (tornado 0.3 -4)
439,925,432 bytes 3.53x 00:00:35 42,31 Mbps (thor 0.96 e4)
391,134,937 bytes 3.97x 00:00:41 36,12 Mbps (tornado 0.3 -5)
361,782,952 bytes 4.29x 00:00:49 30,22 Mbps (tornado 0.3 -6)
334,050,397 bytes 4.65x 00:01:08 21,78 Mbps (tornado 0.3 -7)
313,161,387 bytes 4.96x 00:01:39 14,96 Mbps (tornado 0.3 -
293,323,778 bytes 5.29x 00:02:46 8,92 Mbps (tornado 0.3 -9)
281,340,239 bytes 5.52x 00:03:58 6,22 Mbps (arc 0.4 -m3)
261,024,407 bytes 5.95x 00:04:13 5,85 Mbps (arc 0.4 -m4)
255,807,105 bytes 6.07x 00:04:30 5,48 Mbps (arc 0.4 -m5)
236,817,364 bytes 6.56x 00:04:32 5,44 Mbps (arc 0.4 -m6)
232,225,864 bytes 6.69x 00:04:54 5 Mbps (arc 0.4 -m7)
[/code]

[Nania Francesco]

I added winturtle what scored 303,958,161 bytes 5.11x 00:01:32 (winturtle 1.6.0b) at the text file.

HI!

[Sportman]

although anyway its better to ask Sportman to repeat his tests with lzturbo -p1 and wider variety of compression settings - including -12, -43 and so on

The lzturbo single core compression results for the text file:

620,349,828 bytes 00:00:24 (lzturbo 0.9 -11)
577,393,628 bytes 00:00:26 (lzturbo 0.9 -21)
538,775,874 bytes 00:00:27 (lzturbo 0.9 -31)
514,256,524 bytes 00:00:26 (lzturbo 0.9 -12)
501,936,142 bytes 00:00:31 (lzturbo 0.9 -41)
486,427,888 bytes 00:00:41 (lzturbo 0.9 -13)
480,395,200 bytes 00:00:40 (lzturbo 0.9 -51)
471,614,418 bytes 00:00:28 (lzturbo 0.9 -22)
459,685,779 bytes 00:00:29 (lzturbo 0.9 -32)
455,784,287 bytes 00:01:57 (lzturbo 0.9 -15)
455,348,249 bytes 00:01:28 (lzturbo 0.9 -14)
444,951,768 bytes 00:00:43 (lzturbo 0.9 -23)
442,801,215 bytes 00:00:30 (lzturbo 0.9 -42)
439,577,736 bytes 00:02:37 (lzturbo 0.9 -16)
436,639,115 bytes 00:00:43 (lzturbo 0.9 -33)
436,366,927 bytes 00:03:44 (lzturbo 0.9 -17)
436,142,695 bytes 00:00:36 (lzturbo 0.9 -52)
435,412,948 bytes 00:05:35 (lzturbo 0.9 -1
420,251,155 bytes 00:00:44 (lzturbo 0.9 -43)
417,937,457 bytes 00:01:33 (lzturbo 0.9 -24)
415,584,164 bytes 00:00:50 (lzturbo 0.9 -53)
411,747,799 bytes 00:01:31 (lzturbo 0.9 -34)
409,358,200 bytes 00:02:03 (lzturbo 0.9 -25)
403,799,548 bytes 00:01:58 (lzturbo 0.9 -35)
403,187,289 bytes 00:02:48 (lzturbo 0.9 -26)
399,933,429 bytes 00:10:22 (lzturbo 0.9 -19)
399,042,600 bytes 00:04:03 (lzturbo 0.9 -27)
398,138,723 bytes 00:02:40 (lzturbo 0.9 -36)
396,500,353 bytes 00:06:10 (lzturbo 0.9 -2
394,451,231 bytes 00:01:32 (lzturbo 0.9 -44)
394,412,850 bytes 00:03:47 (lzturbo 0.9 -37)
392,205,980 bytes 00:05:38 (lzturbo 0.9 -3
390,548,501 bytes 00:01:37 (lzturbo 0.9 -54)
385,074,655 bytes 00:01:58 (lzturbo 0.9 -45)
381,395,174 bytes 00:02:05 (lzturbo 0.9 -55)
377,494,823 bytes 00:02:40 (lzturbo 0.9 -46)
374,203,844 bytes 00:10:55 (lzturbo 0.9 -29)
373,988,196 bytes 00:02:47 (lzturbo 0.9 -56)
371,597,088 bytes 00:03:45 (lzturbo 0.9 -47)
368,221,395 bytes 00:03:54 (lzturbo 0.9 -57)
367,263,752 bytes 00:05:34 (lzturbo 0.9 -4
363,984,157 bytes 00:05:46 (lzturbo 0.9 -5
363,150,187 bytes 00:11:00 (lzturbo 0.9 -39)
334,517,059 bytes 00:11:01 (lzturbo 0.9 -49)
322,372,806 bytes 00:11:31 (lzturbo 0.9 -59)

[Bulat Ziganshin]

thanks, Nimdamsk and Sportman

afaiqs, this adds one more line to single-core P.F.:
436,142,695 bytes 00:00:36 (lzturbo 0.9 -52)

[Sportman]

The lzturbo single core compression results for the virtual disk file:

646,055,708 bytes 00:00:20 (lzturbo 0.9 -11)
622,925,404 bytes 00:00:20 (lzturbo 0.9 -21)
616,103,903 bytes 00:00:22 (lzturbo 0.9 -31)
537,165,162 bytes 00:00:29 (lzturbo 0.9 -41)
478,021,920 bytes 00:00:18 (lzturbo 0.9 -12)
454,598,316 bytes 00:00:38 (lzturbo 0.9 -13)
447,601,002 bytes 00:00:21 (lzturbo 0.9 -32)
446,322,634 bytes 00:00:18 (lzturbo 0.9 -22)
442,372,989 bytes 00:01:26 (lzturbo 0.9 -14)
439,253,036 bytes 00:01:47 (lzturbo 0.9 -15)
437,239,596 bytes 00:02:23 (lzturbo 0.9 -16)
435,912,111 bytes 00:04:13 (lzturbo 0.9 -17)
424,018,180 bytes 00:00:39 (lzturbo 0.9 -33)
419,459,611 bytes 00:00:38 (lzturbo 0.9 -23)
413,490,715 bytes 00:00:19 (lzturbo 0.9 -42)
413,450,692 bytes 00:01:26 (lzturbo 0.9 -34)
413,353,157 bytes 00:52:30 (lzturbo 0.9 -19)
410,864,473 bytes 00:01:50 (lzturbo 0.9 -35)
409,169,120 bytes 00:02:26 (lzturbo 0.9 -36)
408,314,603 bytes 00:01:25 (lzturbo 0.9 -24)
408,054,516 bytes 00:03:42 (lzturbo 0.9 -37)
405,608,879 bytes 00:01:51 (lzturbo 0.9 -25)
403,832,111 bytes 00:02:29 (lzturbo 0.9 -26)
402,657,908 bytes 00:03:48 (lzturbo 0.9 -27)
392 095,907 bytes 00:00:43 (lzturbo 0.9 -43)
389,759,379 bytes 00:02:05 (lzturbo 0.9 -54)
387,333,145 bytes 00:02:29 (lzturbo 0.9 -55)
385,647,069 bytes 00:03:07 (lzturbo 0.9 -56)
384,493,527 bytes 00:04:21 (lzturbo 0.9 -57)
381,522,925 bytes 00:01:21 (lzturbo 0.9 -44)
378,749,753 bytes 00:01:48 (lzturbo 0.9 -45)
376,804,039 bytes 00:02:25 (lzturbo 0.9 -46)
375,461,281 bytes 00:03:35 (lzturbo 0.9 -47)

wrong file size (lzturbo 0.9 -5
wrong file size (lzturbo 0.9 -4
wrong file size (lzturbo 0.9 -3
wrong file size (lzturbo 0.9 -2
wrong file size (lzturbo 0.9 -1
skipped to long time (lzturbo 0.9 -59)
skipped to long time (lzturbo 0.9 -49)
skipped to long time (lzturbo 0.9 -39)
skipped to long time (lzturbo 0.9 -29)
crash (lzturbo 0.9 -53)
crash (lzturbo 0.9 -52)
crash (lzturbo 0.9 -51)

[Sportman]

For the third compression speed test I used one big Apache log file (combined log format) from 5 full days (only one front-end server), durilca light 0.5 crashed and slim023d gave a wrong output file size, test platform same as in the first two tests.

Input:
2,072,612,014 bytes

Output:

size bytes, ratio, hh:mm:ss (compressor version level)

325,897,962 bytes 06.36x 00:00:40 (lzc 0.08 1)
322,723,099 bytes 06.42x 00:00:10 (lzop 1.01 -1)
314,724,768 bytes 06.59x 00:00:22 (brieflz)
289,782,256 bytes 07.15x 00:00:35 (lzc 0.08 2)
283,882,865 bytes 07.30x 00:00:14 (tornado 0.3 -1)
273,922,238 bytes 07.57x 00:00:19 (thor 0.96 e1)
263,109,969 bytes 07.88x 00:00:14 (slug 1.1b)
261,398,846 bytes 07.93x 00:00:33 (lzc 0.08 3)
256,701,817 bytes 08.07x 00:00:21 (rar 3.71 zip fastest)
251,764,369 bytes 08.23x 00:00:20 (quicklz 1.4.0 b7 1)
251,453,496 bytes 08.24x 00:00:19 (thor 0.96 e2)
239,752,615 bytes 08.64x 00:00:31 (lzc 0.08 4)
236,220,977 bytes 08.77x 00:00:22 (rar 3.71 zip fast)
231,638,114 bytes 08.95x 00:00:14 (tornado 0.3 -2)
223,116,434 bytes 09.29x 00:00:31 (lzc 0.08 5)
220,163,423 bytes 09.41x 00:01:46 (lzop 1.01 -9)
218,305,275 bytes 09.49x 00:00:38 (quicklz 1.4.0 b7 3)
210,457,429 bytes 09.85x 00:00:44 (lzc 0.08 6)
205,751,464 bytes 10.07x 00:00:27 (7za 4.57 zip 1)
205,751,375 bytes 10.07x 00:00:27 (7za 4.57 gzip 1)
201,161,165 bytes 10.30x 00:00:19 (quicklz 1.4.0 b7 2)
200,546,858 bytes 10.33x 00:00:54 (lzc 0.08 7)
192,488,505 bytes 10.77x 00:01:07 (lzc 0.08
191,253,575 bytes 10.84x 00:00:13 (arc 0.4 -m1)
191,253,342 bytes 10.84x 00:00:14 (tornado 0.3 -3)
190,244,634 bytes 10.89x 00:00:46 (rar 3.71 zip normal)
188,726,409 bytes 10.98x 00:02:45 (7za 4.57 zip 5)
188,726,320 bytes 10.98x 00:02:46 (7za 4.57 gzip 5)
187,066,762 bytes 11.08x 00:00:55 (cabarc 1.0 default)
185,857,134 bytes 11.15x 00:01:19 (lzc 0.08 9)
183,376,720 bytes 11.30x 00:00:31 (rar 3.71 -1)
182,815,283 bytes 11.34x 00:01:00 (rar 3.71 zip good)
182,474,854 bytes 11.36x 00:01:02 (rar 3.71 zip max)
179,955,463 bytes 11.52x 00:01:30 (lzc 0.08 10)
178,494,355 bytes 11.61x 00:00:17 (lzturbo 0.91 -51)
178,494,355 bytes 11.61x 00:00:17 (lzturbo 0.91 -11)
177,843,226 bytes 11.65x 00:00:12 (tornado 0.3 -4)
176,932,825 bytes 11.71x 00:01:10 (7za 4.57 bzip2 1)
176,402,456 bytes 11.75x 00:01:05 (mcomp 1.0 -d64f)
173,182,224 bytes 11.97x 00:00:17 (lzturbo 0.91 -52)
172,347,376 bytes 12.03x 00:00:25 (thor 0.96 e5)
170,380,209 bytes 12.16x 00:00:48 (7za 4.57 7z 1)
167,843,880 bytes 12.35x 00:00:19 (thor 0.96 e3)
151,762,191 bytes 13.66x 00:00:18 (tornado 0.3 -5)
149,288,413 bytes 13.88x 00:00:18 (lzturbo 0.91 -53)
147,827,042 bytes 14.02x 00:10:35 (lzw 0.2)
144,175,249 bytes 14.38x 00:00:23 (tornado 0.3 -6)
140,021,579 bytes 14.80x 00:00:19 (lzturbo 0.91 -54)
134,604,252 bytes 15.40x 00:00:19 (lzturbo 0.91 -55)
133,290,489 bytes 15.55x 00:00:41 (tornado 0.3 -7)
131,385,399 bytes 15.78x 00:00:27 (lzturbo 0.91 -56)
129,543,276 bytes 16.00x 00:02:14 (lzxq 0.2)
129,181,254 bytes 16.04x 00:00:31 (lzturbo 0.91 -57)
127,898,180 bytes 16.21x 00:01:05 (tornado 0.3 -
126,935,084 bytes 16.33x 00:00:22 (flashzip 0.02)
125,875,055 bytes 16.47x 00:00:56 (lzturbo 0.91 -5
123,656,031 bytes 16.76x 00:03:44 (winace 2.69 superfast)
122,758,379 bytes 16.88x 00:03:51 (winace 2.69 fast)
122,147,496 bytes 16.97x 00:02:06 (tornado 0.3 -9)
120,998,466 bytes 17.13x 00:01:48 (rar 3.71 -2)
120,445,178 bytes 17.21x 00:04:36 (winace 2.69 normal)
120,288,520 bytes 17.23x 00:00:19 (thor 0.96 e4)
120,198,851 bytes 17.24x 00:04:44 (winace 2.69 good)
120,075,719 bytes 17.26x 00:04:54 (winace 2.69 maximum)
119,054,729 bytes 17.41x 00:03:25 (tornado 0.3 -10)
117,783,846 bytes 17.60x 00:03:54 (7za 4.57 7z 5)
117,049,609 bytes 17.71x 00:08:16 (tornado 0.3 -11)
116,639,789 bytes 17.77x 00:02:28 (rar 3.71 -3)
115,607,357 bytes 17.93x 00:15:42 (tornado 0.3 -12)
115,223,851 bytes 17.99x 00:00:34 (sr2)
106,277,359 bytes 19.50x 00:02:40 (mcomp 1.0 -ff)
105,939,471 bytes 19.56x 00:02:49 (mcomp 1.0 -f3f)
104,456,261 bytes 19.84x 00:01:47 (7za 4.57 bzip2 5)
103,156,939 bytes 20.09x 00:05:13 (lzturbo 0.91 -59)
100.256.643 bytes 20.67x 00:02:19 (hook 1.3 55)
099,850,064 bytes 20.76x 00:05:55 (lzpm 0.15 1)
099,084,865 bytes 20.92x 00:00:30 (flashzip 0.03 1)
098,877,380 bytes 20.96x 00:00:57 (rings v1.2)
096,970,828 bytes 21.37x 00:09:52 (lzpm 0.15 2)
095,163,053 bytes 21.78x 00:00:41 (sr3)
094,531,596 bytes 21.93x 00:00:32 (flashzip 0.03 2)
094,027,513 bytes 22.04x 00:00:44 (winturtle 1.6.0b)
093,198,609 bytes 22.24x 00:02:42 (sbc 0.97 -m3)
092,711,839 bytes 22.36x 00:02:27 (sbc 0.97 -m2)
092,710,862 bytes 22.36x 00:02:34 (rar 3.71 -4)
092,402,445 bytes 22.43x 00:00:34 (arc 0.4 -m2)
092,226,953 bytes 22.47x 00:02:14 (sbc 0.97 -m1)
091,963,302 bytes 22.54x 00:00:35 (flashzip 0.03 3)
090,639,802 bytes 22.87x 00:00:40 (flashzip 0.03 4)
090,141,575 bytes 22.99x 00:04:21 (m99 fast)
090,140,069 bytes 22.99x 00:00:51 (flashzip 0.03 5)
089,789,957 bytes 23.08x 00:02:58 (rar 3.71 -5)
084,831,675 bytes 24.43x 00:02:39 (dark 0.51 default)
083,193,002 bytes 24.91x 00:04:45 (m99 max)
082,466,152 bytes 25.13x 02:44:16 (lzpm 0.15 9)
077,772,776 bytes 26.65x 00:01:24 (tarsaLZP)
074,339,792 bytes 27.88x 00:01:27 (arc 0.4 -m3)
073,675,683 bytes 28.13x 00:01:35 (arc 0.4 -m4)
071,863,039 bytes 28.84x 00:01:48 (arc 0.4 -m5)
071,533,787 bytes 28.97x 00:01:58 (arc 0.4 -m6)
071,422,166 bytes 29.02x 00:02:19 (arc 0.4 -m7)
070,933,784 bytes 29.22x 00:02:12 (pim 2.10)
069,756,321 bytes 29.71x 00:02:46 (hook 1.3 12
068,874,324 bytes 30.09x 01:00:39 (bbb 1280)
068,045,105 bytes 30.46x 00:15:45 (xwrt 3.2 -l14)
066,691,479 bytes 31.08x 00:02:56 (hook 1.3 256)
064,106,615 bytes 32.33x 00:03:03 (hook 1.3 512)
063,881,826 bytes 32.44x 00:05:30 (ccm 1.3 0)
062,431,793 bytes 33.20x 00:03:11 (hook 1.3 1024)
062,063,165 bytes 33.40x 00:03:13 (hook 1.3 1280)
059,393,173 bytes 34.90x 00:13:32 (ppmonstr default)
054,590,902 bytes 37.97x 00:28:22 (lpaq1)
054,573,912 bytes 37.98x 00:07:26 (ccmx 1.30 7)
049,147,820 bytes 42.17x 00:22:25 (lpaq8e)
048,401,329 bytes 42,82x 00:07:14 (paq9a -9)

[donotdisturb]

Sportman
Thanks for the marathon tests (This is a lot of work). I've some suggestion for future tests:
- you can use enwik9 from http://cs.fit.edu/~mmahoney/compression/text.html.
this is a widely accepted test-file.
- Please perform the test without a virtual disk. This is a real-life benchmark, as users are compressing to disks.
- Please report the tests with your quad-core system (there are enough old single core benchmarks on the web).
- Please report the decompression times (it is also important).
- Please insert a blank line after 5 lines or after 50mb in the report.
- Please use the last lzturbo version 0.92 with the following switches: -11, -12, -21, -22, -41...-48 and -59.

Thanks again!

[nimdamsk]

Sorted list with P.F. filter. * - these progams take advantage of multicore CPU, dunno how many cores they can use though

322,723,099 bytes 06.42x 00:00:10 (lzop 1.01 -1) 
177,843,226 bytes 11.65x 00:00:12 (tornado 0.3 -4) 
173,182,224 bytes 11.97x 00:00:17 (lzturbo 0.91 -52) * 
149,288,413 bytes 13.88x 00:00:18 (lzturbo 0.91 -53) * 
120,288,520 bytes 17.23x 00:00:19 (thor 0.96 e4) 
099,084,865 bytes 20.92x 00:00:30 (flashzip 0.03 1) 
094,531,596 bytes 21.93x 00:00:32 (flashzip 0.03 2) 
092,402,445 bytes 22.43x 00:00:34 (arc 0.4 -m2) 
091,963,302 bytes 22.54x 00:00:35 (flashzip 0.03 3) 
090,639,802 bytes 22.87x 00:00:40 (flashzip 0.03 4) 
090,140,069 bytes 22.99x 00:00:51 (flashzip 0.03 5) 
077,772,776 bytes 26.65x 00:01:24 (tarsaLZP) 
074,339,792 bytes 27.88x 00:01:27 (arc 0.4 -m3) 
073,675,683 bytes 28.13x 00:01:35 (arc 0.4 -m4) * 
071,863,039 bytes 28.84x 00:01:48 (arc 0.4 -m5) * 
071,533,787 bytes 28.97x 00:01:58 (arc 0.4 -m6) * 
070,933,784 bytes 29.22x 00:02:12 (pim 2.10) 
069,756,321 bytes 29.71x 00:02:46 (hook 1.3 128) 
066,691,479 bytes 31.08x 00:02:56 (hook 1.3 256) 
064,106,615 bytes 32.33x 00:03:03 (hook 1.3 512) 
062,431,793 bytes 33.20x 00:03:11 (hook 1.3 1024) 
062,063,165 bytes 33.40x 00:03:13 (hook 1.3 1280) 
048,401,329 bytes 42,82x 00:07:14 (paq9a -9)

[Nania Francesco]

Thanks for the Tests! HOOK is impressive !

[Bulat Ziganshin]

* - these progams take advantage of multicore CPU
073,675,683 bytes 28.13x 00:01:35 (arc 0.4 -m4) *

in this test file was compressed as text so lzma was not used and freearc utilized only one core

shortly speaking, current FA version uses 2 cores only internally in LZMA, everyhing else is executed sequentially


donotdisturb
you forget the rule #1 of your fair testing - dont test any program that outperforms my own

[tester]

Is it possible to obtain the Apache log file you used in tests? I'm looking for the big real-world Apache logs and your file looks very interesting.

[donotdisturb]

Hi,
Bulat Ziganshin To speak about fairness, you should read your posts above:

can you add p.f. with lzturbo results multiplied by 4?

although anyway its better to ask Sportman to repeat his tests with lzturbo -p1 (in single core mode)

Ive just made some suggestions, because as you already know the test-files used by sportmann are not representative. Everyone
is free to make testing with his own files, but i think there are many
test corpus on the web.

Go out and look at the shops. Nearly 100% pcs are dual-cores. We will soon have 8 cores cpus, but still most programmers are optimizing for single core (with small internal cache) because lot of benchmarker are using single core cpus ( LTBC, maximumcompression, ...) and this is not helping today users.

You are also free to make your suggestions( Ex. binary files) !

[Black_Fox1]

test-files used by sportmann are not representative

unlike many other testing corpora this is at least a real-life file somebody might really want to compress
Perhaps data like ReactOS source code should be used next time?

[Bulat Ziganshin]

To speak about fairness,

i was interested to know pure power of algorithms and asked for it. now i see that your progran provides advantages in multicore/io processing but not the core algorithm (except for -*9 modes). you are interested in "fair" advertisement of your program so you ask for conditions that are best for your program

[Nania Francesco]

For me the test-files used by Sportmann are very very interesting!
!!!
Hi!

[Sportman]

- you can use enwik9 from http://cs.fit.edu/~mmahoney/compression/text.html.
this is a widely accepted test-file.
- Please perform the test without a virtual disk. This is a real-life benchmark, as users are compressing to disks.
- Please report the tests with your quad-core system (there are enough old single core benchmarks on the web).
- Please report the decompression times (it is also important).
- Please insert a blank line after 5 lines or after 50mb in the report.
- Please use the last lzturbo version 0.92 with the following switches: -11, -12, -21, -22, -41...-48 and -59.

I had planned not to test enwik9 because Matt Mahoney already publish nice compression and timing info and I prefer test data whats not already used for tunning archivers. But to see enwik9 under the same test hardware and with settings more optimized for speed then compression and that we can compare the speed with this reference file to other hardware platforms I decide to use enwik9 as test four. Because enwik9 is smaller in size then the test files I used so far, the quickest archivers are close to each other in the 4-5 second range, I didnt added tenths of a second because thats the measurement error when I repeat the same test so they are round 3.50 and 4.49 = 4.

I dont use a virtual disk in memory but three SSD 32GB disks in RAID0, they are the only disks in my system right now. The price of SSD disk halve every year so in some years they shall become main stream; they are quicker, no noise, less energy, small size and save to store data, the only bad thing right now is the price tag and storage size.

I agree there must be tested with multi cores when software support it and makes it quicker. I expect within a year 8 core shall be launched and today you buy an Intel 2.4GHz quad core PC without monitor starting from 500 euro so thats already becoming main stream.

Testing decompression is a lot of extra work and time I hope it fit in my busy scedule one day

Its more practicle for me to maintain this list without blank lines.

I added some extra lzturbo settings to this test.

Input:
1,000,000,000 bytes (enwik9)

Output:

size bytes, ratio, hh:mm:ss (compressor version level)

531,348,504 bytes 1.88x 00:00:09 (tornado 0.3 -1)
488,397,982 bytes 2.05x 00:00:06 (thor 0.96 e1)
477,632,222 bytes 2.09x 00:00:08 (lzop 1.01 -1)
465,570,241 bytes 2.15x 00:00:11 (quicklz 1.4.0 b7 1)
430,453,227 bytes 2.32x 00:00:04 (lzturbo 0.92 -11)
425,384,313 bytes 2.35x 00:00:16 (brieflz)
411,416,252 bytes 2.43x 00:00:09 (thor 0.96 e2)
405,243,545 bytes 2.47x 00:00:08 (quicklz 1.4.0 b7 2)
404,250,979 bytes 2.47x 00:00:05 (slug 1.1b)
400,137,917 bytes 2.50x 00:00:04 (lzturbo 0.92 -21)
396,725,688 bytes 2.52x 00:00:31 (quicklz 1.4.0 b7 3)
396,674,866 bytes 2.52x 00:00:11 (tornado 0.3 -2)
378,104,171 bytes 2.64x 00:00:24 (rar 3.71 zip fastest)
374,228,865 bytes 2.67x 00:00:34 (lzc 0.08 1)
373,842,149 bytes 2.67x 00:00:04 (lzturbo 0.92 -12)
367,671,220 bytes 2.72x 00:00:11 (thor 0.96 e3)
367,633,910 bytes 2.72x 00:09:10 (lzw 0.2)
366,349,786 bytes 2.73x 00:01:59 (lzop 1.01 -9)
356,270,881 bytes 2.81x 00:00:13 (arc 0.4 -m1)
356,270,650 bytes 2.81x 00:00:12 (tornado 0.3 -3)
354,254,674 bytes 2.82x 00:00:31 (lzc 0.08 2)
353,355,826 bytes 2.83x 00:00:30 (rar 3.71 zip fast)
353,208,137 bytes 2.83x 00:00:25 (rar 3.71 -1)
343,709,645 bytes 2.91x 00:00:04 (lzturbo 0.92 -22)
336,735,307 bytes 2.97x 00:00:31 (lzc 0.08 3)
336,667,368 bytes 2.97x 00:00:35 (7za 4.57 zip 1)
336,667,283 bytes 2.97x 00:00:36 (7za 4.57 gzip 1)
334,121,578 bytes 2.99x 00:00:15 (tornado 0.3 -4)
333,963,812 bytes 2.99x 00:00:04 (lzturbo 0.92 -41)
325,434,766 bytes 3.07x 00:00:06 (lzturbo 0.92 -51)
324,125,809 bytes 3.09x 00:01:09 (cabarc 1.0 default)
323,582,146 bytes 3.09x 00:00:59 (rar 3.71 zip normal)
322,460,651 bytes 3.10x 00:01:12 (rar 3.71 zip good)
322,454,406 bytes 3.10x 00:01:13 (rar 3.71 zip max)
321,822,399 bytes 3.11x 00:00:32 (lzc 0.08 4)
321,539,056 bytes 3.11x 00:00:05 (lzturbo 0.92 -42)
318,750,591 bytes 3.14x 00:00:07 (lzturbo 0.92 -52)
315,611,172 bytes 3.17x 00:00:35 (thor 0.96 e5)
314,092,324 bytes 3.18x 00:00:24 (thor 0.96 e4)
313,425,420 bytes 3.19x 00:01:21 (mcomp 1.0 -d64f)
312,324,268 bytes 3.20x 00:02:03 (7za 4.57 zip 5)
312,324,183 bytes 3.20x 00:02:03 (7za 4.57 gzip 5)
309,207,655 bytes 3.23x 00:00:39 (lzc 0.08 5)
305,303,656 bytes 3.28x 00:01:06 (7za 4.57 7z 1)
300,731,066 bytes 3.33x 00:00:28 (tornado 0.3 -5)
299,443,551 bytes 3.34x 00:00:18 (flashzip 0.02)
298,366,550 bytes 3.35x 00:00:52 (lzc 0.08 6)
296,756,947 bytes 3.37x 00:00:09 (lzturbo 0.92 -43)
295,664,161 bytes 3.38x 00:00:29 (7za 4.57 bzip2 1)
293,818,146 bytes 3.40x 00:00:11 (lzturbo 0.92 -53)
292,637,877 bytes 3.42x 00:05:26 (lzxq 0.2)
292,316,213 bytes 3.42x 00:00:37 (tornado 0.3 -6)
289,031,071 bytes 3.46x 00:01:07 (lzc 0.08 7)
287,961,230 bytes 3.47x 00:00:16 (lzturbo 0.92 -44)
285,325,724 bytes 3.50x 00:00:19 (lzturbo 0.92 -54)
282,983,224 bytes 3.53x 00:00:52 (tornado 0.3 -7)
280,151,529 bytes 3.57x 00:01:23 (lzc 0.08
279,653,211 bytes 3.58x 00:00:33 (flashzip 0.03 1)
278,724,262 bytes 3.59x 00:00:23 (lzturbo 0.92 -55)
276,089,557 bytes 3.62x 00:01:17 (tornado 0.3 -
275,405,491 bytes 3.63x 00:03:12 (winace 2.69 superfast)
274,561,263 bytes 3.64x 00:03:18 (winace 2.69 fast)
273,906,319 bytes 3.65x 00:00:34 (sr2)
273,498,098 bytes 3.66x 00:00:33 (lzturbo 0.92 -56)
273,252,464 bytes 3.66x 00:01:39 (lzc 0.08 9)
270,726,027 bytes 3.69x 00:04:09 (winace 2.69 normal)
270,590,963 bytes 3.70x 00:04:14 (winace 2.69 good)
270,520,320 bytes 3.70x 00:04:17 (winace 2.69 maximum)
269,450,668 bytes 3.71x 00:00:46 (lzturbo 0.92 -57)
269,421,440 bytes 3.71x 00:00:31 (flashzip 0.03 2)
268,745,450 bytes 3.72x 00:02:11 (tornado 0.3 -9)
267,696,308 bytes 3.74x 00:01:49 (rar 3.71 -2)
266,565,255 bytes 3.75x 00:02:01 (lzc 0.08 10)
264,100,178 bytes 3.79x 00:03:45 (tornado 0.3 -10
263,941,568 bytes 3.79x 00:01:56 (lzturbo 0.92 -5
262,694,462 bytes 3.81x 00:02:50 (lzpm 0.15 1)
260,606,910 bytes 3.84x 00:00:40 (flashzip 0.03 3)
260,499,364 bytes 3.84x 00:02:46 (rar 3.71 -3)
259,412,590 bytes 3.85x 00:07:19 (tornado 0.3 -11)
256,016,972 bytes 3.91x 00:13:27 (tornado 0.3 -12)
254,000,955 bytes 3.94x 00:00:41 (7za 4.57 bzip2 5)
253,480,730 bytes 3.95x 00:00:54 (flashzip 0.03 4)
253,031,980 bytes 3.95x 00:00:44 (sr3)
248,094,851 bytes 4.03x 00:01:21 (flashzip 0.03 5)
247,804,676 bytes 4.04x 00:07:46 (lzpm 0.15 2)
247,467,863 bytes 4.04x 00:01:03 (winturtle 1.6.0b)
235,567,823 bytes 4.25x 00:25:26 (lzpm 0.15 9)
234,377,084 bytes 4.27x 00:05:00 (mcomp 1.0 -ff)
233,366,203 bytes 4.29x 00:00:54 (arc 0.4 -m2)
233,272,334 bytes 4.29x 00:05:29 (mcomp 1.0 -f3f)
232,701,587 bytes 4.30x 00:02:38 (lzturbo 0.92 -59)
229,726,573 bytes 4.35x 00:01:51 (hook 1.3 55)
229,695,548 bytes 4.35x 00:00:46 (rings v1.2)
227,933,639 bytes 4.39x 00:06:27 (7za 4.57 7z 5)
226,167,934 bytes 4.42x 00:02:03 (sbc 0.97 -m1)
224,998,267 bytes 4.44x 00:02:29 (sbc 0.97 -m2)
224,187,546 bytes 4.46x 00:04:31 (sbc 0.97 -m3)
218,507,990 bytes 4.58x 00:01:38 (dark 0.51 default)
216,225,820 bytes 4.62x 00:02:58 (rar 3.71 -5)
216,110,243 bytes 4.63x 00:02:29 (rar 3.71 -4)
215,301,412 bytes 4.64x 00:01:28 (tarsaLZP)
213,134,257 bytes 4.69x 00:02:18 (m99 fast)
210,124,871 bytes 4.76x 00:04:21 (pim 2.10)
206,716,896 bytes 4.84x 00:03:53 (ccm 1.3 0)
206,665,863 bytes 4.84x 00:03:16 (m99 max)
204,560,781 bytes 4.89x 00:02:48 (arc 0.4 -m3)
198,840,697 bytes 5.03x 00:03:09 (arc 0.4 -m4)
195,750,817 bytes 5.11x 00:03:21 (hook 1.3 12
195,145,815 bytes 5.12x 00:03:24 (arc 0.4 -m5)
189,696,223 bytes 5.27x 00:03:35 (arc 0.4 -m6)
188,339,606 bytes 5.31x 00:03:35 (hook 1.3 256)
186,395,591 bytes 5.36x 00:09:55 (cmm4 0.0)
185,893,165 bytes 5.38x 00:04:15 (ccmx 1.30 7)
184,179,683 bytes 5.43x 00:04:37 (durilca light 0.5 default)
183,487,758 bytes 5.45x 00:03:52 (hook 1.3 512)
179,969,118 bytes 5.56x 00:04:18 (hook 1.3 1024)
179,128,312 bytes 5.58x 00:04:25 (hook 1.3 1280)
168,467,267 bytes 5.94x 00:16:44 (lpaq1)
168,327,404 bytes 5.94x 00:17:36 (ppmonstr default)
165,193,368 bytes 6.05x 00:21:50 (paq9a -9)
164,032,650 bytes 6.10x 00:26:10 (bbb 1280)
161,987,713 bytes 6.17x 00:17:14 (lpaq8e)
151,565,187 bytes 6.60x 00:12:03 (xwrt 3.2 -l14)

[Sportman]

Is it possible to obtain the Apache log file you used in tests? Im looking for the big real-world Apache logs and your file looks very interesting.

No, Im sorry. Big websites are often not fan of sharing their traffic details, but strange enough they almost all use google analytics to analyze and store their data for free.

[nimdamsk]

P.F. filtered overall:

333,963,812 bytes 2.99x 00:00:04 (lzturbo 0.92 -41) 
321,539,056 bytes 3.11x 00:00:05 (lzturbo 0.92 -42) 
318,750,591 bytes 3.14x 00:00:07 (lzturbo 0.92 -52) 
296,756,947 bytes 3.37x 00:00:09 (lzturbo 0.92 -43) 
293,818,146 bytes 3.40x 00:00:11 (lzturbo 0.92 -53) 
287,961,230 bytes 3.47x 00:00:16 (lzturbo 0.92 -44) 
285,325,724 bytes 3.50x 00:00:19 (lzturbo 0.92 -54) 
278,724,262 bytes 3.59x 00:00:23 (lzturbo 0.92 -55) 
269,421,440 bytes 3.71x 00:00:31 (flashzip 0.03 2) 
260,606,910 bytes 3.84x 00:00:40 (flashzip 0.03 3) 
254,000,955 bytes 3.94x 00:00:41 (7za 4.57 bzip2 5) 
253,031,980 bytes 3.95x 00:00:44 (sr3) 
229,695,548 bytes 4.35x 00:00:46 (rings v1.2) 
215,301,412 bytes 4.64x 00:01:28 (tarsaLZP) 
213,134,257 bytes 4.69x 00:02:18 (m99 fast) 
204,560,781 bytes 4.89x 00:02:48 (arc 0.4 -m3) 
198,840,697 bytes 5.03x 00:03:09 (arc 0.4 -m4) 
195,750,817 bytes 5.11x 00:03:21 (hook 1.3 128) 
195,145,815 bytes 5.12x 00:03:24 (arc 0.4 -m5) 
188,339,606 bytes 5.31x 00:03:35 (hook 1.3 256) 
183,487,758 bytes 5.45x 00:03:52 (hook 1.3 512) 
179,969,118 bytes 5.56x 00:04:18 (hook 1.3 1024) 
179,128,312 bytes 5.58x 00:04:25 (hook 1.3 1280) 
151,565,187 bytes 6.60x 00:12:03 (xwrt 3.2 -l14)

The same with promrams using one cpu:

404,250,979 bytes 2.47x 00:00:05 (slug 1.1b) 
367,671,220 bytes 2.72x 00:00:11 (thor 0.96 e3) 
356,270,650 bytes 2.81x 00:00:12 (tornado 0.3 -3) 
334,121,578 bytes 2.99x 00:00:15 (tornado 0.3 -4) 
299,443,551 bytes 3.34x 00:00:18 (flashzip 0.02) 
295,664,161 bytes 3.38x 00:00:29 (7za 4.57 bzip2 1) 
269,421,440 bytes 3.71x 00:00:31 (flashzip 0.03 2) 
260,606,910 bytes 3.84x 00:00:40 (flashzip 0.03 3) 
254,000,955 bytes 3.94x 00:00:41 (7za 4.57 bzip2 5) 
253,031,980 bytes 3.95x 00:00:44 (sr3) 
229,695,548 bytes 4.35x 00:00:46 (rings v1.2) 
215,301,412 bytes 4.64x 00:01:28 (tarsaLZP) 
213,134,257 bytes 4.69x 00:02:18 (m99 fast) 
204,560,781 bytes 4.89x 00:02:48 (arc 0.4 -m3) 
198,840,697 bytes 5.03x 00:03:09 (arc 0.4 -m4) 
195,750,817 bytes 5.11x 00:03:21 (hook 1.3 128) 
195,145,815 bytes 5.12x 00:03:24 (arc 0.4 -m5) 
188,339,606 bytes 5.31x 00:03:35 (hook 1.3 256) 
183,487,758 bytes 5.45x 00:03:52 (hook 1.3 512) 
179,969,118 bytes 5.56x 00:04:18 (hook 1.3 1024) 
179,128,312 bytes 5.58x 00:04:25 (hook 1.3 1280) 
151,565,187 bytes 6.60x 00:12:03 (xwrt 3.2 -l14)

User can't make choice without seeing decompression times. Usually people compress once and decompress many times. Besides, AFAIK, multicore using can't speed up decompression time.

[Bulat Ziganshin]

The price of SSD disk halve every year so in some years they shall become main stream; they are quicker, no noise, less energy, small size and save to store data

shortly said: HDD IS YESTERDAY

Besides, AFAIK, multicore using cant speed up decompression time.

really? when input data are just split into independetly compresssed chunks, both the compression and decompression becomes faster. this degrades compression, though - its why this approach isnt used in lzma

The same with promrams using one cpu:
254,000,955 bytes 3.94x 00:00:41 (7za 4.57 bzip2 5)

bzip2 by itself is rather slow. its multithreaded in 7z, compressing each 1mb separately

Usually people compress once and decompress many times.

yes, especially when doing daily backups