TurboBench - Back to the future (incl. LzTurbo)

[dnd]

Compressor Benchmark : Back to the future version incl. LzTurbo

benchmark program source code : TuboBench Compressor Benchmark

Download TurboBench binaries for windows and linux.

[dnd]

Some advices for general benchmarking with TurboBench
- Download always the latest TurboBench version
- Use more than one file distribution to benchmark (ex. text and binary),
unless you want to benchmark specific data.
- Do not use exclusively small files, if you are benchmarking the fast compressors (10MB - 100MB is ok)
- Do not use exclusively incompressible or difficult to compress files. Use for ex. files compressible to 20-50% with zlib,6
- Disable the bios "intel turbo boost" for accurate timings, disable antivirus programs, disable indexing,...
- Better use linux for benchmarking if possible. The windows timings can be more volatile (like on my windows 10 PC)
- Better use a desktop PC, if possible

[inikep]

On Windows 10 x64 it shows a window with "Didn't find entry _ZNSt13runtime_errorC1EPKc in turbobench.exe library".

[dnd]

Uploaded additional version with static linking. Please download the new windows package

[Jarek]

Hi Hamid,
As some here generally question the objectivity of your tests, maybe you would consider adding lzturbo to the open-source Squash benchmark to remove the doubts once for all:
http://encode.su/threads/2160-Squash...ll=1#post44955

Skymmer, if you believe that there is still tornado code in lzturbo, why don't you just make its branch with a comparable speed?
I don't think evaluating a person by a single event a decade ago is a mature behavior - let's be adults here, live in the present instead of the past.
And the above image is very offensive and personal - a serious discussion forum is not a place for something like this, would lead to a ban in many places.

[Sportman]

Compressor Benchmark : Back to the future version incl. LzTurbo

Input:
5,057,265 bytes - HTML 12 Chinese website frontpages

Output:
847970 16.8 30.40 434.21 brotli 5
898117 17.8 69.50 1202.88 lzturbo 32
949135 18.8 188.61 1139.08 lzturbo 31
955323 18.9 28.01 388.82 zlib 9
965170 19.1 47.12 385.38 zlib 6
970812 19.2 118.76 411.20 brotli 1
1022655 20.2 265.19 1029.05 lzturbo 30.1
1028920 20.3 280.86 1038.58 lzturbo 30
1063970 21.0 288.95 678.09 zstd
1087284 21.5 85.28 1799.41 lzturbo 22
1099424 21.7 48.01 2457.57 lz4 9
1137067 22.5 105.03 343.01 zlib 1
1139463 22.5 261.29 1743.64 lzturbo 21
1207721 23.9 167.83 2275.74 lzturbo 12
1232009 24.4 385.62 1547.92 lzturbo 20.1
1258513 24.9 239.50 2156.35 lzturbo 11
1275791 25.2 330.71 509.98 gipfeli
1312620 26.0 555.64 1728.40 lzturbo 20
1313735 26.0 386.83 1065.31 shrinker
1360591 26.9 509.65 2004.98 lzturbo 10.1
1395164 27.6 574.68 2252.95 lz4 1
1398000 27.6 618.14 2069.92 lzturbo 10
1454606 28.8 470.30 1299.39 snappy
1454606 28.8 548.64 1292.12 snappy-c
2025737 40.1 424.40 598.79 density 2

CPU: Intel Xeon E5-1650 3.20GHz
Memory: 128GB DDR 1600MHZ
Storage: RAID5 8 x 3TB HD 7200 rpm (897.7 MB/s read, 930.8 MB/s write)

[jibz]

When you have the time, you could update to zstd 0.1.3 and brieflz 1.1.0.

[dnd]

Update:
- brieflz 1.1.0 included
- zstd downloaded at Oct 21 2015

[inikep]

if you believe that there is still tornado code in lzturbo, why don't you just make its branch with a comparable speed?
I don't think evaluating a person by a single event a decade ago is a mature behavior - let's be adults here, live in the present instead of the past.

I have two remarks:
1. I agree with Jarek. Currently there is no much of tornado in lzturbo. Just look at following results, especially at decompression speed.
2. Hamid: I don't like claims e.g. "Method 1 - compress better and faster, decompress 2x! faster than lz4.". Maybe on your computer with your files it's faster. But on my laptop with my file even using your turbobench LZ4 is faster in compression and decompression speed.

COMPRESSED_SIZE   RATIO   CSPEED MB/s   DSPEED MB/s      PROGRAM
    64872319      61.9     492.93         2406.33        lz4 1 
    54773521      52.2      25.40         2040.08        lz4 9 

    64525551      61.5     396.32         1550.18        lzturbo 10
    62636909      59.7     366.29         1616.98        lzturbo 10.1
    59362708      56.6     154.24         1676.34        lzturbo 11
    57006462      54.4      95.55         1881.74        lzturbo 12
    54606867      52.1       2.07         1562.68        lzturbo 19
    60996280      58.2     414.95         1487.27        lzturbo 20
    57297733      54.6     262.06         1244.70        lzturbo 20.1
    52745184      50.3     140.84         1355.29        lzturbo 21
    50091022      47.8      32.13         1223.40        lzturbo 22
    45425792      43.3       1.92          868.88        lzturbo 29
    49056137      46.8     180.64          680.60        lzturbo 30
    47802940      45.6     162.49          678.38        lzturbo 30.1
    45624425      43.5     102.82          723.80        lzturbo 31
    42829469      40.8      24.80          662.63        lzturbo 32
    38856325      37.1       1.70          605.33        lzturbo 39
    35761497      34.1       1.26           45.80        lzturbo 49

    71905811      68.6       234.3        315.1   tornado 0.6 -1
    60986804      58.2       198.1        289.3   tornado 0.6 -2
    47941636      45.7       103.7        138.9   tornado 0.6 -3
    45995919      43.9        73.7        146.1   tornado 0.6 -4
    42800293      40.8        23.8         94.6   tornado 0.6 -5
    42136737      40.2        18.3         95.5   tornado 0.6 -6
    41000640      39.1         8.8         96.9   tornado 0.6 -7
    40796313      38.9         5.1         97.1   tornado 0.6 -8
    40678078      38.8         3.9         97.1   tornado 0.6 -9
    40686544      38.8         2.4         97.8   tornado 0.6 -10
    40338967      38.5         9.0         94.4   tornado 0.6 -11
    39688697      37.9         6.5         94.8   tornado 0.6 -12
    39435464      37.6         5.0         96.1   tornado 0.6 -13
    39172223      37.4         3.5         97.0   tornado 0.6 -14
    39016242      37.2         2.6         97.3   tornado 0.6 -15
    38724042      36.9         2.3         97.8   tornado 0.6 -16

The above results are obtained using 1 core of Intel Core i5-4300U, Windows 10 64-bit and 3 iterations. The input file (100 MB) is a concatanation of carefully selected files from installed version of Windows 8.1 64-bit. The file can be downloaded from: https://docs.google.com/uc?id=0BwX7d...xport=download

[dnd]

inikep, thank you for testing and your clarifications.

I've just uploaded a new version with release code for lzturbo,1x (previous was a dev. version)
Please retest again the lzturbo,10,11,12 and 19.

Of course, the speed is dependent on the underlying hardware, compiler version and testfiles,
but lzturbo 19 compress always better decompress always a lot faster than lz4 on the hardware I've used.
Additionally, Cyan is constantly changing lz4.

May be "decompress up to 2x" or a % indication is more correct.

[Sportman]

Update:
- brieflz 1.1.0 included
- zstd downloaded at Oct 21 2015

Can't find download link TurboBench anymore, zstd v0.2 update 4 hours ago.

[dnd]

@Sportman
Please, check again

[inikep]

May be "decompress up to 2x" or a % indication is more correct.

I would write "decompresses even faster", but if you take I/O into account they are equal (2723 MB vs 2407 MB). The more important is if your decompression is safe and doesn't go outside an output buffer.

Please retest again the lzturbo,10,11,12 and 19.

LZ4 -1 still has better compression speed, but this is not important for me. The more important is that LZ4 -9 has 10 times faster compression comparing to lzturbo 19 (and slightly slower decompression). That's why the statement "compress better and faster, decompress 2x faster than Lz4" is ridiculous for me. It destroys a little bit a good impression of using lzturbo.

    64872319      61.9     499.38         2407.79        lz4 1  
    54773521      52.2      25.57         1987.88        lz4 9  

    64525551      61.5     447.47         2603.02        lzturbo 10
    62636909      59.7     362.98         2409.43        lzturbo 10.1
    59362708      56.6     153.53         2455.43        lzturbo 11
    57006462      54.4      96.62         2655.50        lzturbo 12
    54606867      52.1       2.08         2723.54        lzturbo 19

[Sportman]

Input:
635,346,739 byte - IIS log file

Output:
31129105 4.9 228.62 2474.45 lzturbo 32
32311925 5.1 75.54 1189.49 brotli 5
34422186 5.4 589.72 2585.91 lzturbo 31
36928778 5.8 237.85 1107.79 brotli 1
38394382 6.0 832.89 2390.45 lzturbo 30.1
38830596 6.1 856.73 2407.12 lzturbo 30
41055482 6.5 313.29 4491.56 lzturbo 22
42650280 6.7 640.00 1571.74 zstd
45509470 7.2 811.30 4441.48 lzturbo 21
49457618 7.8 63.72 629.54 zlib 9
50340642 7.9 1167.28 3866.06 lzturbo 20.1
50914462 8.0 78.70 622.16 zlib 6
54941433 8.6 75.06 4150.01 lz4 9
58369983 9.2 1249.21 3927.25 lzturbo 20
62367579 9.8 366.56 5534.65 lzturbo 12
62711505 9.9 174.26 570.93 zlib 1
64097173 10.1 554.21 5341.56 lzturbo 11
68829751 10.8 991.67 5020.73 lzturbo 10.1
70020627 11.0 1001.57 2250.08 shrinker
70106737 11.0 1158.17 5104.38 lzturbo 10
71659476 11.3 1133.76 3813.82 lz4 1
79491636 12.5 721.10 1068.05 gipfeli
97432923 15.3 1012.74 2308.41 snappy
97432923 15.3 1092.22 2169.67 snappy-c
102428657 16.1 912.99 845.23 density 2

[dnd]

- Include (>50) allmost all popular, latest or fastest compressors in one compiled package
- Set number of iterations, number of runs, benchamrks, set max. time per run
- Benchmarking Entropy Coders, Lz77 and BWT compressors
- Text, html, csv, markdown and other output formats
- show pareto frontier
- Transfer speed sheet for different connections or devices: GPRS,3G,4G,DSL,Network,HDD,SSD,RAM


benchmark program source code: TuboBench Compressor Benchmark

See new Benchmark w/ vBulletin output format pasted to encode.su

[nemequ]

I see you're distributing a compiled binary which includes software not compatible with the GPL (lzturbo). Since you're including several GPL'd libraries you're violating their license. I believe (IANAL) that anyone who wrote GPL'd code in any library you are distributing has standing for a copyright infringement lawsuit.

[SvenBent]

I see you're distributing a compiled binary which includes software not compatible with the GPL (lzturbo). Since you're including several GPL'd libraries you're violating their license. I believe (IANAL) that anyone who wrote GPL'd code in any library you are distributing has standing for a copyright infringement lawsuit.

Nothing new really try googling lzturbo and tornado. GPL issues seems to be a common problem for lzturbo.

[dnd]

I'm sorry, but there are two different executables in each package, one w/o gpl libraries and the other including only gpl programs. The benchmark program source code is also available.
To avoid such a confusion, I'm offering now the gpl version in a separate 7-zip file.

[nemequ]

I'm sorry, but there are two different executables in each package, one w/o gpl libraries and the other including only gpl programs. The benchmark program source code is also available.

Ah, nice, that should work.

To avoid such a confusion, I'm offering now the gpl version in a separate 7-zip file.

You don't need to do that; separate executables should be enough.

GPL compliance is fun, isn't it?

[dnd]

Turbobench update incl. Transfer Speed Plot:

- new - automatic update & merge of result files
- new - Text, html, csv, markdown and other output formats without retesting
- new - html output with sortable tables
- new - Transfer speed sheet for different connections or devices: GPRS,2G,3G,4G,DSL,Network,HDD,SSD,RAM
- Speedup html plot
- Linux and Windows binaries
- All in one executable, no hassless installing of additional package

[GOZARCK]

i have an error in win binary. "libgcc_s_seh-1.dll" is missing

[dnd]

Please, use "turbobenchs" (with s)

[dnd]

Turbobench update:

- New plugin: libzpaq
- New plugin: shoco
- New version: zstd
- New update brotli and all submodules
- New option "-s" to benchmark small files accurately
- New option "-g" disable update of result file
- stdin / stdout file I/O
- Source code: 32 bits ready for linux and windows

[dnd]

Turbobench github update:

- Gipfeli updated
- New: Clang support
- New: Ratio/Speed plot

[dnd]

Turbobench github update all submodules:

- zstd v0.4.2
- lz5 v1.3.2

Compile the latest versions for linux, windows (mingw):

          git clone --recursive git://github.com/powturbo/TurboBench.git

          make

[dnd]

Turbobench github update all submodules.

- New: build in peak memory usage and reporting on html output.
Only for linux and tracking only malloc/free memory calls.

- New: Sample codec "mycodec" to show how to include and build your own functions.

- New Entropy Coders: fpaq0p_sh, fqz0, ppmdec, nania rANS, fpaqc

[dnd]

TurboBench update

Turbobench github update all submodules.

- zstd
- lz5
- CSC
- brotli
- libzling

- New: miniz
- New: Visual Studio c++ 2015 support

- New: You can now disable the brotli dictionary in TurboBench with the parameter 'D' :

./turbobench -ebrotli,11,11D canterbury/cp.html

      C Size  ratio%     C MB/s     D MB/s   Name            File              (bold = pareto) MB=1.000.0000
        6897    28.0      0.73     291.61   brotli 11       cp.html
        7444    30.3      0.74     281.80   brotli 11D      cp.html

[Gonzalo]

trying to compile on linux Ubuntu x64, up-to-date:

make: *** No hay ninguna regla para construir el objetivo '/usr/lib/x86_64-linux-gnu/libz.a', necesario para 'turbobench'.  Alto.

No rules to build libz...

I tried to use your binaries first but they crashed.

Any clue?

[dnd]

this is what I hate, downloading a package that won't compile, especially when cmake, other dependencies are required or when you want to use a newer sub-module version.
This is why I'm trying to make "TurboBench" compiling very simple and only linking to sub-modules.

I have now updated the makefile. Please, try again.

[Gonzalo]

I have now updated the makefile. Please, try again.

Thanks, but it didn't work either

make: *** No hay ninguna regla para construir el objetivo '-lz', necesario para 'turbobench'.  Alto.