SLZ - included in TurboBench (only github)
html8 : 100MB random html pages from a 2GB Alexa Top sites corpus.
number of pages = 1178
average length = 84886 bytes.
The pages (length + content) are concatenated into a single html8 file, but compressed/decompressed separately.
compress: page1,page2,...pageN
decompress : Page1,page2,...pageN
This avoids the cache effects like in other benchmarks, where small files are processed repeatedly in the L1/L2 cache,
showing unrealistic results.
size: 100,000,000 bytes.
Single thread in memory benchmark
cpu: Sandy bridge i7-2600k at 4.2 Ghz, all with gcc 5.4, ubuntu 16.04
Code:
C Size ratio% C MB/s D MB/s Name C Mem Peak D Mem Peak (bold = pareto) MB=1.000.0000
16461059 16.5 0.79 449.97 brotli 11 10,616,320 241,016
19615056 19.6 0.13 398.95 zopfli 33,644,072 14,320
19815500 19.8 6.14 685.06 libdeflate 12 17,938,928 71,312
20163816 20.2 92.93 559.73 brotli 4 35,314,344 193,272
20363869 20.4 40.28 435.27 zlib 9 274,064 14,320
20485533 20.5 63.34 433.08 zlib 6 274,064 14,320
20566179 20.6 115.35 678.32 libdeflate 6 2,202,032 71,312
20624793 20.6 86.26 431.00 zlib_ng 6 274,272 14,320
22630144 22.6 165.19 634.85 libdeflate 1 2,202,032 15,184
23143931 23.1 329.10 524.32 brotli 1 1,193,296 16,810,696
23723266 23.7 139.94 398.91 zlib 1 274,064 14,320
24070531 24.1 411.35 488.54 brotli 0 145,920 16,810,696
28214601 28.2 371.16 433.05 slz 6 0 14,320 (64k stack + 132k static memory at compression)
28214601 28.2 371.23 433.02 slz 9 0 14,320
28425348 28.4 671.00 2145.63 lzturbo 20 *
29466128 29.5 445.13 483.71 slz 1d 0 14,320 (deflate)
29476316 29.5 371.19 424.83 slz 1 0 14,320
29995215 30.0 309.16 414.03 zlib_ng 1 175,968 14,320 (optimized for intel SSE4.2)
Hardware: ODROID C2 - ARM 64 bits - 1.536Ghz CPU, OS: Ubuntu 16.04, gcc 5.3
Code:
C Size ratio% C MB/s D MB/s Name
16461059 16.5 0.14 87.22 brotli 11
20163816 20.2 14.06 102.77 brotli 4
20363869 20.4 8.14 128.23 zlib 9
20485533 20.5 13.04 127.66 zlib 6
22630144 22.6 21.31 143.02 libdeflate 1
23143931 23.1 44.01 93.11 brotli 1
23723266 23.7 32.90 117.04 zlib 1
24070531 24.1 76.59 84.36 brotli 0
28214601 28.2 54.30 127.64 slz 6
28214601 28.2 54.33 127.64 slz 9
28425348 28.4 143,06 542,98 lzturbo 20
29476316 29.5 51.58 126.06 slz 1
Skylake i7-6700 - 3.7GHz
Code:
C Size ratio% C MB/s D MB/s Name
16461059 16.5 0.76 407.68 brotli 11
20163816 20.2 89.92 512.58 brotli 4
20363869 20.4 34.31 370.89 zlib 9
20485533 20.5 55.47 369.05 zlib 6
23143931 23.1 269.01 487.64 brotli 1
23723266 23.7 125.46 340.55 zlib 1
28214601 28.2 332.37 371.57 slz 9
28214601 28.2 332.43 371.41 slz 6
28425348 28.4 611.31 1691.52 lzturbo 20
29476316 29.5 336.27 364.31 slz 1
* LzTurbo,20 incl. as indication, not gz,br compatible
Note: Single thread peak memory - Only malloc,calloc,... are tracked. No Stack, No mmap
Input/Output buffer not counted.
Remark:
Compression speed slz vs. zlib:
- 2,65x faster on intel i7 ( 20% faster than zlib-ng SSE4.2)
- 1,65x faster on ARM
- "brotli 1" can also be an alternative (but memory usage is high)
- Edit: "brotli 0" faster than slz, but decompression is slow on arm