Page 1 of 4 123 ... LastLast
Results 1 to 30 of 101

Thread: enwik10 benchmark results

  1. #1
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    881
    Thanks
    80
    Thanked 315 Times in 219 Posts

    enwik10 benchmark results

    Input:
    10,000,000,000 bytes - enwik10 (1GB EN, 1GB DE, 1GB FR, 1GB RU, 1GB ES, 1GB JA, 1GB IT, 1GB PL, 1GB ZH, 1GB PT)

    Code:
    Output:
    5,039,464,624 bytes,     20.596 sec. -      7.684 sec., lizard -10 (v1.0.0)
    5,034,758,325 bytes,     18.449 sec. -      7.311 sec., lz4 -1 (v1.9.2)
    4,931,432,031 bytes,     28.371 sec. -     20.632 sec., qpress64 -L1T1 (v1.1)
    4,883,575,606 bytes,     34.957 sec. -     50.879 sec., thor e1 (v0.96 alpha)
    4,666,386,317 bytes,     26.686 sec. -      4.827 sec., lzturbo -10 -p0 (v1.2)
    4,665,488,020 bytes,     25.931 sec. -      8.760 sec., lizard -20 (v1.0.0)
    4,395,542,076 bytes,     47.994 sec. -     20.250 sec., qpress64 -L2T1 (v1.1)
    4,375,561,039 bytes,     26.721 sec. -     10.684 sec., lizard -30 (v1.0.0)
    4,371,496,854 bytes,     46.907 sec. -      7.282 sec., lz4x -1 (v1.60)
    4,310,953,883 bytes,     27.091 sec. -      8.435 sec., lzturbo -20 -p0 (v1.2)
    4,260,604,506 bytes,     46.419 sec. -     55.936 sec., nz -cf -t1 (v0.09 alpha)
    4,250,963,422 bytes,    105.659 sec. -     55.171 sec., brieflz (v1.05)
    4,220,364,579 bytes,    128.482 sec. -     16.681 sec., qpress64 -L3T1 (v1.1)
    4,217,868,600 bytes,     46.518 sec. -      8.474 sec., lzturbo -11 -p0 (v1.2)
    3,909,521,247 bytes,     32.603 sec. -     11.287 sec., lizard -40 (v1.0.0)
    3,913,220,262 bytes,    102.474 sec. -      8.227 sec., lzturbo -12 -p0 (v1.2)
    3,895,465,180 bytes,    107.015 sec. -      7.319 sec., lz4x -5 (v1.60)
    3,891,832,957 bytes,    129.520 sec. -     13.702 sec., blz4 -5 (v0.2.0)
    3,860,617,999 bytes,    121.830 sec. -      7.159 sec., lizard -15 (v1.0.0)
    3,848,919,298 bytes,     89.150 sec. -      7.451 sec., lz4 -3 (v1.9.2)
    3,833,998,225 bytes,     58.913 sec. -     28.757 sec., blzpack -1 (v1.3.0)
    3,823,273,187 bytes,    136.146 sec. -     59.070 sec., gzip -1 (v1.3.12)
    3,770,151,519 bytes,     34.216 sec. -     26.236 sec., brotli -q 0 (v1.0.7)
    3,770,151,519 bytes,     34.141 sec. -     26.256 sec., brotli -q 0 --large_window=30 (v1.0.7)
    3,765,930,127 bytes,     43.910 sec. -     11.690 sec., lzturbo -21 -p0 (v1.2)
    3,729,651,972 bytes,     78.545 sec. -     67.090 sec., thor e3 (v0.96 alpha)
    3,727,069,603 bytes,    132.560 sec. -      7.187 sec., lz4 -5 (v1.9.2)
    3,713,877,399 bytes,    113.441 sec. -      8.040 sec., lizard -35 (v1.0.0)
    3,694,752,709 bytes,    180.757 sec. -      7.362 sec., lz4 -7 (v1.9.2)
    3,689,371,924 bytes,    214.152 sec. -      7.134 sec., lz4 -9 (v1.9.2)
    3,673,955,779 bytes,    143.553 sec. -     57.417 sec., gzip -2 (v1.3.12)
    3,669,477,746 bytes,  1,143.735 sec. -      7.320 sec., lizard -19 (v1.0.0)
    3,660,882,443 bytes,    767.399 sec. -      7.633 sec., lz4x -9 (v1.60)
    3,659,315,526 bytes,  5,722.456 sec. -      8.793 sec., lzturbo -19 -p0 (v1.2)
    3,653,797,048 bytes,    410.854 sec. -      8.592 sec., lizard -25 (v1.0.0)
    3,638,532,812 bytes,     23.309 sec. -     11.745 sec., zstd -1 (v1.4.4)
    3,621,357,269 bytes,     62.690 sec. -     84.463 sec., nz -cF -t1 (v0.09 alpha)
    3,660,882,443 bytes,    882.019 sec. -     13.095 sec., blz4 -9 (0.2.0)
    3,583,341,678 bytes,    133.771 sec. -     47.076 sec., rar -m1 -ma5 -mt1 (v5.80)
    3,551,047,816 bytes,    168.021 sec. -     55.993 sec., gzip -3 (v1.3.12)
    3,545,817,857 bytes,     42.569 sec. -     24.609 sec., brotli -q 1 (v1.0.7)
    3,545,817,857 bytes,     41.997 sec. -     24.646 sec., brotli -q 1 --large_window=30 (v1.0.7)
    3,517,193,402 bytes,     56.106 sec. -     47.633 sec., arc -m1x -mt1 (v0.67)
    3,517,193,402 bytes,     56.045 sec. -     47.756 sec., arc -m1 -mt1 (v0.67)
    3,496,587,503 bytes,     43.607 sec. -     18.271 sec., lzturbo -30 -p0 (v1.2)
    3,408,619,518 bytes,  2,301.506 sec. -      8.174 sec., lizard -29 (v1.0.0)
    3,403,958,298 bytes,    180.169 sec. -     55.199 sec., gzip -4 (v1.3.12)
    3,373,871,144 bytes,    175.480 sec. -     41.977 sec., 7z -tzip -mx1 -mmt1 (v19.02)
    3,373,871,144 bytes,    175.698 sec. -     42.025 sec., 7z -tzip -mx3 -mmt1 (v19.02)
    3,373,717,657 bytes,    129.235 sec. -     11.727 sec., lzturbo -22 -p0 (v1.2)
    3,343,138,171 bytes,  1,183.661 sec. -      8.654 sec., lizard -39 (v1.0.0)
    3,325,794,738 bytes,     33.481 sec. -     12.952 sec., zstd -2 (v1.4.4)
    3,313,143,745 bytes,    319.411 sec. -     59.965 sec., nz -cdp -t1 (v0.09 alpha)
    3,294,626,623 bytes,    230.555 sec. -     54.414 sec., gzip -5 (v1.3.12)
    3,292,612,761 bytes,    149.939 sec. -     32.771 sec., bcrush -1 (v0.2.0)
    3,292,420,658 bytes,     37.645 sec. -      9.609 sec., zhuff -c0 -t1 (v0.99beta)
    3,281,731,612 bytes,    374.211 sec. -     59.403 sec., nz -cdP -t1 (v0.09 alpha)
    3,277,334,825 bytes,    245.406 sec. -     59.173 sec., nz -cd -t1 (v0.09 alpha)
    3,268,318,786 bytes,     84.742 sec. -     22.775 sec., brotli -q 2 (v1.0.7)
    3,268,318,786 bytes,     84.320 sec. -     22.739 sec., brotli -q 2 --large_window=30 (v1.0.7)
    3,263,468,943 bytes,    143.662 sec. -     34.122 sec., crush cf
    3,249,895,474 bytes,    307.467 sec. -     53.869 sec., gzip -6 (v1.3.12)
    3,242,491,764 bytes,    341.968 sec. -     53.809 sec., gzip -7 (v1.3.12)
    3,240,805,000 bytes,    422.917 sec. -     10.572 sec., lizard -45 (v1.0.0)
    3,237,924,757 bytes,    390.420 sec. -     53.756 sec., gzip -8 (v1.3.12)
    3,237,812,198 bytes,    392.835 sec. -     53.771 sec., gzip -9 (v1.3.12)
    3,220,150,965 bytes,     94.157 sec. -     22.117 sec., brotli -3 (v1.0.7)
    3,158,641,894 bytes,    150.717 sec. -      9.848 sec., zhuff -c1 -t1 (v0.99beta)
    3,149,492,468 bytes,     54.666 sec. -     20.956 sec., lzturbo -31 -p0 (v1.2)
    3,145,771,456 bytes,     50.885 sec. -     34.649 sec., etincelle (RC2)
    3,142,120,444 bytes,    244.059 sec. -     57.728 sec., thor e5 (v0.96 alpha)
    3,137,189,321 bytes,     43.010 sec. -     13.343 sec., zstd -3 (v1.4.4)
    3,127,103,081 bytes,    811.434 sec. -     40.315 sec., 7z -tzip -mx5 -mmt1 (v19.02)
    3,113,460,664 bytes,  2,257.262 sec. -     40.351 sec., 7z -tzip -mx7 -mmt1 (v19.02)
    3,112,379,088 bytes,    164.636 sec. -     60.788 sec., thor e4 (v0.96 alpha)
    3,111,145,623 bytes,  5,571.823 sec. -     40.397 sec., 7z -tzip -mx9 -mmt1 (v19.02)
    3,095,434,783 bytes,    266.915 sec. -     74.614 sec., zpaq -m1 -t1 (v7.15)
    3,095,575,993 bytes,    133.367 sec. -     21.501 sec., brotli -q 4 --large_window=30 (v1.0.7)
    3,095,248,795 bytes,    137.881 sec. -     20.738 sec., brotli -q 4 (v1.0.7)
    3,078,914,611 bytes,    240.124 sec. -      9.381 sec., zhuff -c2 -t1 (v0.99beta)
    3,072,049,859 bytes,     46.019 sec. -     14.061 sec., zstd -4 (v1.4.4)
    3,028,529,073 bytes,    326.396 sec.-      75.082 sec., 7z -t7z -mx1 -mmt1 (v19.02)
    3,010,547,116 bytes,  2,420.188 sec. -     10.074 sec., lizard -49 (v1.0.0)
    3,004,577,735 bytes,     52.249 sec. -     53.043 sec., slug (v1.27b)
    2,993,531,459 bytes,     72.476 sec. -     14.123 sec., zstd -5 (v1.4.4)
    2,948,591,837 bytes,    731.412 sec. -    103.179 sec., 7z -tbzip2 -mx1 -mmt1 (v19.02)
    2,947,761,814 bytes,    196.714 sec. -     53.101 sec., arc -m2x -mt1 (v0.67)
    2,945,920,748 bytes,    219.401 sec. -     57.051 sec., arc -m2 -mt1 (v0.67)
    2,921,997,106 bytes,     96.765 sec. -     14.029 sec., zstd -6 (v1.4.4)
    2,901,657,843 bytes,    381.952 sec. -     85.875 sec., zpaq -m111 -t1 (v7.15)
    2,872,161,150 bytes,    251.839 sec. -     20.411 sec., brotli -q 5 (v1.0.7)
    2,872,160,117 bytes,    244.212 sec. -     21.151 sec., brotli -q 5 --large_window=30 (v1.0.7)
    2,819,369,488 bytes,    131.372 sec. -     13.304 sec., zstd -7 (v1.4.4)
    2,812,779,013 bytes,    412.488 sec. -     64.311 sec., 7z -t7z -mx3 -mmt1 (v19.02)
    2,811,491,748 bytes,    858.957 sec. -    296.051 sec., nlzm -window:30 cf
    2,802,836,601 bytes,    713.652 sec. -     58.594 sec., nz -cDp -t1 (v0.09 alpha)
    2,780,718,316 bytes,    168.903 sec. -     12.921 sec., zstd -8 (v1.4.4)
    2,780,438,122 bytes,    335.040 sec. -     19.575 sec., brotli -q 6 (v1.0.7)
    2,780,429,266 bytes,    324.231 sec. -     20.307 sec., brotli -q 6 --large_window=30 (v1.0.7)
    2,776,664,445 bytes,    767.958 sec. -     58.799 sec., nz -cDP -t1 (v0.09 alpha)
    2,770,795,320 bytes,    761.115 sec. -     58.366 sec., nz -cD -t1 (v0.09 alpha)
    2,766,202,609 bytes,    511.078 sec. -    111.412 sec., arc -m3x -mt1 (v0.67)
    2,764,992,952 bytes,    530.674 sec. -    115.419 sec., arc -m3 -mt1 (v0.67)
    2,750,214,835 bytes,    238.516 sec. -     12.896 sec., zstd -9 (v1.4.4)
    2,746,032,286 bytes,  7,587.384 sec. -     29.749 sec., crush cx
    2,694,582,971 bytes,    288.670 sec. -     12.842 sec., zstd -10 (v1.4.4)
    2,669,751,039 bytes,    374.093 sec. -     13.039 sec., zstd -11 (v1.4.4)
    2,660,370,879 bytes,    153.103 sec. -     19.993 sec., lzturbo -32 -p0 (v1.2)
    2,645,099,063 bytes,    560.829 sec. -     13.136 sec., zstd -12 (v1.4.4)
    2,642,417,079 bytes,    603.066 sec. -     18.940 sec., brotli -q 7 (v1.0.7)
    2,639,801,619 bytes,    606.180 sec. -     19.867 sec., brotli -q 7 --large_window=30 (v1.0.7)
    2,635,875,708 bytes,  6,429.396 sec. -     14.196 sec., lzturbo -29 -p0 (v1.2)
    2,621,986,425 bytes,    842.478 sec. -    131.427 sec., 7z -tbzip2 -mx3 -mmt1 (v19.02)
    2,614,435,940 bytes,    745.449 sec. -     12.699 sec., zstd -13 (v1.4.4)
    2,604,699,073 bytes,  1,722.851 sec. -     81.555 sec., zpaq -m2 -t1 (v7.15)
    2,600,495,919 bytes,  2,546.352 sec. -     27.126 sec., bcrush -9 --optimal (v0.2.0)
    2,573,497,114 bytes,  1,007.449 sec. -     18.956 sec., brotli -q 8 (v1.0.7)
    2,569,453,043 bytes,    967.368 sec. -     12.997 sec., zstd -14 (v1.4.4)
    2,564,942,693 bytes,  1,094.956 sec. -     20.000 sec., brotli -q 8 --large_window=30 (v1.0.7)
    2,559,864,772 bytes,    455.693 sec. -    201.243 sec., balz c (v1.20)
    2,539,608,782 bytes,  1,356.777 sec. -     13.194 sec., zstd -15 (v1.4.4)
    2,536,732,397 bytes,    327.778 sec. -    380.364 sec., sr3 c
    2,533,351,070 bytes,    692.367 sec. -     34.851 sec., rar -m2 -ma5 -mt1 (v5.80)
    2,530,792,260 bytes,    891.915 sec. -    136.339 sec., 7z -tbzip2 -mx5 -mmt1 (v19.02)
    2,530,138,208 bytes,  2,337.551 sec. -    136.101 sec., 7z -tbzip2 -mx7 -mmt1 (v19.02)
    2,530,138,098 bytes,  8,036.375 sec. -    135.975 sec., 7z -tbzip2 -mx9 -mmt1 (v19.02)
    2,512,659,846 bytes,  1,734.988 sec. -     18.909 sec., brotli -q 9 (v1.0.7)
    2,491,076,452 bytes,  2,246.489 sec. -     20.668 sec., brotli -q 9 --large_window=30 (v1.0.7)
    2,465,667,570 bytes,  1,738.196 sec. -    102.153 sec., arc -m4x -mt1 (v0.67)
    2,464,789,337 bytes,  1,757.395 sec. -    106.073 sec., arc -m4 -mt1 (v0.67)
    2,464,084,154 bytes, 13,308.304 sec. -     32.158 sec., blzpack -9 --optimal -b1g (v1.3.0)
    2,450,374,607 bytes,  1,495.895 sec. -     12.653 sec., zstd -16 (v1.4.4)
    2,429,129,903 bytes,  1,492.971 sec. -     34.008 sec., rar -m3 -ma5 -mt1 (v5.80)
    2,393,693,507 bytes,    372.309 sec. -    402.165 sec., ppmd e -m256 -m16 
    2,380,937,714 bytes,  1,784.430 sec. -    188.703 sec., balz cx (v1.20)
    2,380,073,078 bytes,  2,210.100 sec. -     13.032 sec., zstd -17 (v1.4.4)
    2,377,987,975 bytes,  2,855.844 sec. -     34.128 sec., rar -m4 -ma5 -mt1 (v5.80)
    2,364,340,233 bytes,  5,658.734 sec. -     15.333 sec., zstd -22 (v1.4.4)
    2,359,812,241 bytes,  2,255.704 sec. -     96.261 sec., zpaq -m211 -t1 (v7.15)
    2,357,818,671 bytes,  3,953.092 sec. -     34.300 sec., rar -m5 -ma5 -mt1 (v5.80)
    2,347,338,736 bytes,  2,601.788 sec. -     13.256 sec., zstd -18 (v1.4.4)
    2,339,988,137 bytes,    372.592 sec. -     73.660 sec., orz (v1.6.1)
    2,337,506,087 bytes,  2,596.459 sec. -     13.159 sec., zstd -18 --single-thread (v1.4.4)
    2,303,077,470 bytes,  3,402.543 sec. -     13.394 sec., zstd -19 (v1.4.4)
    2,299,225,559 bytes,  3,196.329 sec. -     13.559 sec., zstd -19 --single-thread (v1.4.4)
    2,269,340,648 bytes,  4,296.761 sec. -     14.471 sec., zstd -20 (v1.4.4)
    2,229,014,084 bytes,  5,052.537 sec. -     15.063 sec., zstd -21 (v1.4.4)
    2,220,027,943 bytes,  7,439.064 sec. -     22.690 sec., brotli -q 10 (v1.0.7)
    2,205,939,963 bytes,  3,458.376 sec. -     52.265 sec., 7z -t7z -mx5 -mmt1 (v19.02)
    2,197,171,322 bytes,  3,947.477 sec. -     14.471 sec., zstd -20 --ultra --single-thread (v1.4.4)
    2,172,589,967 bytes, 14,232.804 sec. -     20.310 sec., brotli -q 11 (v1.0.7)
    2,168,961,223 bytes,  4,460.496 sec. -     87.027 sec., arc -m5x -mt1 (v0.67)
    2,166,669,782 bytes,  4,468.655 sec. -     91.262 sec., arc -m5 -mt1 (v0.67)
    2,154,049,598 bytes,  5,317.170 sec. -  5,339.703 sec., bbb cf (v1)
    2,144,118,264 bytes, 33,314.677 sec. - 20,566.793 sec., bbb c (v1.8)
    2,136,340,302 bytes,  4,604.696 sec. -     15.121 sec., zstd -21 --ultra --single-thread (v1.4.4)
    2,129,702,726 bytes,  1,048.281 sec. -    218.249 sec., flashzip -mx3 -k7 -b1024 -t1 (v1.1.3)
    2,126,337,015 bytes,  6,682.010 sec. -     21.238 sec., lzturbo -39 -p0 (v1.2)
    2,114,649,146 bytes,  4,297.935 sec. -     53.732 sec., 7z -t7z -mx7 -mmt1 (v19.02)
    2,108,472,866 bytes,  5,035.285 sec. -     86.917 sec., arc -m6x -mt1 (v0.67)
    2,105,664,967 bytes,  5,058.601 sec. -     91.096 sec., arc -m6 -mt1 (v0.67)
    2,080,479,075 bytes,  5,257.974 sec. -     15.484 sec., zstd -22 --ultra --single-thread (v1.4.4)
    2,059,053,547 bytes,  4,909.124 sec. -     55.188 sec., 7z -t7z -mx9 -mmt1 (v19.02)
    2,051,647,414 bytes,  5,626.398 sec. -     86.924 sec., arc -m7x -mt1 (v0.67)
    2,048,387,913 bytes,  5,643.999 sec. -     90.970 sec., arc -m7 -mt1 (v0.67)
    2,028,356,329 bytes,  8,110.938 sec. -     87.672 sec., lzturbo -49 -p0 (v1.2)
    1,998,346,156 bytes,  6,299.177 sec. -     86.288 sec., arc -m8x -mt1 (v0.67)
    1,995,184,284 bytes,  6,329.592 sec. -     90.224 sec., arc -m8 -mt1 (v0.67)
    1,976,138,624 bytes,  6,585.154 sec. -     85.647 sec., arc -m9x -mt1  (v0.67)
    1,975,858,589 bytes,  1,772.406 sec. -  1,595.793 sec., zpaq -m3 -t1 (v7.15)
    1,973,568,508 bytes,  6,626.946 sec. -     89.762 sec., arc -m9 -mt1 (v0.67)
    1,963,826,556 bytes, 10,050.010 sec. -     28.048 sec., brotli -q 10 --large_window=30 (v1.0.7)
    1,951,631,481 bytes, 24,754.394 sec. -    273.575 sec., nlzm -window:30 c
    1,921,561,064 bytes, 17,200.759 sec. -     27.147 sec., brotli -q 11 --large_window=30 (v1.0.7)
    1,907,435,970 bytes,  1,188.583 sec. -    360.458 sec., nz -co -t1 (v0.09 alpha)
    1,899,403,918 bytes,  1,327.809 sec. -    375.295 sec., nz -cO -t1 (v0.09 alpha)
    1,852,597,854 bytes,    403.030 sec. -    948.316 sec., bsc -b1024 -m6 -e2 -T (v3.1.0)
    1,819,439,647 bytes,  1,593.126 sec. -  1,713.603 sec., ccm 7 (v1.30c)
    1,776,602,973 bytes,  5,371.128 sec. -  1,595.793 sec., zpaq -m4 -t1 (v7.15)
    1,722,407,658 bytes,    778.796 sec. -    401.317 sec., m99 -b1000000000 -t1 (beta)
    1,718,776,810 bytes,  1,267.296 sec. -    896.259 sec., rings -m8 -t1 (v2.5)
    1,708,824,130 bytes,  8,236.253 sec. -  6,174.735 sec., bbb cfm329 (v1)
    1,691,093,945 bytes,  3,971.057 sec. -  4,102.109 sec., cmm4 77 (v0.1e)
    1,690,830,567 bytes,  6,520.194 sec. -  6,912.919 sec., lpq1 (v2)
    1,679,330,904 bytes,    757.786 sec. -    885.980 sec., turborc -26 (v20.01, Jan)
    1,679,319,006 bytes,    765.832 sec. -    540.897 sec., turborc -26 (v20.01, Feb)
    1,675,874,699 bytes,    781.839 sec. -    198.309 sec., bwtturbo -59 -t0 (v20.2)
    1,670,916,589 bytes,  1,890.327 sec. -  2,031.961 sec., zpaq -m311 -t1 (v7.15)
    1,667,596,745 bytes, 16,867.597 sec. - 16,993.461 sec., zpaq -m5 -t1 (v7.15)
    1,648,810,080 bytes,    957.511 sec. -    530.447 sec., bsc -m0 -b1024 -e1 -T  (v3.1.0)
    1,644,097,084 bytes, 21,097.196 sec. -     93.130 sec., rz (v1.03.7)
    1,644,097,084 bytes, 21,468.090 sec. -     93.510 sec., rz -d 1023M (v1.03.7)
    1,641,713,813 bytes, 10,036.931 sec. -  6,287.222 sec., bbb cm1000 (v1)
    1,639,360,679 bytes,  7,899.369 sec. -  7,839.332 sec., nz -cc -t1 (v0.09 alpha)
    1,638,441,156 bytes,  1,030.489 sec. -    640.502 sec., bsc -m0 -b1024 -e2 -T (v3.1.0)
    1,638,368,188 bytes, 37,005.367 sec. - 21,654.024 sec., bbb cm1000 (v1.6)
    1,635,996,697 bytes,  9,828.001 sec. -  7,679.925 sec., bbb cm1000 (v1.8)
    1,632,628,624 bytes,  1,146.133 sec. -  1,284.451 sec., bcm -9 (v1.40)
    1,625,985,127 bytes,  6,338.974 sec. -  6,519.437 sec., zpaq -m411 -t1 (v7.15)
    1,595,984,275 bytes,  2,165.922 sec. -  2,160.740 sec., zcm -m8 -t1 (v0.93)
    1,499,545,945 bytes, 18,897.682 sec. - 19,180.835 sec., zpaq -m511 -t1 (v7.15)
    1,450,364,034 bytes,  2,701.335 sec. -  2,433.988 sec., mcm -x -m11 (v0.83)
    Test sytem:
    CPU: Intel Core i9 9900K OC at 5.2GHz (hyper threading disabled)
    Memory: 4 x 8GB = 32GB DDR4 at 4273MHz, timings 17-18-18-18
    Storage: 2 x Samsung M.2 NVMe 960 PRO 512GB = 1TB in RAID 0 (via motherboard BIOS)
    Storage speed: 3,500MB/s read 2,100MB/s write each NVMe (Samsung) as RAID 0 3,272MB/s read, 2,704MB/s write (AS SSD Benchmark)
    OS: Windows 10 Pro v1909 (not used services disabled, page file disabled)
    Time: Timer 9.01 (Igor Pavlov, Public Domain, 2009-05-31)

    enwik10 is created with (in this order):
    enwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    dewiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    frwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    ruwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    eswiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    jawiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    itwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    plwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    zhwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    ptwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)

    MD5 hashes:
    E72AD677A071156C82A8BF6A42BBC776 enwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    7C91E91BC82C65EFB6C8C04592E14DD0 dewiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    2F6166B188AE9058468AC2D60E506F73 frwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    ED8ECA4510032DC49A6E5BB101C6B6F2 ruwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    636698795D7D1B9D821A4E40CE937816 eswiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    267D4DADA461A80EB9CE3B895CB01528 jawiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    B356D4CE79D52A634C609A9C269A5310 itwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    3686B5884B2C2DB760090F5DE53205D6 plwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    6C1CD33FF77A739FF6F3AFF8C4C2871B zhwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    77FF2EEC04C8437D86E1EF6515DA2931 ptwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    C78C429EB199C18C3FCB8190B9E31FEC enwik10 (merged in above order 10,000,000,000 bytes)

    SHA-1 hashes:
    B08A8ED7AE8128EE4A736368DFD8DB52564F4195 enwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    49B1D4B7DE17E582A0C439DA8AF81ED317C1A2B2 dewiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    D75F345A01749AA33778BEDD221AC8A8C8AA0664 frwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    21DCF00A7F6479FEEA6B48A3833463586A29184E ruwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    7540726185D119C7E71EC16F21CA6C51CCB6DDEF eswiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    2380935E63CEDE71E0FBC1DD0938EE1669C6D489 jawiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    D2D66908D5FAEDE33F5185535711D59590F6349B itwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    F1EA6FE9C20C3E49257B2FF2A3FC4F23B56A5C47 plwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    416C0955773D513D12F344A43382E1D4732F30C2 zhwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    16D8E82EB1BC0927B706D2BBA1BA5765E452FE61 ptwiki-20200101-pages-articles-multistream.xml (first 1,000,000,000 bytes)
    4A70DA5D8C11CFAF5744213BE069C0780C6D30FB enwik10 (merged in above order 10,000,000,000 bytes)

    enwik10 date/time:
    ‎January ‎13, ‎2020, ‏‎02:30:45 (AM)
    Last edited by Sportman; 18th February 2020 at 01:39.

  2. Thanks (11):

    Cyan (15th January 2020),dnd (15th January 2020),encode (22nd January 2020),JamesB (23rd January 2020),JamesWasil (25th January 2020),Jarek (16th January 2020),jibz (24th January 2020),kaitz (21st January 2020),mhajicek (5th February 2020),michael maniscalco (23rd January 2020),Shelwien (15th January 2020)

  3. #2
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    74
    Thanks
    15
    Thanked 5 Times in 5 Posts
    where can i get the test file ? could you upload to this forum ?

  4. #3
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    881
    Thanks
    80
    Thanked 315 Times in 219 Posts
    Quote Originally Posted by bwt View Post
    where can i get the test file ? could you upload to this forum ?
    To big (also for my public cloud account) 1.644GB as Razor compressed.

  5. #4
    Member
    Join Date
    Apr 2018
    Location
    Indonesia
    Posts
    74
    Thanks
    15
    Thanked 5 Times in 5 Posts
    so could you test crush v1.4 in enwik10 benchmark ? thanks

  6. #5
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,717
    Thanks
    271
    Thanked 1,185 Times in 656 Posts
    You can download them from the source: https://dumps.wikimedia.org/other/st...dumps/current/

  7. #6

  8. Thanks (3):

    JamesB (15th January 2020),Shelwien (15th January 2020),suryakandau@yahoo.co.id (15th January 2020)

  9. #7
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,532
    Thanks
    755
    Thanked 674 Times in 365 Posts
    pretty meaningless test as ja/pl text compression is of no much interest. this reminds me Matt's testset that included large DNA among usual files

  10. #8
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    881
    Thanks
    80
    Thanked 315 Times in 219 Posts
    Quote Originally Posted by Bulat Ziganshin View Post
    pretty meaningless test as ja/pl text compression is of no much interest.
    For who I advice it's useful.

    The order is by most written Wikipedia text languages.
    This mean more languages with new keywords and 2020 text with modern keywords (obamacare, bitcoin, whatsapp etc.) and bigger test file because hardware is faster with more storage and more memory. For example my 2008 benchmarks where on 3 x 60GB SSD's in RAID 5.

  11. #9
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,532
    Thanks
    755
    Thanked 674 Times in 365 Posts
    my message is mainly to those who think it's a good way to compare compressors. the next stage may be fine-tuning them for i18l texts

  12. #10
    Member FatBit's Avatar
    Join Date
    Jan 2012
    Location
    Prague, CZ
    Posts
    190
    Thanks
    0
    Thanked 36 Times in 27 Posts
    You can use www.uschovna.cz = up to 30 GB/14 days, free of charge and without registration.

    Best regards,

    FatBit

  13. #11
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    565
    Thanks
    67
    Thanked 198 Times in 147 Posts
    The strongest bsc is with option -m0 -e2 (bwt)

  14. #12
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    881
    Thanks
    80
    Thanked 315 Times in 219 Posts
    Quote Originally Posted by dnd View Post
    The strongest bsc is with option -m0 -e2 (bwt)
    Indeed, added.

  15. #13
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    752
    Thanks
    216
    Thanked 283 Times in 165 Posts
    Sad to see no large window results for brotli. People get confused when small window (practical) brotli is compared with huge window size in other algorithms.

    Try --large_window 30

  16. #14
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,717
    Thanks
    271
    Thanked 1,185 Times in 656 Posts
    I made and uploaded enwik10, please check: https://mega.nz/#!9f4DxYJR!wgX4bEvR9...ZIQ9jyTo1QJ0Hg
    Also size:
    Code:
    https://mega.nz/#!9f4DxYJR!wgX4bEvR96sPCcVLWu05TUA3WmgM9ZIQ9jyTo1QJ0Hg
    1,872,327,733 // 7z a -mx=9 -mf=off -ms=1t -m0=lzma:d=1536m:lc=8:lp=0:pb=0:fb=273 enwik10.7z enwik10

  17. Thanks:

    Sportman (23rd January 2020)

  18. #15
    Programmer michael maniscalco's Avatar
    Join Date
    Apr 2007
    Location
    Boston, Massachusetts, USA
    Posts
    139
    Thanks
    20
    Thanked 94 Times in 31 Posts
    Quote Originally Posted by Shelwien View Post
    I made and uploaded enwik10, please check: https://mega.nz/#!9f4DxYJR!wgX4bEvR9...ZIQ9jyTo1QJ0Hg
    Also size:
    Code:
    https://mega.nz/#!9f4DxYJR!wgX4bEvR96sPCcVLWu05TUA3WmgM9ZIQ9jyTo1QJ0Hg
    1,872,327,733 // 7z a -mx=9 -mf=off -ms=1t -m0=lzma:d=1536m:lc=8:lp=0:pb=0:fb=273 enwik10.7z enwik10
    I tarred the 10 files together so there's a bit more overhead in my test. M99 is still a good competitor for compression vs speed as well.

    Code:
    m99 e enwik10 enwik10.m99 -b1000000000
    compressed: 10000015360 -> 1720236193 bytes.  ratio = 17.2023%
    Elapsed time: 396.497 seconds : 24.0525 MB/sec
    
    m99 d enwik10.m99 enwik10
    Elapsed time: 236.818 seconds
    Code:
    lscpu
    CPU(s):              12
    On-line CPU(s) list: 0-11
    Thread(s) per core:  2
    Core(s) per socket:  6
    Socket(s):           1
    NUMA node(s):        1
    Vendor ID:           GenuineIntel
    CPU family:          6
    Model:               158
    Model name:          Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz

  19. #16
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    881
    Thanks
    80
    Thanked 315 Times in 219 Posts
    Quote Originally Posted by Jyrki Alakuijala View Post
    Try --large_window 30
    Added, till -q 3 no effect, -q 4 negative effect, -q 5 till -q -9 small positive effect, -q 10 and -q 11 big positive effect.

  20. Thanks:

    Jyrki Alakuijala (24th January 2020)

  21. #17
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    881
    Thanks
    80
    Thanked 315 Times in 219 Posts
    Quote Originally Posted by michael maniscalco View Post
    M99 is still a good competitor for compression vs speed as well.
    Is there a Windows binary? I could only find an old 32-bit m99 (3/26/2007) binary, but that one crash by start.

  22. #18
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,717
    Thanks
    271
    Thanked 1,185 Times in 656 Posts
    https://github.com/michaelmaniscalco/m99
    http://nishi.dreamhosters.com/u/m99_v0.7z

    M99 with modified source to compile on windows. Seems to work [?]

  23. Thanks (2):

    michael maniscalco (23rd January 2020),Sportman (24th January 2020)

  24. #19
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    881
    Thanks
    80
    Thanked 315 Times in 219 Posts
    Quote Originally Posted by michael maniscalco View Post
    M99 is still a good competitor for compression vs speed as well.
    Indeed, added.

  25. #20
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    881
    Thanks
    80
    Thanked 315 Times in 219 Posts
    Quote Originally Posted by Shelwien View Post
    Seems to work [?]
    Yes!

    -b1048576 -t1 - 2,599,535,211 bytes:
    437.143 sec. - 275.518 sec., m99_clang90_k8
    433.678 sec. - 315.293 sec., m99_gcc82_k8
    446.727 sec. - 275.459 sec., m99_ic19_SSE2

  26. Thanks:

    michael maniscalco (24th January 2020)

  27. #21
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    155
    Thanks
    15
    Thanked 17 Times in 15 Posts
    Quote Originally Posted by Sportman View Post
    Yes!

    -b1048576 -t1 - 2,599,535,211 bytes:
    437.143 sec. - 275.518 sec., m99_clang90_k8
    433.678 sec. - 315.293 sec., m99_gcc82_k8
    446.727 sec. - 275.459 sec., m99_ic19_SSE2

    crush v1.9 use -9 option on enwik10:
    10000000000 -> 2498911772 in 13323.99s
    2498911772 -> 10000000000 in 351.06s

  28. #22
    Programmer michael maniscalco's Avatar
    Join Date
    Apr 2007
    Location
    Boston, Massachusetts, USA
    Posts
    139
    Thanks
    20
    Thanked 94 Times in 31 Posts
    Quote Originally Posted by Sportman View Post
    Yes!

    -b1048576 -t1 - 2,599,535,211 bytes:
    437.143 sec. - 275.518 sec., m99_clang90_k8
    433.678 sec. - 315.293 sec., m99_gcc82_k8
    446.727 sec. - 275.459 sec., m99_ic19_SSE2
    I'm curious as to why you disable multithreading? That's where most of the underlying algorithms really shine.

    - Michael

  29. #23
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,717
    Thanks
    271
    Thanked 1,185 Times in 656 Posts
    Maybe make another benchmark with MT?
    Half of the listed programs actually support MT.

  30. #24
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    881
    Thanks
    80
    Thanked 315 Times in 219 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    crush v1.9 use -9 option on enwik10
    Compare fail.

  31. #25
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    881
    Thanks
    80
    Thanked 315 Times in 219 Posts
    Quote Originally Posted by michael maniscalco View Post
    I'm curious as to why you disable multithreading?
    To have fair single core compare.
    I had planned to do a multi-core benchmark, but to do that, I need to pump up the liquid cooling pump speed and radiator fans speed from unhearable to a very loud level and many programs already failed with the single-core benchmark and this big file.

  32. Thanks (2):

    michael maniscalco (24th January 2020),Shelwien (24th January 2020)

  33. #26
    Programmer michael maniscalco's Avatar
    Join Date
    Apr 2007
    Location
    Boston, Massachusetts, USA
    Posts
    139
    Thanks
    20
    Thanked 94 Times in 31 Posts
    Quote Originally Posted by Shelwien View Post
    Maybe make another benchmark with MT?
    Half of the listed programs actually support MT.
    Exactly. Some algorithms benefit more from MT than others. Even some implementations of the same algorithm scale better than others. In the case of M99 you get basically the same compression ratio and same memory requirements no matter how many cores you use. But the throughput increases dramatically. You can't say the same of other BWT implementations. Good luck using multiple cores along with 1G block sizes for BSC for instance. You'll be in swap-ville in no time!

    So it all depends on what the test is intended to guage, i guess.

  34. #27
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    155
    Thanks
    15
    Thanked 17 Times in 15 Posts
    Quote Originally Posted by Sportman View Post
    Compare fail.

    What do you mean by compare fail ? There is no error while I compress n decompress it

  35. #28
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    565
    Thanks
    67
    Thanked 198 Times in 147 Posts
    Added, till -q 3 no effect, -q 4 negative effect, -q 5 till -q -9 small positive effect, -q 10 and -q 11 big positive effect.

    TurboBench Compression Benchmark is using large window brotli per default for quality levels 10 and 11 only.
    For lower levels there are little benefits in compression ratio, but the decompression can be significantly slower.

    @Shelwien, @bwt
    is it possible to delete the duplicated list ? it's really confusing when people are copying large posts when replying

    Last edited by dnd; 24th January 2020 at 14:12.

  36. #29
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    881
    Thanks
    80
    Thanked 315 Times in 219 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    What do you mean by compare fail ? There is no error while I compress n decompress it
    Yes no error, length ok, but decompressed output content not ok.

    For example original:
    |last=Stubblefield|first=Phillip G. |

    Crush 1.9 output:
    |last=Stubblt;ref |first=Phillip G. |

  37. #30
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    752
    Thanks
    216
    Thanked 283 Times in 165 Posts
    Such a large benchmark comes with inconveniences in use and distribution. Therefore it should carry some benefit, too. Consider quantifying this, for example showing how much the order of compression algorithms change if you only take 100 or 500 MB in each corpus vs the full 1GB.

Page 1 of 4 123 ... LastLast

Similar Threads

  1. Benchmark results on GitHub
    By Piotr Tarsa in forum Data Compression
    Replies: 40
    Last Post: 30th May 2018, 06:47
  2. large window brotli results available for a several corpora
    By Jyrki Alakuijala in forum Data Compression
    Replies: 9
    Last Post: 3rd May 2018, 09:48
  3. Replies: 3
    Last Post: 30th July 2011, 15:48
  4. Strange gcc4.3 results with paq8o8
    By Hahobas in forum Forum Archive
    Replies: 8
    Last Post: 22nd March 2008, 20:44
  5. TC 5.1dev7x test results
    By LovePimple in forum Forum Archive
    Replies: 8
    Last Post: 23rd January 2007, 23:00

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •