Results 1 to 25 of 25

Thread: Meltdown and Spectre impacts on modern compression codecs

  1. #1
    Member SolidComp's Avatar
    Join Date
    Jun 2015
    Location
    USA
    Posts
    353
    Thanks
    131
    Thanked 54 Times in 38 Posts

    Meltdown and Spectre impacts on modern compression codecs

    Hi all – Does anyone know if the Meltdown and Spectre patches slow down compression or decompression in general? I'm not clear on what kinds of operations suffer the most slowdowns. I've heard databases are impacted, and certain kinds of I/O, but I've not dug into the details yet.

    Jyrki – Is brotli slower on patched systems? Is Zstd slower? What about something as basic as zlib/gzip?

    Joe

  2. Thanks:

    Jyrki Alakuijala (5th April 2018)

  3. #2
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    876
    Thanks
    242
    Thanked 325 Times in 198 Posts
    Quote Originally Posted by SolidComp View Post
    Hi all – Does anyone know if the Meltdown and Spectre patches slow down compression or decompression in general? I'm not clear on what kinds of operations suffer the most slowdowns. I've heard databases are impacted, and certain kinds of I/O, but I've not dug into the details yet.

    Jyrki – Is brotli slower on patched systems? Is Zstd slower? What about something as basic as zlib/gzip?

    Joe
    I don't know, but I'm very interested to learn about this if anyone can share benchmark results.

  4. #3
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Input 187,074,408,448 bytes - all Bitcoin blockchain blocks till block 516,394 as .tar
    Output 152,713,202,154 bytes - botli -1

    Virtual Memory = 2 MB
    Physical Memory = 3 MB

    OS boot single core:

    Meltdown and Spectre disabled:

    Kernel User Process Global
    59.890 613.500 673.390 850.586
    62.796 611.000 673.796 848.259
    60.843 613.890 674.734 852.859

    Meltdown and Spectre enabled:

    Kernel User Process Global
    73.687 613.593 687.281 878.694
    74.312 610.078 684.390 885.106
    75.453 611.203 686.656 886.246

    ------------------------------

    OS boot multi core:

    Meltdown and Spectre disabled:

    Kernel User Process Global
    57.187 604.843 662.031 673.131
    59.343 609.437 668.781 679.900
    57.921 606.796 664.718 675.992

    Meltdown and Spectre enabled:

    Kernel User Process Global
    68.890 606.656 675.546 686.262
    68.531 611.375 679.906 699.245
    69.109 604.968 674.078 684.374

    ---------------------------------------------------------------

    Input 187,074,408,448 bytes - all Bitcoin blockchain blocks till block 516,394 as .tar
    Output 148,029,657,529 bytes - zstd -1

    Virtual Memory = 14 MB
    Physical Memory = 17 MB

    OS boot single core:

    Meltdown and Spectre disabled:

    Kernel User Process Global
    42.140 1198.734 1240.875 1370.639
    39.578 1209.593 1249.171 1377.740
    39.125 1210.765 1249.890 1379.169

    Meltdown and Spectre enabled:

    Kernel User Process Global
    45.453 1215.265 1260.718 1398.678
    47.328 1206.718 1254.046 1394.858
    47.687 1196.531 1244.218 1385.758

    ---------------------------------

    OS boot multi core:

    Meltdown and Spectre disabled:

    Kernel User Process Global
    63.687 656.328 720.015 643.378
    64.687 655.453 720.140 633.118
    66.859 654.281 721.140 632.139

    Meltdown and Spectre enabled:

    Kernel User Process Global
    76.062 656.359 732.421 633.314
    74.281 655.250 729.531 648.363
    72.546 656.562 729.109 647.317

  5. Thanks (4):

    Cyan (9th April 2018),Dresdenboy (20th June 2020),Gotty (13th April 2018),JamesB (11th April 2018)

  6. #4
    Member
    Join Date
    Sep 2008
    Location
    France
    Posts
    889
    Thanks
    483
    Thanked 279 Times in 119 Posts
    Definitely a pretty sensible impact.

    Since Meltdown & Spectre mitigation is a kernel topic, I guess most of the extra-load is within kernel-related stuff,
    such as threading, task-switching, and most I/O interactions generally.

    It would be interesting to know more about the sources of these slow-down,
    to understand if user code can do something to reduce triggering them.

  7. #5
    Member
    Join Date
    Jul 2014
    Location
    Mars
    Posts
    200
    Thanks
    136
    Thanked 13 Times in 12 Posts
    +1 to know

  8. #6
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    925
    Thanks
    58
    Thanked 116 Times in 93 Posts
    when you disabled the meldtodnw spectra patc h did you do it through the registry or avoide installing the patch all together ?

  9. #7
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by SvenBent View Post
    when you disabled the meldtodnw spectra patc h did you do it through the registry or avoide installing the patch all together ?
    I did not touched the BIOS and OS patches, I used the InSpectre utility https://www.grc.com/inspectre.htm to disable or enable Meltdown and Spectre before reboot.

  10. #8
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Spectre/Meltdown fixes in HPC: Want the bad news or the bad news? It's slower, say boffins:
    https://www.theregister.co.uk/2018/0...e_batters_hpc/

  11. #9
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Intel Gags Customers from Publishing Performance Impact of Microcode Updates:
    https://www.techpowerup.com/247028/i...rocode-updates

  12. #10
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    We have simplified the Intel license to make it easier to distribute CPU microcode updates and posted the new version here:
    https://twitter.com/imadsousou/statu...80311753072640
    https://01.org/mcu-path-license-2018

    Not updated here yet:
    https://downloadcenter.intel.com/dow...code-Data-File

  13. #11
    Member
    Join Date
    Feb 2015
    Location
    United Kingdom
    Posts
    176
    Thanks
    28
    Thanked 74 Times in 44 Posts
    Looks like we can expect some more performance loss now that hyperthreading is considered a security risk.
    https://www.theregister.co.uk/2018/0...yperthreading/
    https://www.techspot.com/news/75240-...intel-cpu.html

  14. #12
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts

  15. #13
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    876
    Thanks
    242
    Thanked 325 Times in 198 Posts
    Quote Originally Posted by Sportman View Post
    Memory, network and disk/flash can stop the cpu(s) making progress, and particularly so on multi-core systems. Compression can help with all of these, but a compression system is often a memory system hog, and no benchmarking that I know of looks into the memory system load. I believe the impact of the memory system can be 50-100 % speed differences in multiprocessing scaling. For example, two codecs run in A seconds and B seconds, but if you run 10 copies of these codecs in parallel, the other codec runs in A*2 seconds and the other B*4 seconds.

  16. #14
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    MDS (Micro-architectural Data Sampling) Tool from RIDL (Rogue In-Flight Data Load) Team:
    https://mdsattacks.com/files/mdstool-win.zip
    https://mdsattacks.com/files/mdstool-linux.zip
    https://github.com/vusec/ridl

  17. Thanks:

    Shelwien (19th May 2019)

  18. #15

  19. #16
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts

  20. #17
    Member SolidComp's Avatar
    Join Date
    Jun 2015
    Location
    USA
    Posts
    353
    Thanks
    131
    Thanked 54 Times in 38 Posts
    I was about to post about TSX to see if anyone had found a use for these instructions in compression. It's surprising that they already have security issues – no one seems to be using TSX since they were disabled in many Haswell or Broadwell CPUs.

  21. #18
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,977
    Thanks
    296
    Thanked 1,306 Times in 742 Posts
    TSX provides a way to restart a chunk of code when a conflict happens during shared memory access: https://gcc.gnu.org/onlinedocs/gcc/x...ntrinsics.html
    I suppose it can be useful for some slow MT compressor, like NNCP.
    Normally compression doesn't require frequent inter-thread communication because smaller processing units mean worse compression.

  22. Thanks:

    JamesWasil (12th June 2020)

  23. #19
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts

  24. Thanks:

    JamesWasil (12th June 2020)

  25. #20
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    The Ongoing CPU Security Mitigation Impact On The Core i9 10900K Comet Lake:
    "At least for the workloads tested this round, when booting the new Intel Core i9 10900K "Comet Lake" processor with the software-controlled CPU security mitigations disabled, the overall performance was elevated by about 6% depending upon the workload."
    https://www.phoronix.com/scan.php?pa...0k-mitigations

  26. Thanks:

    JamesWasil (12th June 2020)

  27. #21

  28. Thanks:

    JamesWasil (12th June 2020)

  29. #22
    Member JamesWasil's Avatar
    Join Date
    Dec 2017
    Location
    Arizona
    Posts
    103
    Thanks
    96
    Thanked 18 Times in 17 Posts
    Things like this make me wish that Cyrix and a handful of other CPU makers had been able to continue unabated. Even if they needed special cooling to run at increased speeds for performance, it would have been a safe and practical alternative to Intel and AMD CPUs today affected by Spectre and Meltdown.

    With more choices besides ARM and alternative (pin compatible) mainstream processors, this might not have been such an issue...

    Well, if anyone needs me, I'll be at home on my 80286 6Mhz XT running SCO Xenix and a Lynx browser safely to read Encode.su lol

  30. #23
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,572
    Thanks
    780
    Thanked 687 Times in 372 Posts
    the problem isn't on cpu manufacturer side, but in CS itself. any company with modern CPU architecture will have the same problem

  31. #24
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    925
    Thanks
    58
    Thanked 116 Times in 93 Posts
    So I did some testing myself cause i was borred

    Windows10 pro Build 18363
    Intel I5 4690S

    I changed to registry values for memory manages in windows, and after reboot verified meltdown and spectre protectio nwas off with GRC inspectre

    GeekBench 5

    Patches Off
    902 2894
    844 2742
    852 2763
    843 2789
    841 2784

    Patches ON
    914 2891
    841 2748
    839 2762
    843 2767
    838 2757


    Winsat CPU

    Patches off
    > CPU LZW Compression 319.47 MB/s
    > CPU AES256 Encryption 1851.00 MB/s
    > CPU Vista Compression 855.06 MB/s
    > CPU SHA1 Hash 2022.84 MB/s
    > Uniproc CPU LZW Compression 87.14 MB/s
    > Uniproc CPU AES256 Encryption 515.95 MB/s
    > Uniproc CPU Vista Compression 234.13 MB/s
    > Uniproc CPU SHA1 Hash 564.32 MB/s

    > CPU LZW Compression 319.45 MB/s
    > CPU AES256 Encryption 1850.78 MB/s
    > CPU Vista Compression 855.84 MB/s
    > CPU SHA1 Hash 2022.08 MB/s
    > Uniproc CPU LZW Compression 88.02 MB/s
    > Uniproc CPU AES256 Encryption 515.98 MB/s
    > Uniproc CPU Vista Compression 234.96 MB/s
    > Uniproc CPU SHA1 Hash 563.92 MB/s

    > CPU LZW Compression 319.17 MB/s
    > CPU AES256 Encryption 1850.86 MB/s
    > CPU Vista Compression 852.82 MB/s
    > CPU SHA1 Hash 2023.01 MB/s
    > Uniproc CPU LZW Compression 87.85 MB/s
    > Uniproc CPU AES256 Encryption 515.97 MB/s
    > Uniproc CPU Vista Compression 236.56 MB/s
    > Uniproc CPU SHA1 Hash 564.43 MB/s

    > CPU LZW Compression 318.79 MB/s
    > CPU AES256 Encryption 1851.34 MB/s
    > CPU Vista Compression 851.52 MB/s
    > CPU SHA1 Hash 2022.77 MB/s
    > Uniproc CPU LZW Compression 87.57 MB/s
    > Uniproc CPU AES256 Encryption 515.72 MB/s
    > Uniproc CPU Vista Compression 237.50 MB/s
    > Uniproc CPU SHA1 Hash 564.34 MB/s

    > CPU LZW Compression 319.87 MB/s
    > CPU AES256 Encryption 1851.17 MB/s
    > CPU Vista Compression 855.08 MB/s
    > CPU SHA1 Hash 2022.65 MB/s
    > Uniproc CPU LZW Compression 87.74 MB/s
    > Uniproc CPU AES256 Encryption 516.03 MB/s
    > Uniproc CPU Vista Compression 236.99 MB/s
    > Uniproc CPU SHA1 Hash 564.21 MB/s


    Patches ON
    > CPU LZW Compression 319.56 MB/s
    > CPU AES256 Encryption 1850.28 MB/s
    > CPU Vista Compression 858.72 MB/s
    > CPU SHA1 Hash 2021.81 MB/s
    > Uniproc CPU LZW Compression 85.56 MB/s
    > Uniproc CPU AES256 Encryption 502.77 MB/s
    > Uniproc CPU Vista Compression 230.05 MB/s
    > Uniproc CPU SHA1 Hash 563.35 MB/s

    > CPU LZW Compression 319.35 MB/s
    > CPU AES256 Encryption 1849.97 MB/s
    > CPU Vista Compression 859.28 MB/s
    > CPU SHA1 Hash 2021.58 MB/s
    > Uniproc CPU LZW Compression 85.54 MB/s
    > Uniproc CPU AES256 Encryption 502.79 MB/s
    > Uniproc CPU Vista Compression 229.36 MB/s
    > Uniproc CPU SHA1 Hash 549.81 MB/s

    > CPU LZW Compression 319.20 MB/s
    > CPU AES256 Encryption 1852.14 MB/s
    > CPU Vista Compression 858.35 MB/s
    > CPU SHA1 Hash 2023.02 MB/s
    > Uniproc CPU LZW Compression 86.52 MB/s
    > Uniproc CPU AES256 Encryption 515.45 MB/s
    > Uniproc CPU Vista Compression 229.39 MB/s
    > Uniproc CPU SHA1 Hash 549.85 MB/s

    > CPU LZW Compression 318.93 MB/s
    > CPU AES256 Encryption 1851.65 MB/s
    > CPU Vista Compression 858.41 MB/s
    > CPU SHA1 Hash 2022.45 MB/s
    > Uniproc CPU LZW Compression 85.50 MB/s
    > Uniproc CPU AES256 Encryption 503.07 MB/s
    > Uniproc CPU Vista Compression 229.60 MB/s
    > Uniproc CPU SHA1 Hash 549.57 MB/s

    > CPU LZW Compression 319.79 MB/s
    > CPU AES256 Encryption 1852.22 MB/s
    > CPU Vista Compression 858.68 MB/s
    > CPU SHA1 Hash 2021.78 MB/s
    > Uniproc CPU LZW Compression 85.49 MB/s
    > Uniproc CPU AES256 Encryption 502.80 MB/s
    > Uniproc CPU Vista Compression 233.93 MB/s
    > Uniproc CPU SHA1 Hash 563.41 MB/s



    Distributed.NET

    Patches OFF
    OGR-NG: Benchmark for core #2 (cj-asm-sse2)
    87,261,778 nodes/sec
    87,990,949 nodes/sec
    87,539,461 nodes/sec
    87,857,098 nodes/sec
    87,956,939 nodes/sec

    RC5-72: Benchmark for core #4 (YK AVX2)
    48,599,183 keys/sec
    48,739,659 keys/sec
    48,608,994 keys/sec
    48,537,309 keys/sec
    48,640,024 keys/sec


    Patches ON
    OGR-NG: Benchmark for core #2 (cj-asm-sse2)
    88,450,000 nodes/sec
    88,756,774 nodes/sec
    88,859,554 nodes/sec
    89,011,961 nodes/sec
    89,208,522 nodes/sec

    RC5-72: Benchmark for core #4 (YK AVX2)
    48,969,666 keys/sec
    48,737,423 keys/sec
    49,042,076 keys/sec
    49,241,224 keys/sec
    49,122,235 keys/sec

  32. #25
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Input:
    66,965,935,529 bytes, BTC 0-xxxx blocks as RAR storage:

    Meltdown and Spectre disabled:

    Output:
    53,410,095,068 bytes, brotli -0:
    183.063 sec.
    183.010 sec.
    183.157 sec.

    Output:
    51,614,569,992 bytes, zstd -1:
    164.479 sec.
    165.214 sec.
    164.929 sec.

    ------------------------------------

    Meltdown and Spectre enabled:

    Output:
    53,410,095,068 bytes, brotli -0:
    184.210 sec.
    184.534 sec.
    184.138 sec.

    Output:
    51,614,569,992 bytes, zstd -1:
    166.276 sec.
    166.591 sec.
    166.774 sec.

Similar Threads

  1. Psychovisual analysis on modern lossy image codecs
    By khavish in forum Data Compression
    Replies: 94
    Last Post: 17th August 2018, 07:17
  2. Psychovisual measurements on modern image codecs
    By khavish in forum Data Compression
    Replies: 28
    Last Post: 28th August 2017, 02:41
  3. Optimized compression codecs on Intel's Clear Linux
    By SolidComp in forum Data Compression
    Replies: 4
    Last Post: 11th March 2017, 01:58
  4. New Compression Codecs Risk Making Zlib Obsolete
    By dnd in forum Data Compression
    Replies: 7
    Last Post: 19th January 2016, 19:01
  5. Modern BWT speed
    By GerryB in forum Data Compression
    Replies: 4
    Last Post: 5th May 2009, 17:28

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •