Page 2 of 2 FirstFirst 12
Results 31 to 34 of 34

Thread: gzip on a chip

  1. #31
    Member SolidComp's Avatar
    Join Date
    Jun 2015
    Location
    USA
    Posts
    239
    Thanks
    95
    Thanked 47 Times in 31 Posts
    Quote Originally Posted by Jyrki Alakuijala View Post
    But... often there is only one compressor/decompressor to be shared by say 18 cores. I very much would prefer the approach where hardware compression is added as new (somewhat) general purpose instructions that happen to be beneficial for reading n bits, helping with parallel ANS, doing faster match finding, hashing, etc.
    Interestingly, Intel has introduce a sort of compromise between the two approaches. Some of the Scalable Xeon/Skylake server chips come with Quick Assist accelerators. These are dedicated chips for compression and encryption, but they're not ASICs. I think they're Atom cores. I'm not sure how or why they're able to hit such high throughput.

  2. #32
    Member SolidComp's Avatar
    Join Date
    Jun 2015
    Location
    USA
    Posts
    239
    Thanks
    95
    Thanked 47 Times in 31 Posts
    Quote Originally Posted by Jyrki Alakuijala View Post
    But... often there is only one compressor/decompressor to be shared by say 18 cores. I very much would prefer the approach where hardware compression is added as new (somewhat) general purpose instructions that happen to be beneficial for reading n bits, helping with parallel ANS, doing faster match finding, hashing, etc.
    By the way, have you thought of some useful CPU instructions to accelerate Brotli or PIK?

  3. #33
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    745
    Thanks
    215
    Thanked 281 Times in 164 Posts
    Quote Originally Posted by SolidComp View Post
    By the way, have you thought of some useful CPU instructions to accelerate Brotli or PIK?
    Brotli encoding: instructions related to prefix coding or match finding

    Brotli decoding: instructions related to prefix coding, possibly with context modeling

    PIK/JPEG XL: more SIMD: SSE4.2, AVX, AVX2, AVX512, AVX1024, AVX2048, ...

  4. #34
    Member
    Join Date
    Jun 2009
    Location
    Kraków, Poland
    Posts
    1,478
    Thanks
    26
    Thanked 122 Times in 96 Posts
    Quote Originally Posted by Jyrki Alakuijala View Post
    Brotli encoding: instructions related to (...) match finding
    SSE4.2 has PCMPxSTRy instructions. Have they been useful for match finding?

    Quote Originally Posted by Jyrki Alakuijala View Post
    PIK/JPEG XL: more SIMD: SSE4.2, AVX, AVX2, AVX512, AVX1024, AVX2048, ...
    ARM SVE (Scalable Vector Extension) actually takes care of that increasing vector length (in case of ARM of course): https://community.arm.com/tools/hpc/...a-architecture
    Rather than specifying a specific vector length, SVE allows CPU designers to choose the most appropriate vector length for their application and market, from 128 bits up to 2048 bits per vector register. SVE also supports a vector-length agnostic (VLA) programming model that can adapt to the available vector length. Adoption of the VLA paradigm allows you to compile or hand-code your program for SVE once, and then run it at different implementation performance points, while avoiding the need to recompile or rewrite it when longer vectors appear in the future. This reduces deployment costs over the lifetime of the architecture; a program just works and executes wider and faster.
    (I've corrected "doption" to "Adoption")

Page 2 of 2 FirstFirst 12

Similar Threads

  1. Replies: 2
    Last Post: 28th December 2017, 12:35
  2. Fast implementation of gzip in FPGA
    By SolidComp in forum Data Compression
    Replies: 1
    Last Post: 8th May 2017, 04:48
  3. gzip - Intel IPP
    By M4ST3R in forum Download Area
    Replies: 5
    Last Post: 2nd June 2010, 16:09
  4. Gzip 1.2.4 hack (OpenWatcom compiles)
    By Rugxulo in forum Data Compression
    Replies: 9
    Last Post: 22nd May 2009, 01:17
  5. gzip-1.2.4-hack - a hacked version of gzip
    By encode in forum Forum Archive
    Replies: 63
    Last Post: 10th September 2007, 05:16

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •