Results 1 to 6 of 6

Thread: gzip - Intel IPP

  1. #1
    Member
    Join Date
    May 2009
    Location
    Europe
    Posts
    67
    Thanks
    0
    Thanked 1 Time in 1 Post

    gzip - Intel IPP

    Intel? Integrated Performance Primitives (Intel? IPP) is an extensive library of multicore-ready, highly optimized software functions for digital media and data-processing applications. Intel IPP offers thousands of optimized functions covering frequently-used fundamental algorithms. Intel IPP functions are designed to deliver performance beyond what optimized compilers alone can deliver.
    The Data Compression-based IPP_GZIP Sample (IPP_GZIP) illustrates the way of implementing effective lossless data compression solution by using Intel? Integrated Performance Primitives Data Compression domain API. Additionally, this sample shows the ways of parallelizing a user application using Intel? Virtual Machine to fully benefit from modern Intel? microprocessor architectures.

    The sample uses dictionary based IPP functions implementing Lempel-Ziv (LZ77) algorithm and original GZIP data formats according to RFC 1950, 1951 and 1952 specifications.

    The resulting compressed data formats are fully compatible with the original GZIP formats, so the utilities are interchangeable. You can compress data with IPP_GZIP and decompress it with GZIP, or, vice versa, compress with GZIP and decompress using GZIP. However, to benefit from using multiprocessor/multicore architectures, use IPP_GZIP.
    I managed to compile a gzip binary using IPP, I wanted to know if IPP's deflate was really fast

    Download

    Intel SSE2 required

    Compression of enwik8 -9:

    1 core:
    gzip 8.8
    pigz 9.3 Homepage binary
    ipp_gzip 7.9

    2 cores:
    pigz 4.8
    ipp_gzip 4.1

    Would be cool to have results from a i7 with 8 threads

    ipp_gzip is the fastest. But since gzip and pigz are compiled with gcc (AFAIK) I'm not sure the speedup actually comes from IPP library itself or because of the compiler. I would have to compile gzip and pigz with the same compiler and settings than ipp_gzip for a fair comparison.

    Anyway ipp_gzip does not show a spectacular speedup compared to gzip (beside multicore support)
    Last edited by M4ST3R; 8th October 2009 at 23:58.

  2. #2
    Member
    Join Date
    Jul 2008
    Location
    Netherlands
    Posts
    16
    Thanks
    2
    Thanked 1 Time in 1 Post
    Intel I7 860, 2.8Ghz

    Compress 1.8s
    Decompress 140ms


  3. #3
    Member
    Join Date
    May 2008
    Location
    Germany
    Posts
    410
    Thanks
    37
    Thanked 60 Times in 37 Posts
    1 core - ipp_gzip - 7.9s
    2 core - ipp_gzip - 4.1s
    4 core - ipp_gzip - 1.8s - impressive speedup ..

    @edcassa

    can you please make a test for pbzip2 with your Intel I7 860 too ?

    a) pbzip2 - win32-binary from leszer.net

    http://www.leszer.net/scripts/downlo...zip2_win32.zip

    b) pbzip2 - win32-binary from M4ST3R

    http://www.fileden.com/files/2006/8/...0.5_M4ST3R.rar

    thanks in advance

    best regards

  4. #4
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,507
    Thanks
    742
    Thanked 665 Times in 359 Posts
    we have tried it and found that all speed improvement comes from decreased compression

  5. #5
    Member
    Join Date
    May 2008
    Location
    Germany
    Posts
    410
    Thanks
    37
    Thanked 60 Times in 37 Posts
    @Bulat Ziganshin:

    thank you very much for your answer

    "we have tried it and found that all speed improvement comes from decreased compression"

    but can you please write any explaination ???

    bzip2 and pbzip2 creates identical outputfiles ...

    if one program has faster compression and another program has faster decompression it should be possible to use in each case the best program

    best regards

  6. #6
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,507
    Thanks
    742
    Thanked 665 Times in 359 Posts
    i tell about intel gzip

Similar Threads

  1. Gzip 1.2.4 hack (OpenWatcom compiles)
    By Rugxulo in forum Data Compression
    Replies: 9
    Last Post: 22nd May 2009, 01:17
  2. Compiler related: Intel's code slower on AMD-CPUs?!
    By Vacon in forum Data Compression
    Replies: 5
    Last Post: 10th May 2008, 18:56
  3. gzip-1.2.4-hack - a hacked version of gzip
    By encode in forum Forum Archive
    Replies: 63
    Last Post: 10th September 2007, 05:16
  4. Replies: 0
    Last Post: 26th July 2007, 19:47
  5. Parallel implmentation of gzip: pigz
    By nimdamsk in forum Forum Archive
    Replies: 2
    Last Post: 13th March 2007, 21:44

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •