Results 1 to 11 of 11

Thread: which tools support parallel compress

  1. #1
    Member
    Join Date
    May 2008
    Location
    CN
    Posts
    45
    Thanks
    0
    Thanked 0 Times in 0 Posts

    which tools support parallel compress

    rar
    7zip
    gzip?

  2. #2
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,507
    Thanks
    742
    Thanked 665 Times in 359 Posts
    rar&7zip

  3. #3
    Member
    Join Date
    May 2008
    Location
    CN
    Posts
    45
    Thanks
    0
    Thanked 0 Times in 0 Posts
    3Q,
    why gzip cant?

  4. #4
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,507
    Thanks
    742
    Thanked 665 Times in 359 Posts
    pigz = parallel gzip
    pbzip2 = parallel bzip2

  5. #5
    Member
    Join Date
    May 2008
    Location
    CN
    Posts
    45
    Thanks
    0
    Thanked 0 Times in 0 Posts
    got pigz = parallel gzip
    pbzip2 = parallel bzip2
    here
    http://www.leszer.net/index.php/my-software.html

  6. #6
    Member
    Join Date
    May 2008
    Location
    CN
    Posts
    45
    Thanks
    0
    Thanked 0 Times in 0 Posts
    pigz is very fast
    gzip 390M only 3-5seconds

  7. #7
    Member
    Join Date
    May 2008
    Location
    Germany
    Posts
    410
    Thanks
    37
    Thanked 60 Times in 37 Posts
    i think, it is worth to try the new bsc version 2.3.0

    see: http://encode.su/threads/586-bsc-new...ing-compressor

    downloadable from: http://libbsc.com/downloads.aspx

    if you want fast compression, then try the -m2 option ("ST5"): "bsc e inputfile outputfile -m2"

    in my tests i am using:

    "bsc e inputfile outputfile -m2 -cp -b128"
    or "bsc e inputfile outputfile -m2 -cp -b190"
    or "bsc e inputfile outputfile -m2 -cp -b232"


    if you want to try to use more cores
    it seems worth to compare 7-Zip 9.15 beta:

    7z.exe a -mx3 -m0=bzip2 g:\archivfile.7z f:\FileBackups\dir1

    and

    7z.exe a -mx3 -m0=lzma2 g:\archivfile2.7z f:\FileBackups\dir1

    best regards

  8. #8
    Member
    Join Date
    May 2008
    Location
    CN
    Posts
    45
    Thanks
    0
    Thanked 0 Times in 0 Posts
    can you tell me why ppmd cant use multi core

  9. #9
    Member
    Join Date
    May 2008
    Location
    CN
    Posts
    45
    Thanks
    0
    Thanked 0 Times in 0 Posts
    bsc is not normal

    D:\renkoudmp\1306>time 0<cr
    当前时间: 22:06:00.68
    输入新时间:

    D:\renkoudmp\1306>bsc e d:\renkoudmp\1306\hu1306.dmp d:\renkoudmp\1306\test.bsc -m2 -cp -b128
    This is bsc, Block Sorting Compressor. Version 2.3.0. 9 August 2010.
    Copyright (c) 2009-2010 Ilya Grebnov <Ilya.Grebnov@libbsc.com>.

    d:\renkoudmp\1306\hu1306.dmp compressed 393103360 into 16795257 in 12.371 seconds.

    D:\renkoudmp\1306>time 0<cr
    当前时间: 22:06:13.10
    输入新时间:

  10. #10
    Member
    Join Date
    May 2008
    Location
    CN
    Posts
    45
    Thanks
    0
    Thanked 0 Times in 0 Posts
    it' strange use -f =>fast and better ratio

    D:\renkoudmp\1306>time 0<cr
    当前时间: 22:08:12.89
    输入新时间:

    D:\renkoudmp\1306>bsc e d:\renkoudmp\1306\hu1306.dmp d:\renkoudmp\1306\test.bsc -m2 -cp -b190
    This is bsc, Block Sorting Compressor. Version 2.3.0. 9 August 2010.
    Copyright (c) 2009-2010 Ilya Grebnov <Ilya.Grebnov@libbsc.com>.

    d:\renkoudmp\1306\hu1306.dmp compressed 393103360 into 16967258 in 14.040 seconds.

    D:\renkoudmp\1306>time 0<cr
    当前时间: 22:08:26.96
    输入新时间:

    D:\renkoudmp\1306>bsc
    This is bsc, Block Sorting Compressor. Version 2.3.0. 9 August 2010.
    Copyright (c) 2009-2010 Ilya Grebnov <Ilya.Grebnov@libbsc.com>.

    Usage: bsc <e|d> inputfile outputfile <switches>

    Switches:
    -b<size> Block size in megabytes, default: -b25
    minimum: -b1, maximum: -b1024
    -m<algo> Block sorting algorithm, default: -m3
    -m0 Sort Transform of order 3
    -m1 Sort Transform of order 4
    -m2 Sort Transform of order 5
    -m3 Burrows Wheeler Transform
    -c<ctx> Contexts for sorting, default: -cf
    -cf Following contexts
    -cp Preceding contexts
    -ca Autodetect (experimental)
    -H<size> LZP hash table size in bits, default: -H16
    minimum: -H10, maximum: -H28
    -M<size> LZP minimum match length, default: -M128
    minimum: -M4, maximum: -M255
    -f Enable fast compression mode, default: disable
    -l Enable LZP, default: enable
    -r Enable Reordering, default: disable
    -s Enable Segmentation, default: disable
    -p Disable all preprocessing techniques
    -t Disable parallel blocks processing, default: enable
    -T Disable multi-core systems support, default: enable

    Switches may be combined into one, like -b128p

    D:\renkoudmp\1306>tz

    D:\renkoudmp\1306>time 0<cr
    当前时间: 22:10:04.43
    输入新时间:

    D:\renkoudmp\1306>bsc e d:\renkoudmp\1306\hu1306.dmp d:\renkoudmp\1306\test.bsc -m2 -cp -b190 -f
    This is bsc, Block Sorting Compressor. Version 2.3.0. 9 August 2010.
    Copyright (c) 2009-2010 Ilya Grebnov <Ilya.Grebnov@libbsc.com>.

    d:\renkoudmp\1306\hu1306.dmp compressed 393103360 into 15577173 in 12.043 seconds.

    D:\renkoudmp\1306>time 0<cr
    当前时间: 22:10:16.51
    输入新时间:

  11. #11
    Member Karhunen's Avatar
    Join Date
    Dec 2011
    Location
    USA
    Posts
    91
    Thanks
    2
    Thanked 1 Time in 1 Post
    The binaries I know about available for Winboxes are lzop ( lzo parallel ), pigz ( parallel gzip ), ZPAQ of course, and plrzip ( parallel lzip ). IMO, I find LZO to be best for raw disk images ( poor compression, but fast speed ). Suggest you peruse the MOC website which should be http://heartofcomp.altervista.org/MOC/MOC.htm

Similar Threads

  1. Comparison of lossless PNG compression tools
    By Surfer in forum Data Compression
    Replies: 54
    Last Post: 19th September 2011, 23:58
  2. Compress-LZF
    By spark in forum Data Compression
    Replies: 2
    Last Post: 16th October 2009, 01:08
  3. parallel compression with batches
    By evg in forum Data Compression
    Replies: 4
    Last Post: 17th September 2009, 19:14
  4. Interesting tools
    By lunaris in forum Data Compression
    Replies: 2
    Last Post: 26th August 2009, 00:50
  5. LZTURBO 0.1 parallel compressor
    By donotdisturb in forum Forum Archive
    Replies: 5
    Last Post: 7th October 2007, 23:44

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •