Results 1 to 9 of 9

Thread: Pipe multiple files into SREP

  1. #1
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    873
    Thanks
    49
    Thanked 106 Times in 84 Posts

    Pipe multiple files into SREP

    Kinda noobish question here

    but i have several 3GB files that contains compressed data but that are 99% identical with the other files.

    so far putting these files inside a .7z file with storage compression, and then running SREP 1.5 on it gave nice results about the file size of one file) however decompression is a bit slow to the lot of I/O.
    first srep decompression then the 7.zip


    so i wondering if there was a way to pipe the date directly out of a "SREP archive" bu using other files congestion tools like tar .

    1: has to be done by command line for batch file operations
    2: needs to work under windows 7

  2. #2
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,505
    Thanks
    741
    Thanked 665 Times in 359 Posts
    SuperREP 1.91: lz77 preprocessor that occupies only 2-3% of filesize for compression
    September 2, 2010 homepage: http://freearc.org/research/SREP.aspx

    Usage: SREP [options] infile [outfile]
    infile/outfile may be denoted as "-" that means stdin/stdout
    Options are:
    -m1: check matches by SHA1 digest (compression memory = 6-7% of filesize)
    -m2: check matches by rereading old data (compression memory = 2-3% of filesize)
    -m3: byte-accurate matches and rereading (compression memory = 4-6% of filesize)
    -d: decompression (requires only 16 mb of memory besides of OS I/O buffers)
    -delete: delete source file after successful (de)compression
    -index=FILENAME: read/write index of compressed data into separate file
    -temp=FILENAME: keep uncompressed data in the file in stdin-to-stdout mode
    -l: minimum LZ match length, default 512
    -c: size of hash chunk, by default as small as required to find all LZ matches
    -s: print stats

  3. #3
    Member PAQer's Avatar
    Join Date
    Jan 2010
    Location
    Russia
    Posts
    22
    Thanks
    3
    Thanked 0 Times in 0 Posts
    Considering direct decompression: 7-zip supports writing to stdout, srep reading from stdin, you can directly decompress .7z or .lzma archive which can contain srep file to filestream like precomp's pcf.
    If you are talking about 7z > srep > tar_unpack without any tmp files, guess this is not possible, because srep will be used custom tempfile (as option).

  4. #4
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    873
    Thanks
    49
    Thanked 106 Times in 84 Posts
    nah just .srep file -> SREP decompression -> 7zip "decompresion" -> original files

    but ima look into the command line version of 7-zip for stdin/stdout usage

  5. #5
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    I use pipeing 7z & srep often, because its sometime is a good time saver. Maybe the following commandline is helpfully to you to use it further in your script...

    compress:
    srep.exe -m1 7zcontainer2compress.7z - | 7z.exe a -t7z -mx -m0=lzma2 -md8m 7zcontainer2compress.7z.srep -si

    decompression:
    7z.exe x 7zcontainer2decompress.7z.srep -so | srep.exe -m1 -d - 7zcontainer2decompress.7z

    But there is a small drawback of piping, you can't be sure that the exit code that was returned is correctly.

  6. #6
    Member
    Join Date
    Jun 2008
    Location
    Berlin
    Posts
    12
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Bulat Ziganshin View Post
    SuperREP 1.91: lz77 preprocessor that occupies only 2-3% of filesize for compression
    September 2, 2010 homepage: http://freearc.org/research/SREP.aspx
    Strange, I can only find find older SREP versions on the homepage.

  7. #7
    Member Surfer's Avatar
    Join Date
    Mar 2009
    Location
    oren
    Posts
    203
    Thanks
    18
    Thanked 7 Times in 1 Post
    Quote Originally Posted by Sotho Tal Ker View Post
    Strange, I can only find find older SREP versions on the homepage.
    http://freearc.org/download/research/srep191.zip

  8. #8
    Member
    Join Date
    Feb 2010
    Location
    Nordic
    Posts
    200
    Thanks
    41
    Thanked 36 Times in 12 Posts
    -deleted-
    Last edited by willvarfar; 18th January 2011 at 13:20.

  9. #9
    Member
    Join Date
    Oct 2018
    Location
    Russia
    Posts
    7
    Thanks
    2
    Thanked 1 Time in 1 Post
    Quote Originally Posted by Biozynotiker View Post
    I use pipeing 7z & srep often, because its sometime is a good time saver. Maybe the following commandline is helpfully to you to use it further in your script...

    compress:
    srep.exe -m1 7zcontainer2compress.7z - | 7z.exe a -t7z -mx -m0=lzma2 -md8m 7zcontainer2compress.7z.srep -si

    decompression:
    7z.exe x 7zcontainer2decompress.7z.srep -so | srep.exe -m1 -d - 7zcontainer2decompress.7z

    But there is a small drawback of piping, you can't be sure that the exit code that was returned is correctly.
    It doesn't work for me. I tried different options, but I get an error.
    Can you show a full script?
    Is it possible to add precomp in this line?

    1. 7z > precomp > srep > 7z
    2. 7z > srep > precomp > 7z


    //edit
    it was 3.2 version, 3.93a works
    Last edited by Peachill; 28th May 2019 at 23:51.

Similar Threads

  1. convert swf files to avi files
    By Jabilo in forum The Off-Topic Lounge
    Replies: 13
    Last Post: 26th October 2016, 12:39
  2. Precomp and srep
    By vlv in forum Data Compression
    Replies: 2
    Last Post: 25th December 2010, 12:16
  3. Srep with multiple files support ?
    By SvenBent in forum Data Compression
    Replies: 3
    Last Post: 30th September 2010, 20:41
  4. -Delete back in srep
    By SvenBent in forum Data Compression
    Replies: 0
    Last Post: 13th December 2009, 21:36
  5. Replies: 13
    Last Post: 6th August 2006, 19:10

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •