Page 1 of 2 12 LastLast
Results 1 to 30 of 60

Thread: m7zRepacker

  1. #1
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts

    m7zRepacker

    Finally I finished my own 7-Zip brute forcer/optimizer. Any kind of feedback are welcome!

    -------------------------------------------------------------------------------
    m7zRepacker (for 7-Zip V9.20) // V1.0.32.301 // (C) 2010-2011 Mario Scholz
    -------------------------------------------------------------------------------

    Quick Description:
    - m7zRepacker is an command line brute forcer/optimizer for 7-Zip (V9.20), like Ultra7z.
    - m7zRepacker can brute force nearly all archives, that 7-Zip could decompress.


    Main differences between m7zRepacker and Ultra7z:
    - m7zRepacker tries to get always the best compression, so there's no fast mode or something like that
    - m7zRepacker normally squeeze out more bits & bytes of the given archive, but is sloooooooow as hell!
    - m7zRepacker keeps source archive untouched (creates a renamed new archive)


    There are 3 brute force modes:
    - m1 = Brute force executables as group, brute force files by extension (default mode)
    - m2 = Brute force executables as group, brute force files as one group
    - m3 = Brute force executables as group, brute force all files separately (not recommended)

    Note: m3 is only useful in very rare cases!!! e.g. same file extension, but different content
    This could be useful on some games, or for some uncompressed sub-archives in the main archive


    Features:
    - There are 3 different brute force modes
    - There are 57 different compression presets for brute force (14 of them for executables)
    - Auto-switching (brute force) between LZMA/LZMA2/BCJ2/PPMd/Delta/Deflate/Bzip2 and some Fastbyte values
    - Detecting Executables by header, not by extension (currently Windows & Linux executables only)
    - Detecting UPXed Executables and brute force them separately
    - Auto-switching between x86/x64/Itanium version of 7-Zip (V9.20) [7z.exe & 7z.dll]
    - Definable paths to 7-Zip versions (m7zRepacker.ini)
    - If you have enough memory and cpu power, you can run multiple instances (checks also for existing temp folders)
    - To control memory usage, all compression presets are limited to 2 threads
    - Supports Errorlevel/Exitcodes ("0 = no error", "1 = an error occurred")
    - Shows progress status on window title
    - Full Unicode support for paths and archive content (tested only on Windows 7, you have been warned!!!)


    Known issues & Notes:
    - It's so damn sloooooooooooooooooow! It's a perfect time waster!
    - Some Anti-Virus *maybe* detect a "bad behavior", because m7zRepacker runs 7-Zip very often in serial, on very fast pc's up to 25+ times a second, especially in -m3 brute force mode!
    - I never get any problem with repacked archives, but check the repacked archive twice, if you have important data in it!!!
    - Itanium support *maybe* get rid in further versions


    ToDo:
    - Better support for files without extensions. Currently they are not brute forced (except executables) but compressed with LZMA (-mx9)


    Download:
    m7zRepacker V1.0.32.301 (includes 7zip)

    Changelog:
    Code:
    ----- V1.0.32.301 --------------------------------------------------------------
    [FIX] Compression Preset 46+47 allocate more RAM than it should on CPUs with
          more than 2 cores. Now it's limited to 2 threads like all other Presets.
    
    ----- V1.0.32.300 --------------------------------------------------------------
    [FIX] Some Unicode fixes in -m3 brute force mode. (7-Zip sometimes stops with a 
          "missing file" error message.)
    [ADD] Filename of the repacked archive now contains used parameters, not just
          the brute force mode. Useful for batch brute forcing.
          e.g. "File-(Repacked-M1).7z"  ->  "File [Repacked_m1_d128_mem128].7z"
    [ADD] Results for repacked archive can be saved in a file log file. This can
          be enabled or disabled in the "m7zRepacker.ini".
    
    ----- V1.0.32.282 --------------------------------------------------------------
    [ADD] Windows-Cache-Management seems to have a problem, if 7-Zip runs very
          often in a short amount of time and with many small files, especially in
          m3 mode. The result is a very high cpu load and high memory usage. To
          avoid this Windows behavior, m7zRepacker pauses some seconds after every
          250's run of 7-Zip, so the Windows-Cache-Management is able to free memory
          and cpu load. By default m7zRepacker waits 5 seconds. The time to wait is
          definable in the m7zRepacker.ini file.
    [ADD] User Request: The lowest [mem] value is now "-mem24" instead of "-mem48"
    [ADD] User Request: New switch to extract encrypted archives ["password"]
    [MOD] The paths keys in the m7zRepacker.ini changed their name, so you have to
          check the keys/values, if you changed it to your needs.
    
    ----- V1.0.32.238 --------------------------------------------------------------
    [FIX] forget to activate the BCJ2 presets (for non executables in -m2 and -m3
          brute force modes) in the last release. (BCJ2+LZMA2 works *sometimes*
          better than DELTA+LZMA2 in some situations e.g. 8bit wav files)
    
    ----- V1.0.32.236 --------------------------------------------------------------
    [ADD] Public Release
    Last edited by Biozynotiker; 6th February 2011 at 12:24. Reason: new version

  2. Thanks (5):

    Bekk (1st April 2018),lorents17 (3rd November 2015),msat59 (26th June 2015),necros (9th May 2017),surfersat (14th October 2014)

  3. #2
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    Some quick results:

    FreeArc-0.666-linux-i386.tar (13.905.920 Bytes uncompressed):
    Code:
    7Zip -mx -md128m                  |   3.049.385 Bytes
    FreeArc 0.666 -mx -md128          |   2.856.804 Bytes
    Ultra7z (-extreme)                |   2.816.108 Bytes
    Ultra7z (-extreme -d128 -fb273)   |   2.811.727 Bytes
    m7zRepacker -m2 -d128 -mem512     |   2.800.495 Bytes
    All sound files from the game "Max Payne 1" as wav files (690mb uncompressed):
    Code:
    7Zip -mx -md128m                    |  412.040.122 Bytes
    Ultra7z (-extreme -d128 -mem1536m)  |  240.101.059 Bytes
    Rar -m5                             |  232.197.830 Bytes
    FreeArc 0.666 -mx -md128            |  227,068,911 Bytes (Time: 74sec)
    m7zRepacker -m3 -d128 -mem1536      |  221.032.648 Bytes (Time: 5h30min)
    (All results from Intel Core 2 Duo, E6850 @ 3.00 Ghz, 8GB Ram, Windows 7 64Bit)
    Last edited by Biozynotiker; 18th January 2011 at 19:53. Reason: Added FreeArc results

  4. #3
    Member
    Join Date
    Jun 2009
    Location
    Kraków, Poland
    Posts
    1,475
    Thanks
    26
    Thanked 121 Times in 95 Posts
    7zip has delta encoding???

  5. #4
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    9.x includes delta filter

  6. #5
    Member PAQer's Avatar
    Join Date
    Jan 2010
    Location
    Russia
    Posts
    22
    Thanks
    3
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Biozynotiker
    All sound files from the game "Max Payne 1" as wav files (690mb uncompressed):
    Something wrong here, original Max Payne (same for the sequel) contain ADPCM inside WAV container, only few WAVs are PCM.

  7. #6
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    Quote Originally Posted by PAQer View Post
    Something wrong here, original Max Payne (same for the sequel) contain ADPCM inside WAV container, only few WAVs are PCM.
    The original files are ADPCM thats right, but not the one i used. To run the game without errors on Win Vista, all sound files converted back to pcm/wav files with a tool (written by the max payne developers) that was provided in the max payne forum some years ago. The results should only show, that the delta presets works better than the one in Ultra7z 0.12.

    Edit:
    I think i have found the old forum thread.
    Last edited by Biozynotiker; 16th January 2011 at 18:44.

  8. #7
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    New version available, see first post for download an changelog...

  9. #8
    Member zody's Avatar
    Join Date
    Aug 2009
    Location
    Germany
    Posts
    90
    Thanks
    0
    Thanked 1 Time in 1 Post
    You can't enter password for file to repack...would be a nice to have feature

  10. #9
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    Quote Originally Posted by zody View Post
    You can't enter password for file to repack...would be a nice to have feature
    Could be useful, maybe in the next version. Thx for the idea.

  11. #10
    Member zody's Avatar
    Join Date
    Aug 2009
    Location
    Germany
    Posts
    90
    Thanks
    0
    Thanked 1 Time in 1 Post
    Choosing TEMP file location could be useful as well; e.g. putting them on a ram drive could give more speed - especially for small files

  12. #11
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    Quote Originally Posted by zody View Post
    Choosing TEMP file location could be useful as well; e.g. putting them on a ram drive could give more speed - especially for small files
    I'm currently bit busy to work on m7zRepacker, but i think a user specified temp folder location is implemented in one of the next versions.

    I found some spare time today and added FreeArc results (and time) to the second post.
    What should i say, m7zRepacker beats FreeArc in THIS case, BUT FreeArc is so damn fast, its simple amazing.
    Last edited by Biozynotiker; 18th January 2011 at 20:05. Reason: typos, typos, typos

  13. #12
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    New version available, download link updated on the first post...

    Changelog:
    Code:
    V1.0.32.282
    - [ADD] Windows-Cache-Management seems to have a problem, if 7-Zip runs very
            often in a short amount of time and with many small files, especially in
            m3 mode. The result is a very high cpu load and high memory usage. To
            avoid this Windows behavior, m7zRepacker pauses some seconds after every
            250's run of 7-Zip, so the Windows-Cache-Management is able to free memory
            and cpu load. By default m7zRepacker waits 5 seconds. The time to wait is
            definable in the m7zRepacker.ini file.
    - [ADD] User Request: The lowest [mem] value is now "-mem24" instead of "-mem48"
    - [ADD] User Request: New switch to extract encrypted archives ["password"]
    - [MOD] The paths keys in the m7zRepacker.ini changed their name, so you have to
            check the keys/values, if you changed it to your needs.

  14. Thanks:

    necros (9th May 2017)

  15. #13
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    New version available, download link updated on the first post...

    Changelog:

    Code:
    V1.0.32.300
    - [FIX] Some unicode fixes in -m3 bruteforce mode. (7-Zip sometimes stops with a 
            "missing file" error message.)
    - [ADD] Filename of the repacked archive now contains used parameters, not just
            the bruteforce mode. Useful for batch brute forceing.
            e.g. "File-(Repacked-M1).7z"  ->  "File [Repacked_m1_d128_mem128].7z"
    - [ADD] Results for repacked archive can be saved in a file log file. This can
            be enabled or disabled in the "m7zRepacker.ini".

  16. Thanks (2):

    Bekk (1st April 2018),necros (9th May 2017)

  17. #14
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,


    would you mind distributing your stuff in 7z-format?
    I mean... you are distributing a 7z-repacker!

    Best regards!

  18. #15
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    Hehe, i know thats a bit odd, but i use rar for all and everything for my daily jobs, because rar is the only archiver with a good container format, that 'total commander' supports by default.

  19. #16
    Tester
    Black_Fox's Avatar
    Join Date
    May 2008
    Location
    [CZE] Czechia
    Posts
    471
    Thanks
    26
    Thanked 9 Times in 8 Posts
    Hm, true, it only opens ZIP and RAR (and some others, but that's not relevant now). You can use plugin from http://www.totalcmd.net/plugring/7zip_new.html, it would be a shame if you used RAR even if you actually don't and do just because of this missing little feature AFAIK 7z lacks only in GUI, not in format or compression quality.
    I am... Black_Fox... my discontinued benchmark
    "No one involved in computers would ever say that a certain amount of memory is enough for all time? I keep bumping into that silly quotation attributed to me that says 640K of memory is enough. There's never a citation; the quotation just floats like a rumor, repeated again and again." -- Bill Gates

  20. #17
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    There is no bug free or 100% working 7zip plugin for total commander, i tried them all. Thats a must for daily use with important data (e.g. @work).

  21. #18
    Tester
    Black_Fox's Avatar
    Join Date
    May 2008
    Location
    [CZE] Czechia
    Posts
    471
    Thanks
    26
    Thanked 9 Times in 8 Posts
    Ah, I see... Then let's hope Christian Ghisler updates TC with 7z support in the future.
    I am... Black_Fox... my discontinued benchmark
    "No one involved in computers would ever say that a certain amount of memory is enough for all time? I keep bumping into that silly quotation attributed to me that says 640K of memory is enough. There's never a citation; the quotation just floats like a rumor, repeated again and again." -- Bill Gates

  22. #19
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,

    maybe it's not the most elegant solution, but with Double Commander (a free TC-clone) and multiarc.ini with this settings most things work for me (browsing, adding, extracting).

    Best regards! (and sorry for getting off-topic for this short period!)

  23. #20
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts

    Exclamation

    New version available, download link updated on the first post...

    Changelog:
    Code:
    V1.0.32.301
    [FIX] Compression Preset 46+47 allocate more RAM than it should on CPUs with
          more than 2 cores. Now it's limited to 2 threads like all other Presets.
    I decided to release this as a Hotfix before the next planned release, because this could be very frustrating, when you brute force very big archives and then maybe it stops after some days with an memory allocation error.

    I bought a new Sandy Bridge system (Core-i7-2600k/16GB RAM) some days ago. It's the first time i could test 1GB dictionary with many cores, so I ran into the same painful situation. Please update your older versions.

    BTW: There are over 1200 (real) downloads since the first release, thats way more then i expected! The download archive is now available in 7z format, as a wish from Vacon
    Last edited by Biozynotiker; 6th February 2011 at 16:48.

  24. Thanks (2):

    Bekk (1st April 2018),necros (9th May 2017)

  25. #21
    Member
    Join Date
    Jun 2009
    Location
    Puerto Rico
    Posts
    184
    Thanks
    80
    Thanked 16 Times in 12 Posts
    First of all, This is an amazing software!

    Second,
    This is a GUI I'm working with for this great utility:


  26. #22
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    @moisesmcardona
    Glad you find m7zRepacker useful. To avoid misunderstandings, i would rename "Select Memory Size" to something PPMd related, because this value is used by 7zip's PPMd and is also needed during decompression.

  27. #23
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    876
    Thanks
    474
    Thanked 175 Times in 85 Posts
    I have done my first tests with your marvelous tool m7zRepacker and must say it
    optimizes better than Ultra7z. So far, I have tested -m1 switch only which explains the
    bad results on FreeDB. I really like its overall performance. Thanks for all your efforts.

    Code:
    TESTSET        |       7z best |  Ultra7z 0.12  |  m7zRepacker  |  
    -----------------------------------------------------------------------------------------
    3D             |    77.486.562 |   75.755.385   |   75.083.064  |   99,11%
    Bitmap2        |   322.522.170 |  305.217.772   |  305.217.811  |  100,00%
    CDImage        |   289.418.950 |  289.085.574   |  286.988.667  |   99,27%
    Crossplatform  |    27.684.375 |   27.684.264   |   26.465.095  |   95,60%
    D.N.A.         |    17.660.724 |   17.660.724   |   17.653.454  |   99,96%
    Driver         |    50.577.998 |   50.577.998   |   49.918.291  |   98,70%
    FreeDB         |    99.614.855 |   99.614.855   |  124.129.310  |  124,61%
    Gutenberg      |    34.983.554 |   34.983.554   |   35.252.764  |  100,77%
    Installer      |   577.106.251 |  577.106.251   |  577.083.064  |  100,00%
    Mobile         |   477.058.532 |  482.563.823   |  480.703.738  |   99,61%
    Modules        |    39.329.934 |   38.453.225   |   36.780.897  |   95,65%
    Office2        |   240.775.814 |  237.579.896   |  237.119.426  |   99,81%
    Savegames      |    50.357.414 |   50.357.414   |   50.660.171  |  100,60%
    Sources        |    36.126.480 |   35.267.583   |   35.224.561  |   99,88%
    Waveforms2     |   499.239.087 |  498.737.257   |  498.737.302  |  100,00%
    Wikipedia      |   179.928.849 |  179.928.849   |  177.245.373  |   98,51%
    -----------------------------------------------------------------------------------------
                   | 3.019.871.549 |3.000.574.424   |3.014.262.988  |  100,46%

  28. #24
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    Thanks for testing! I know, this took some time to test it

    From my own tests, the -m2 and -m3 switch do often the magic, if Ultra7z/7Zip compress better or equal as m7zRepacker.
    I'm a bit disappointed from the waveforms2 results, are they 24/96bit audio files? Maybe i must optimize my delta offset scans.

  29. #25
    Tester

    Join Date
    May 2008
    Location
    St-Petersburg, Russia
    Posts
    182
    Thanks
    3
    Thanked 0 Times in 0 Posts
    Stephan Bush, did you use "-extreme3" preset for Ultra7z?
    Biozynotiker, have you any plans create same utility for FreeArc?
    Last edited by squxe; 15th February 2011 at 13:01.

  30. #26
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    876
    Thanks
    474
    Thanked 175 Times in 85 Posts
    The Ultra7z switches are those mentioned in the SqueezeChart; not the -extreme3 switch of which I didn't know it existed.

    The waveforms are 16 bit stereo waveforms without tags.

  31. #27
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    @squxe:
    I think that freearc autodetect file content by default, so i don't know how good an optimizer for freearc can be. 7zip currently lacks of autodetection, so ultra7z and m7zrepacker was born. The main problem is my spare time to test this, but never say never

    @Stephan Busch
    Thats a bit odd, because Ultra7z and m7zRepacker detect 16bit audio by delta filter, 7zip not by default. The size for all three competitiors was nearly the same. Did you use delta filter for your test?

  32. #28
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    876
    Thanks
    474
    Thanked 175 Times in 85 Posts
    yes, of course the normal 7-Zip result was achieved with the correct delta filter applied.

  33. #29
    Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    116
    Thanks
    18
    Thanked 32 Times in 11 Posts
    @Stefan Bush
    Ok, this explains much.

    Maybe there is a copy 'n' paste error in your result list. The value for 7zip/Mobile is 477.058.532. This is the exact value from "OCAMYD 1.66 RC" in SqueezeChart.
    Last edited by Biozynotiker; 16th February 2011 at 01:48.

  34. #30
    Member
    Join Date
    Jul 2014
    Location
    Mars
    Posts
    180
    Thanks
    124
    Thanked 11 Times in 10 Posts
    hi there, where to get gui for this utility? and how to know if new ver is out?

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •