RAR 5.0!!!

[Bulat Ziganshin]

RAR 5.0 beta 1: http://rarlab.com/rarnew.htm

x32: http://www.rarlab.com/rar/wrar50b1.exe

x64: http://www.rarlab.com/rar/winrar-x64-50b1.exe

one user said that language dlls from rar 4.x works in the new version

[Bulat Ziganshin]

quick test:

D:\Testing\MsOffice>timer rar5 a a -r -md1024m -s999999 -r -m5 -ma5|tail
Kernel Time = 3.338 = 00:00:03.338 = 7%
User Time = 209.774 = 00:03:29.774 = 448%
Process Time = 213.112 = 00:03:33.112 = 455%
Global Time = 46.754 = 00:00:46.754 = 100%


Archive size: 323 733 189 bytes


D:\Testing\MsOffice>timer rar5 t a|tail
Kernel Time = 0.390 = 00:00:00.390 = 12%
User Time = 4.742 = 00:00:04.742 = 155%
Process Time = 5.132 = 00:00:05.132 = 167%
Global Time = 3.058 = 00:00:03.058 = 100%




D:\Testing\MsOffice>timer arc a a -mx -r -t
Compressed 5,798 files, 810,411,321 => 309,363,041 bytes. Ratio 38.1%
Compression time: cpu 244.30 secs, real 148.39 secs. Speed 5,461 kB/s
Testing time: cpu 17.89 secs, real 18.37 secs. Speed 44,113 kB/s


D:\Testing\MsOffice>timer arc a a -mex5 -t -r
Compressed 5,798 files, 810,411,321 => 322,675,398 bytes. Ratio 39.8%
Compression time: cpu 255.75 secs, real 37.73 secs. Speed 21,477 kB/s
Testing time: cpu 20.39 secs, real 4.54 secs. Speed 178,455 kB/s

[Intrinsic]

Awesome! finally RAR being brought upto date wrt dictionary size etc.

[encode]

Uh, I see the new checksum for my CHK (WinRAR v5 supports BLAKE2)

[Bulat Ziganshin]

blake2sp, to be concrete. btw, its SSE version has speed of 4-5 cpb, and can be calculated in 8 threads so i hope that your program will run at least 5 GB/s (unlike rar beta that uses slow C++ implementation)

[Skymmer]

Oh gosh O_O

[SvenBent]

jsut a quick dirty test bfore bedtimes

1590 files,90 folders (update for FFXI)
7-zip lzma2 ultra/256mb/273/solid 70,0 MB (73.498.201 bytes)
WinRar 5.0 Best/256mb/Solid 68,5 MB (71.863.575 bytes)

i should have compared on memory usage udring compression rather than dictionary size but meh

[Bulat Ziganshin]

rar5 beta2 is out, beta1 sometimes created bad archives

[Stephan Busch]

In my tests WinRar 5.0 cannot compress as well as WinRar 4.0 (with auto-selection of best method).
Even with 1 GB dictionary WinRar 5.0 can't come close to LZMA performance. Considering that they will keep this algorithm for the upcoming decade,
they could have done better by including some innovations. I agree that most media files are already compressed, and I also agree that PPMd can be
removed, but why they didn't include a strong and fast BWT (such als LIBBSC) or a dedicated encoder for lossless audio and rgb?

[Bulat Ziganshin]

he removed everything rarely used. text compression is of very small interest for most users, and wav/bmp files too

[Stephan Busch]

yeah.. but its sad news for me that a newer version cannot compress as small as a previous one and that the algorithm of choice here does not seem strong enough.

[Bulat Ziganshin]

he left this area for 7z/nz/fa/... crowd

[Matt Mahoney]

Update says text mode was removed but it looks like option -mc7:128t+ still works and compresses exactly the same as v4.20. I tried -mc7:256t+ on enwik8 but it had no effect on compressed size or memory shown by windows task manager. http://mattmahoney.net/dc/text.html#1984

Also, I tested with -s -m5 on the mingw data set. There is a very slight improvement in compression with no speed change between 4.20 and 5.00b2. http://mattmahoney.net/dc/zpaq.html

[Intrinsic]

I wonder if it's still crazily effective on that old nes/gameboy rom that compressed to like 2% of it's original size, anyone have that around? i don't, or if i do it's buried

[Bulat Ziganshin]

Matt, probably you have compressed to 4.0 format. either use GUI or learn how new cmdline can be used. i've compressed with "rar5 a a -r -md1024m -s999999 -r -m5 -ma5" - note that -ma5 select new archive format, but overal i don't know how to master cmdline correctly

[Matt Mahoney]

Oops, forgot to use -ma5 to select RAR 5 format. mingw benchmark:

Archiver          Create mingw44   Time  Add mingw45   Time Extract (CPU)
-------           --------------  -----  -----------   ---- ------- -----
rar        -s -m5     41,796,989   36.0  112,379,699   50.4   31.8    9.3
rar   -ma5 -s -m5     39,247,216   47.4   80,323,216   59.1   29.9    8.2

Edit: Updated RAR benchmarks with and without -ma5. http://mattmahoney.net/dc/zpaq.html

[SvenBent]

its sads new with the specific compression it basically made rar into a 7-zip competition instead of a sidearm for me

FFXI log - 1folder 1114 files 439.830.918byes (.log extension)
7-zip lzma2/64mb/solid 36,3 MB (38.161.292 bytes)
Rar5 bedst/64mb/solid 44,1 MB (46.313.380 bytes)
rar bedst/4mb/solid 33,8 MB (35.518.952 bytes)


these log files are a bit tricky thought cause they are in very random sizes form 0 or 270bytes to 4 mbytes in size
they also they do have alot of exacontain a lot of "noise" as in misspelling,some japanese characters now and then, names that are only in this game etc etc.
also alot of the files contains the same log passage but with slight different settings so they are very close but slightly difference even over far distances

so they are a bit harder on real texture optimized compressors than a optimal text source

[Matt Mahoney]

A little more testing with enwik8: http://mattmahoney.net/dc/text.html#1984

In the new format (-ma5), the regular compression modes -m1 through -m5 compress a little better and slower and use more memory. The -mc7:128t+ option I used for best compression is silently ignored with -ma5 but works identical to rar 4.20 otherwise.

[zyzzle]

It was a huge mistake and step back to eliminate text / WAV / delta compression from RAR5! Why do this? Why can't those options be left in and checked as necessary? Love the big dictionary sizes, but why can't they extend those to the RAR4 format?

[zyzzle]

Oh, and hopefully someone will compile UNRAR.EXE that will work in MS-DOS / DJGPP for RAR v5.

[Bloax]

Why can't those options be left in and checked as necessary?

Because that would actually be progress!! You can't just have that kind of stuff happening in the .RAR-World-Domination State.

[Matt Mahoney]

Those options still work without -ma5, right?

3,377,110 ATrain.wav
2,757,094 BeautySlept.wav
2,126,216 chanchan.wav
3,305,550 death2.wav
3,529,234 experiencia.wav
3,384,860 female_speech.wav
3,528,826 FloorEssence.wav
3,531,050 ItCouldBeSweet.wav
3,498,470 Layla.wav
3,512,370 LifeShatters.wav
3,081,406 macabre.wav
3,167,160 male_speech.wav
3,209,000 SinceAlways.wav
3,407,918 thear1.wav
3,503,052 TomsDiner.wav
2,095,426 velvet.wav
51,014,742 bytes

37,691,355 rar500b2 -ma5
37,638,846 rar500b2 -ma5 -m5
35,828,833 rar500b2
35,816,426 rar500b2 -m5
35,838,005 rar420
35,825,578 rar420 -m5

[load]

Beta3 is out.

http://www.rarlab.com/rarnew.htm

[chornobyl]

http://www.rarlab.com/rar/wrar50b4.exe
http://www.rarlab.com/rar/winrar-x64-50b4.exe

Version 5.00 beta 4

1. If archiving operation cannot allocate memory required for compression
dictionary, it automatically reduces the dictionary size.

2. Decompression algorithm can store the dictionary in several memory
blocks. It helps to unpack an archive on systems with high level
of memory heap fragmentation, when no single memory block
is large enough to fit the entire compression dictionary.

[Bulat Ziganshin]

my work

[Sportman]

Worm (ab)use WinRAR to create copies:
http://blog.trendmicro.com/trendlabs...rchived-files/

[Matt Mahoney]

Interesting how the worm can update encrypted archives. One reason why closed source encryption software is a bad idea.

[Bulat Ziganshin]

Matt, decryption code available in unrar sources, can be easily used to recreate encryption one. it's just AES. and situation isn't worse than with opens-ource code, so i wonder what you mean?

[GOZARCK]

.................................................. .................................................. ......

[Matt Mahoney]

If the whole archive is encrypted with one password then it should not be possible to update it without knowing it.