MM compression library

**Bulat Ziganshin** · 20th May 2007, 16:49

http://www.haskell.org/bz/mm.zip

i glad to present first open-source multimedia compression library
which allows to add MM compresion features to any open-source compressor
or archiver. the library provided under GPL license and includes 3 modules:

mmdet - multimedia detection. this module allows to examine raw chunk
of data and determine number of channels and width of data samples.
returned result may be, for example, "skip 2 bytes at start of file and
encode rest of data as 6 channels of 24-bit values". floating-point values
and values with non-intel byte order are not recognized. you've also
provided with results of simple order-0 and lz+order0 compression which
allows you to decide whether it has meaning to use MM compression at all.
MMDET module relies on comparison of applying order-0 model to original
data and data after MM preprocessing (substraction of successive values).
additionally, recognition by file header is available, although at this
moment only .WAV file headers are supported

tta - this module based on TrueAudio lossless audio codec developed by
Alexander Djourik and Pavel Zhilin (http://www.true-audio.com/) version 2.0
mmdet module (both WAV header method and entropy tests) used to determine
structure of wave file. Like an original TTA codec, 8, 16 and 24 bits integer
and 32-bit floating-point samples are supported. Unfortunately, in some
rare cases 24-bit compression leads to program crash and 32-bit FP compression
decodes "00 00 80 3F" values as "00 00 00 00" so probably high-precision
samples support will be disabled in FreeArc release

mm - simple multimedia data preprocessor which doesn't reduce data size
but only substracts successive values. running grzip or ppmd on its output
allows to provide compression ratio comparable with that of rar, uharc and sbc.
it uses mmdet module to determine structure of data being compressed

All modules includes main() driver which allows you to compile them as
standalone apps. Use .bat files for this purpose, while provided makefile shows
how to compile the library as part of larger application. You should also know
that mmdet standalone program display its suggestion about selection of optimal
MM model based on simple critery - which model gives the smaller output, while
tta/mm modules uses mmdet library with smarter criteria (which is slightly
different in mm and tta) which allows to pretty reliably recognize
high-precision samples (mmdet in this case may suggest, for example, using
4*8bit model instead of 1*32). You can test mm/tta autodetection features even
with WAV files by using -s switch

I thank Christian Maeder and Uwe Herklotz for their help in development of
mmdet module

ps: can you please help me fix language mistakes in this readme text?

**Bulat Ziganshin** · 20th May 2007, 20:58

just fixed problem with storing non-MM files in MM preprocessor. please reload

also i've tested compression of mod/stm file with and without MM preprocessor:

Arc.exe a a -m4xb
Compressed 15 files, 2.189.379 => 1.208.577 bytes. Ratio 55.2%
Compression time 3.41 secs, speed 641 kb/s. Total 4.06 secs

Arc.exe a a -mmm:1*8+4xb
Compressed 15 files, 2.189.379 => 1.141.624 bytes. Ratio 52.1%
Compression time 3.90 secs, speed 562 kb/s. Total 4.39 secs

**Vacon** · 21st May 2007, 13:07

Hello everyone,

Great work!!!

But why GPL and not L_GPL?
Imagine Igor wants to use it for 7-zip!?

Best regards!

**Bulat Ziganshin** · 21st May 2007, 13:27

1) all my libs are gpl'ed, if Igor will ask about their use in 7zip, i will change license for him. otherwise i prefer to use gpl license
2) it's TTA license. so changing it for TTA module will also require agreement with TTA authors

**Black_Fox1** · 21st May 2007, 14:06

Tested mm on my testset and it successfully preprocessed BMP and PSD files:

File: filesize w/ WinRAR -> filesize w/ mm+winRAR (ratio increase)
PSD: 2 276 844 -> 2 032 855 (10,7%) (original filesize 6 755 163)
BMP: 1 002 299 -> 950 609 (5,1%) (original filesize 3 932 214)

File: filesize w/ 7-Zip -> filesize w/ mm+7-Zip (ratio increase)
PSD: 1 152 376 -> 1 139 150 (1,1%)
BMP: 903 276 -> 768 525 (14,9%)

**Vacon** · 21st May 2007, 15:09

Hello everyone,

thank you Bulat for your quick answer.
I will tell in 7-zip forum, since I don't know if Igor reads here.

Hmmm, why should I?

Bulat did already himself!
Sorry for making noise

Best regards!

**Bulat Ziganshin** · 21st May 2007, 15:49

Black_Fox
according to my tests, using grzip (arc a a -m=grzip:m1:l:a) after MM preprocessor gives even better compression results

**Black_Fox1** · 21st May 2007, 19:55

I'll try it, thanks for the tip, Bulat

I'm not so optimistic about integrating mm(det) into 7-zip, though... Igor wants to have every part of it made by his own - at least I don't know about any part that was taken from other author

**Bulat Ziganshin** · 21st May 2007, 20:10

i know, so i don't tell about this to Igor directly. but he knows that he can use my libs if he want. otherwise, i will do the same in freearc

**nimdamsk** · 21st May 2007, 20:36

Originally Posted by Black_Fox

at least I dont know about any part that was taken from other author

ppmd, bzip2?

**Bulat Ziganshin** · 21st May 2007, 20:53

unforunately, Igor says that he prefers to use parts developed by him. in particular, instead of using ppmd for text with auto detection of file type, he will prefer if possible to use some lzma-based solution

**Black_Fox1** · 21st May 2007, 21:05

Originally Posted by nimdamsk

ppmd, bzip2?

Igor has made his own implemetation surely. He belongs into those compression people group that is able to made own compatible implementation of ppmonstr (closed-source) during time

**Bulat Ziganshin** · 21st May 2007, 21:09

i also belong to such group, theroretically

but to produce something like lzma or ppmd one need several *years* of experiments

**Black_Fox1** · 21st May 2007, 21:42

I'm sure that years won't be enough for anyone to be able to produce ppmonstr compatible archives

In compression circles, it seems that from every 10 compression genies 7 are Russian.

**Bulat Ziganshin** · 21st May 2007, 21:59

i just want to emphasize that knowledge isn't enough. you should try many ideas and spend a lot of time developing state-of-the-art library. it is why i can write something good only in areas of narrow insterests such as dict preprocessing or fast lz engine where competition is pretty small or non-existant

**Vacon** · 21st May 2007, 22:46

Hello everyone,

Originally Posted by Bulat Ziganshin

...one need several *years* of experiments

maybe one should invent a time-compressor...?

Best regards!

**Bulat Ziganshin** · 21st May 2007, 23:19

i work on this and hope to present 0.1 alpha version in a few years

**John** · 22nd May 2007, 03:57

how really it works?
I tried and optmifrog compres more the data
i do: mm.exe -w16 nameinput name output and file size is 12byte more
i do: tta the same the file size it' more smaller, but not more than optimfrog

Any examples?

**encode** · 22nd May 2007, 09:33

Originally Posted by Bulat Ziganshin

i work on this and hope to present 0.1 alpha version in a few years

Within 20 years...

**Bulat Ziganshin** · 22nd May 2007, 10:36

John
1. mm is a *preprocessor* which means that you should compress its output with grzip, ppmd or some other algorithm (and compression ratio will be improved due to preprocessing)

2. tta anyway compress audio wave data much better than mm+grzip. but it's not the best losless audio codec available and i know that optimfrog is really better. i not wrote anything yourself, i just selected among 4-5 open-source codecs one with minimum size of sources and maximum compression and then i've added a bit of code required to use it inside freearc, 7z and so on. but by itself these programs isn't interesting for *end-users*. it's just a library for programmers and *examples* of using this library

**Bulat Ziganshin** · 22nd May 2007, 15:04

Black Fox fixed readme, thanks

**Bulat Ziganshin** · 11th September 2007, 12:31

New release:
v1.1 (2007-09-11)
TTA: fixed problems with processing 24/32-bit samples
TTA: improved compression by 1-2%
MM: fixed error with processing 'offset'

http://www.haskell.org/bz/mm11.zip

**Vacon** · 11th September 2007, 15:15

Hello everyone,

good news

Thank you Bulat!

Best regards!

**nimdamsk** · 11th September 2007, 19:09

Thanks! Waiting for new ARC with Tornado 3 + MM

**Bulat Ziganshin** · 11th September 2007, 19:24

the august version includes both

i don't put new MM version on the site only because there was no users who really need it

**LovePimple** · 11th September 2007, 20:25

Thanks Bulat!

**Vacon** · 11th September 2007, 22:18

Hello everyone,

Originally Posted by Bulat Ziganshin

the august version includes both...

Have I missed something?

Is there an official final?

Best regards!

**Bulat Ziganshin** · 12th September 2007, 00:34

i say about beta version you've seen

**Vacon** · 12th September 2007, 14:26

Hello everyone,

Originally Posted by Bulat Ziganshin

i say about beta version youve seen

Since it was a beta (and isnt officially reachable for some reasons anymore...) -> time for a stable release?

Best regards!

**Bulat Ziganshin** · 12th September 2007, 16:40

i don't yet finished it. there are errors that i need to fix, new features that i need to make really usable, new codecs that i need to finish

Thread: MM compression library

Thread Tools

Search Thread

Display

Similar Threads

QuickLZ ZIP - new zip/deflate library

Posting Permissions