Results 1 to 30 of 138

Thread: BIT Archiver

Hybrid View

Previous Post Previous Post   Next Post Next Post
  1. #1
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts

    BIT Archiver

    BIT Archiver has started for compressing game binaries. In BIT 0.1, ROLZ based schema was used which is derived from QUAD. In past, I want to make only one compressing method in BIT. But today, BIT Archiver becomes a universal archiver step by step. After understanding that there is no universal compressor/model for all kind of data, I've decided to add couple of codecs besides couple of preprocessing for specific file types in future versions.

    In this prerelease, only LWCX mode is working. There is no preprocessing currently. Here is the LWCX details:

    - Order 0-4, 6 context mixer based on Neural Networks
    - 2D dimensional SSE with 32 vertex
    - Semi-Stationary bit modelling (SSE functions have different modelling)
    - Nibble based hashed context

    As you see, this is kind of reinventing the wheel. By doing this, I have learned lots of thing about context mixing (of course there are still lots of thing, too). I want to add only a few well-tuned submodels to it such as match model and sparse model.

    I hope you like it and I hope it hasn't any fatal error

    Download: http://www.osmanturan.com/bit02.zip

  2. #2
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts

    Thumbs up

    Thanks Osman!

  3. #3
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quick (-mem 9) test...

    A10.jpg > 835,942
    AcroRd32.exe > 1,461,060
    english.dic > 580,848
    FlashMX.pdf > 3,709,615
    FP.LOG > 574,636
    MSO97.DLL > 1,822,291
    ohs.doc > 853,469
    rafale.bmp > 762,162
    vcfiu.hlp > 643,792
    world95.txt > 521,153

    Total = 11,764,968 bytes

  4. #4
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Another quick (-mem 9) test...

    Test machine: AMD Sempron 2400+, Windows XP SP2

    Test file: ENWIK8 (100,000,000 bytes)


    Compressed Size: 21,958,938 bytes

    Elapsed Time: 633.50 Seconds

    00 Days 00 Hours 10 Minutes 33.50 Seconds

  5. #5
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Thanks a lot!
    I wonder the result in MOC and Black Fox's benchmark, too. As you see in the SFC test, LWCX suffers from executables. I think, E8E9 transform definetely helps.

    @testers:
    Could you post the timing with your computer specifications besides the compressed result? Mainly, I do some speed optimizations. I want to know it's performance on Celeron, Pentium 4, AMD etc. On my laptop (Core2 Duo 2.2Ghz, 2 GB RAM), it works at 350-800 KB/sec. I hope, it can compress around 500KB/sec-1 MB/sec in the future.

  6. #6
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    On my laptop (Core2 Duo 2.2 GHz, 2 GB RAM). All tests have been done with -9 option.
    --------------------------------------------------------------------------
    Valley.cmb (19,776,230 bytes)
    WinRK 3.0 build 3 Beta (PWCM) -> 5.961.850 bytes (1315 seconds)
    7-Zip 4.57 (Ultra) -> 7.508.392 bytes (~9 seconds)
    WinRAR 3.71 (Best) -> 8,511,238 bytes (~8 seconds)
    CMM4 0.1f (76) -> 8.197.710 bytes @ 607 KB/sec (33.013 seconds)
    BIT 0.2 -> 9.219.167 bytes @ 470 KB/sec (41.476 seconds)
    --------------------------------------------------------------------------
    Brosur1.tif (31,269,732 bytes) A brochure. 20x28 cm 300 dpi CMYK, 2362x3307 pixels, uncompressed TIFF

    CMM4 0.1f (76) -> 3,380,120 bytes @ 996 KB/sec (31.832 seconds)
    BIT 0.2 -> 3,562,068 bytes @ 714 KB/sec (43.089 seconds)
    7-Zip (Ultra) -> 3,948,960 bytes (~6 seconds)
    WinRAR 3.71 (Best) -> 4,478,996 bytes (~9 seconds)
    --------------------------------------------------------------------------
    Design2.tif (49,413,716 bytes): A newspaper advertisement, 25x35 cm 300 dpi CMYK, 2953x4134 pixels, uncompressed TIFF

    WinRAR 3.71 (Best) -> 10,072,714 bytes (~18 seconds)
    CMM4 0.1f (76) -> 11,148,805 bytes @ 755 KB/sec (65.00
    BIT 0.2 -> 11,456,267 bytes @ 634 KB/sec (76.469 seconds)
    7-Zip 4.57 (Ultra) -> 13,023,126 bytes (~15 seconds)
    --------------------------------------------------------------------------
    Calgary Corpus (TAR Version): 3,152,896 bytes
    CMM4 0.1f (76) -> 695,060 bytes (5.421 seconds)
    BIT 0.2 -> 765,273 bytes (6.604 seconds)
    --------------------------------------------------------------------------

    @toffer:
    How did you speed-up your compressor? What about the match model contributions?

  7. #7
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Another test with "-mem 9" on Encode Corpus:

    + Doom3.exe (5,427,200 bytes) -> 1,694,067 bytes @ 350 KB/sec (15.101 seconds)
    + Mech8.s3m (747,600 bytes) -> 338,707 bytes @ 262 KB/sec (2.776 seconds)
    + MPTRACK.exe (1,159,172 bytes) -> 499,970 bytes @ 331 KB/sec (3.432 seconds)
    + PariahInterface.utx (24,375,895 bytes) -> 5,996,489 bytes @ 579 KB/sec (41.076 seconds)
    + Photoshop.exe (19,533,824 bytes) -> 2,146,022 bytes @ 496 KB/sec (38.501 seconds)
    + Reaktor.exe (14,446,592 bytes) -> 2,146,022 bytes @ 619 KB/sec (22.823 seconds)
    + track5.wav (29,644,608 bytes) -> 23,061,944 bytes @ 415 KB/sec (69.748 seconds)
    + TracktorDJStudio3.exe (29,124,024 bytes) -> 5,359,983 bytes @ 605 KB/sec (46.987 seconds)

    Total: 45,822,243 bytes

Similar Threads

  1. Poor compression of bit-version of PPM
    By Stefan in forum Data Compression
    Replies: 20
    Last Post: 16th March 2010, 17:58
  2. Do you have a 64-bit machine at home?
    By encode in forum The Off-Topic Lounge
    Replies: 22
    Last Post: 4th December 2009, 14:09
  3. Bit guessing game
    By Shelwien in forum Data Compression
    Replies: 11
    Last Post: 24th November 2009, 02:22
  4. RINGS Fast Bit Compressor.
    By Nania Francesco in forum Forum Archive
    Replies: 115
    Last Post: 26th April 2008, 22:58
  5. Bit Archive Format
    By osmanturan in forum Forum Archive
    Replies: 39
    Last Post: 29th December 2007, 00:57

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •