Results 1 to 12 of 12

Thread: St compressor

  1. #1
    Member
    Join Date
    Oct 2014
    Location
    Bucharest
    Posts
    6
    Thanks
    0
    Thanked 2 Times in 1 Post

    St compressor

    Upon some consideration I decided to share my PPM model.
    Included is an estimator of compressed file size.
    Your observations and opinions are welcomed.
    In case you make use of the source code, I appreciate crediting it.

    Stefan

  2. Thanks (2):

    Gonzalo (9th October 2014),Nania Francesco (9th October 2014)

  3. #2
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    222
    Thanked 146 Times in 83 Posts
    The important thing is to start with, confront and always try to improve. Looks like a good code PPM but why not create a standard compressor files, if you have trouble you can see fpaq sources, or other open source software. Just remember that a compressor that uses float numbers for the probability (float or double) in theory will not be easily compatible on all processors.

  4. #3
    Member
    Join Date
    Oct 2014
    Location
    Bucharest
    Posts
    6
    Thanks
    0
    Thanked 2 Times in 1 Post
    Not sure what are standard compressor files. Currently the code only estimates file size after compression. Added some updates and bug fixes, for anyone interested you'll find an improved version attached.

  5. #4
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    Normally a program would take command line arguments maybe like this:

    st c input compressed_file
    st d compressed_file output

    I suggest removing #include "stdafx.h", using <> instead of "", and adding <string.h> to make it portable.

  6. #5
    Member
    Join Date
    Oct 2014
    Location
    Bucharest
    Posts
    6
    Thanks
    0
    Thanked 2 Times in 1 Post
    Attached, the latest version of st.

  7. #6
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    876
    Thanks
    474
    Thanked 175 Times in 85 Posts
    How would you compile that?

  8. #7
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    688
    Thanks
    41
    Thanked 173 Times in 88 Posts
    IntelC 11.1.072 -fp:precise compile
    Just to remind - program assumes that the file named enwik7 is nearby the exe file.
    Attached Files Attached Files

  9. Thanks:

    Stephan Busch (15th December 2014)

  10. #8
    Member
    Join Date
    Oct 2014
    Location
    Bucharest
    Posts
    6
    Thanks
    0
    Thanked 2 Times in 1 Post
    Pleased to share an improved version of st.
    Last edited by cloudreflection; 29th March 2015 at 14:57.

  11. #9
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    With enwik8 in the current directory it produces the output (ignoring any command line arguments):

    Estimated filesize 44823736
    77.95

    If you decide to make this into a general purpose compressor (for any file), I will benchmark it.

  12. #10
    Member
    Join Date
    Apr 2009
    Location
    here
    Posts
    204
    Thanks
    172
    Thanked 110 Times in 66 Posts
    i'm getting this (64bit binary):

    Estimated filesize 43984826
    96.77
    and this if compiled with -O3
    Estimated filesize 43984826
    47.26
    edit:
    as 32bit binary (again with -O3) i got the same size as matt.
    Estimated filesize 44823736
    39.37
    the 32bit version is faster than the 64bit, but compression is worse obviously.
    Last edited by load; 4th April 2015 at 20:58.

  13. #11
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 796 Times in 488 Posts
    Yes, I tested with a 32 bit compile. I didn't think 64 bit would be different.

  14. #12
    Member
    Join Date
    Oct 2014
    Location
    Bucharest
    Posts
    6
    Thanks
    0
    Thanked 2 Times in 1 Post
    Added a preprocessing stage and improved the main model.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •