Results 1 to 17 of 17

Thread: cmixHP

  1. #1
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    380
    Thanks
    56
    Thanked 68 Times in 54 Posts

    cmixHP

    this is the source code and binary of cmix for enwik9 benchmark compiled with dev c++ 5.11. I have tested it on xml file using 7-8 gb, the result is
    5345280 bytes -> 242275 bytes in 15156.84 s.
    cross entropy: 0.363
    is it something wrong ? i think it slow...
    Attached Files Attached Files

  2. #2
    Member CompressMaster's Avatar
    Join Date
    Jun 2018
    Location
    Lovinobana, Slovakia
    Posts
    199
    Thanks
    58
    Thanked 15 Times in 15 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    this is the source code and binary of cmix for enwik9 benchmark compiled with dev c++ 5.11. I have tested it on xml file using 7-8 gb, the result is
    5345280 bytes -> 242275 bytes in 15156.84 s.
    cross entropy: 0.363
    is it something wrong ? i think it slow...
    Yes, it is. Why you posted new cmix versions as multiple threads? Better to have only one - there´s "cmix" in order to make forum more clean. Otherwise, it´s garbage and lot of versions of the same software across the site... it´s still about CMIX.

    If I were admin of this forum, I would make one thread per software. @schnaader follows my advice - he created one thread about precomp - all versions will be there.

  3. #3
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    380
    Thanks
    56
    Thanked 68 Times in 54 Posts
    Quote Originally Posted by CompressMaster View Post
    Yes, it is. Why you posted new cmix versions as multiple threads? Better to have only one - there´s "cmix" in order to make forum more clean. Otherwise, it´s garbage and lot of versions of the same software across the site... it´s still about CMIX.

    If I were admin of this forum, I would make one thread per software. @schnaader follows my advice - he created one thread about precomp - all versions will be there.
    because this thread is not official cmix version. in this thread everyone can try cmix with smaller memory usage. by the way why when i use dictionary/english.dic the output can not be produced ??

  4. #4
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    380
    Thanks
    56
    Thanked 68 Times in 54 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    because this thread is not official cmix version. in this thread everyone can try cmix with smaller memory usage. by the way why when i use dictionary/english.dic the output can not be produced ??
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	cmixerr.PNG 
Views:	54 
Size:	18.1 KB 
ID:	7492  

  5. #5
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    380
    Thanks
    56
    Thanked 68 Times in 54 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post

    does it caused by compiled with dev c++ ?? @shelwien could you compiled it using mingw please ? thanx

  6. #6
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,982
    Thanks
    298
    Thanked 1,309 Times in 745 Posts
    This version actually compiles...
    Attached Files Attached Files

  7. Thanks:

    suryakandau@yahoo.co.id (19th March 2020)

  8. #7
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    enwik8:
    15,650,465 bytes, 146,973.154 sec. - 146,730.280 sec., cmixhp (memory comp/decomp just over 8GB)
    Last edited by Sportman; 20th March 2020 at 16:29. Reason: Compare ok

  9. Thanks:

    suryakandau@yahoo.co.id (19th March 2020)

  10. #8
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    380
    Thanks
    56
    Thanked 68 Times in 54 Posts
    Quote Originally Posted by Sportman View Post
    enwik8:
    15,650,465 bytes, 146,973.154 sec. - 146,730.280 sec., cmixhp (memory comp/decomp just over 8GB)

    How about enwik9 ?

  11. #9
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    1,026
    Thanks
    103
    Thanked 410 Times in 285 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    How about enwik9 ?
    No, that takes 2 x 2 weeks.

    Probably:
    122,047,657 bytes

  12. #10
    Member
    Join Date
    Feb 2020
    Location
    here
    Posts
    18
    Thanks
    6
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    How about enwik9 ?
    i'm running
    Last edited by well; 21st March 2020 at 16:00.

  13. #11
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    380
    Thanks
    56
    Thanked 68 Times in 54 Posts
    Quote Originally Posted by Sportman View Post
    No, that takes 2 x 2 weeks.

    Probably:
    122,047,657 bytes

    it is because the lstm model in cmix...so it can not beat phda9 now...if use another technique maybe it can faster...or open sourced of phda9 n improve the phda9 better because the baseline is phda9

  14. #12
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    380
    Thanks
    56
    Thanked 68 Times in 54 Posts
    cmixHP2
    in this version i don't use lstm model.. the result is worse but for enwik9 benchmark it needs <10 gb RAM and the time is <= 100 hours
    1000000000 bytes -> 149086410 bytes in 321779.79 s.
    this is the source and the binary compiled with dev c++. maybe someone could compiled it using mingw please ? thank you
    Attached Files Attached Files

  15. #13
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,982
    Thanks
    298
    Thanked 1,309 Times in 745 Posts
    here, gcc 8.2.
    Seems like a patch of cmix_v7.
    Attached Files Attached Files

  16. #14
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    380
    Thanks
    56
    Thanked 68 Times in 54 Posts
    Quote Originally Posted by Shelwien View Post
    here, gcc 8.2.
    Seems like a patch of cmix_v7.
    it is fork from cmix v7 because from ltcb site the fastest version of cmix is cmix v7....the result of xml file for cmixHP3 is:
    5345280 bytes -> 267754 bytes in 2038.38 s.
    cross entropy: 0.401
    enwik9 on running

  17. #15
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    380
    Thanks
    56
    Thanked 68 Times in 54 Posts
    Quote Originally Posted by Shelwien View Post
    here, gcc 8.2.
    Seems like a patch of cmix_v7.

    @sportman could you test this version on your computer ? Thank you

  18. #16
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    380
    Thanks
    56
    Thanked 68 Times in 54 Posts
    Quote Originally Posted by Sportman View Post
    enwik8:
    15,650,465 bytes, 146,973.154 sec. - 146,730.280 sec., cmixhp (memory comp/decomp just over 8GB)

    Are you sure about compression time ? Because it is forked from cmix v17 with reduced memory usage …..in cmix thread you said that cmix v18 take only 8.76 days for enwik9...is it caused by compiled source code ???

  19. #17
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    380
    Thanks
    56
    Thanked 68 Times in 54 Posts
    cmixHP3
    it needs only <=2 gb RAM and ~100 hours.
    1000000000 bytes -> 148153579 bytes in 377474.20 s.
    cross entropy: 1.185
    Attached Files Attached Files

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •