Page 5 of 5 FirstFirst ... 345
Results 121 to 141 of 141

Thread: Paq8sk

  1. #121
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,148
    Thanks
    702
    Thanked 455 Times in 352 Posts
    122'398'376 - enwik9 -x15 -w -e1,english.dic by Paq8sk28 - terrible time - 266'984,08s and huge memory usage.
    You need to use less memory - similar like paq8pxd do and speedup compressor at least 2x.

  2. Thanks:

    suryakandau@yahoo.co.id (26th June 2020)

  3. #122
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    278
    Thanks
    43
    Thanked 50 Times in 40 Posts
    Paq8sk30
    - tweak text model
    - new hash function
    @sportman could you test it for GDCC text file (TS40.txt) using -x15 option please ? thanx
    Attached Files Attached Files
    Last edited by suryakandau@yahoo.co.id; 27th June 2020 at 15:40.

  4. Thanks:

    moisesmcardona (Today)

  5. #123
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    278
    Thanks
    43
    Thanked 50 Times in 40 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    Paq8sk30
    - tweak text model
    - new hash function
    @sportman could you test it for GDCC text file (TS40.txt) using -x15 option please ? thanx
    the result using -x10 on ts40.txt (GDC competition) on my old laptop is:
    Total 400000000 bytes compressed to 70817349 bytes.
    Time 78542.57 sec, used 15727 MB (3606260641 bytes) of memory

    i wonder how much ts40.txt can be compressed using -x15 option. is it can break below 70.xxx.xxx bytes ?

  6. #124
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,148
    Thanks
    702
    Thanked 455 Times in 352 Posts
    Where is the test file to grab?

  7. #125
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    973
    Thanks
    96
    Thanked 391 Times in 273 Posts
    Quote Originally Posted by Darek View Post
    Where is the test file to grab?
    https://www.dropbox.com/s/mxvwhupsny2hjih/TS40.7z?dl=0

  8. Thanks:

    Darek (27th June 2020)

  9. #126
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    278
    Thanks
    43
    Thanked 50 Times in 40 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    the result using -x10 on ts40.txt (GDC competition) on my old laptop is:
    Total 400000000 bytes compressed to 70817349 bytes.
    Time 78542.57 sec, used 15727 MB (3606260641 bytes) of memory

    i wonder how much ts40.txt can be compressed using -x15 option. is it can break below 70.xxx.xxx bytes ?
    @sportman could you add this result to your gdcc public test set please ? now i test paq8pxd_v89_noAVX2 using -x10 - running

  10. #127
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,148
    Thanks
    702
    Thanked 455 Times in 352 Posts
    I'm running ts40.txt tests - I've try:

    1) just -x15 option
    2) -x15 plus Byron directory
    3) best of above and -w option (sometines get better scores and decompression is OK)

  11. #128
    Member Gotty's Avatar
    Join Date
    Oct 2017
    Location
    Switzerland
    Posts
    466
    Thanks
    320
    Thanked 308 Times in 165 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    Paq8sk30
    - tweak text model
    - new hash function
    Please don't forget to always include the source code. The licensing requires that.

  12. #129
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    973
    Thanks
    96
    Thanked 391 Times in 273 Posts
    enwik8:
    15,643,209 bytes, 8,628.898 sec., paq8sk30 -x15 -w -e1,english.dic

  13. Thanks (2):

    Darek (28th June 2020),suryakandau@yahoo.co.id (28th June 2020)

  14. #130
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    278
    Thanks
    43
    Thanked 50 Times in 40 Posts
    Quote Originally Posted by Sportman View Post
    enwik8:
    15,643,209 bytes, 8,628.898 sec., paq8sk30 -x15 -w -e1,english.dic
    waoo it is 2x faster than paq8sk28. thank you sportman

  15. #131
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    278
    Thanks
    43
    Thanked 50 Times in 40 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    Paq8sk30
    - tweak text model
    - new hash function
    @sportman could you test it for GDCC text file (TS40.txt) using -x15 option please ? thanx

    ​here is the source code of paq8sk30
    Attached Files Attached Files

  16. Thanks (2):

    Darek (28th June 2020),moisesmcardona (30th June 2020)

  17. #132
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,148
    Thanks
    702
    Thanked 455 Times in 352 Posts
    70'587'620 - TS40.txt -x15 by Paq8sk30, time 38154,53s

  18. Thanks:

    suryakandau@yahoo.co.id (29th June 2020)

  19. #133
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    278
    Thanks
    43
    Thanked 50 Times in 40 Posts
    Paq8sk31
    - ​New experimental hash function to improve compression ratio
    Attached Files Attached Files

  20. Thanks:

    moisesmcardona (Today)

  21. #134
    Member Gotty's Avatar
    Join Date
    Oct 2017
    Location
    Switzerland
    Posts
    466
    Thanks
    320
    Thanked 308 Times in 165 Posts
    Quote Originally Posted by Gotty View Post
    Please don't forget to always include the source code. The licensing requires that.

  22. #135
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    278
    Thanks
    43
    Thanked 50 Times in 40 Posts
    Paq8sk32
    - improve text model by new hash function
    the result of enwik8 using -s6 -w -e1,english.dic is
    Total 100000000 bytes compressed to 16285631 bytes.
    Time 20340.22 sec, used 1290 MB (1352947102 bytes) of memory
    faster than paq8sk29
    ​enwik9 is running
    ​Here is the source code...
    here is the binary too
    Attached Files Attached Files

  23. Thanks (2):

    Gotty (Yesterday),moisesmcardona (Today)

  24. #136
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,148
    Thanks
    702
    Thanked 455 Times in 352 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    Paq8sk32
    Total 100000000 bytes compressed to 16285631 bytes.
    Time 20340.22 sec, used 1290 MB (1352947102 bytes) of memory
    Do you have the same score for paq8sk23 or paq8sk28?

  25. #137
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    278
    Thanks
    43
    Thanked 50 Times in 40 Posts
    Quote Originally Posted by Darek View Post
    Do you have the same score for paq8sk23 or paq8sk28?
    ​i just compare paq8sk32 with paq8sk29 on enwik8

  26. #138
    Member
    Join Date
    Dec 2008
    Location
    Poland, Warsaw
    Posts
    1,148
    Thanks
    702
    Thanked 455 Times in 352 Posts
    Ok and?

  27. #139
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    278
    Thanks
    43
    Thanked 50 Times in 40 Posts
    Quote Originally Posted by Darek View Post
    Ok and?
    Paq8sk32 has better score and faster than paq8sk29.

  28. Thanks:

    Darek (Yesterday)

  29. #140
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    973
    Thanks
    96
    Thanked 391 Times in 273 Posts
    enwik8:
    15,641,922 bytes, 8,629.563 sec., paq8sk32 -x15 -w -e1,english.dic

  30. Thanks (2):

    Darek (Today),suryakandau@yahoo.co.id (Today)

  31. #141
    Member
    Join Date
    Aug 2015
    Location
    indonesia
    Posts
    278
    Thanks
    43
    Thanked 50 Times in 40 Posts
    Quote Originally Posted by suryakandau@yahoo.co.id View Post
    Paq8sk32
    - improve text model by new hash function
    the result of enwik8 using -s6 -w -e1,english.dic is
    Total 100000000 bytes compressed to 16285631 bytes.
    Time 20340.22 sec, used 1290 MB (1352947102 bytes) of memory
    faster than paq8sk29
    ​enwik9 is running
    ​Here is the source code...
    here is the binary too
    Paq8sk30 -s1 ts40.txt is:
    Total 400000000 bytes compressed to 80211410 bytes.
    Time 76785.83 sec, used 658 MB (690640027 bytes) of memory

    Paq8sk32 -s1 ts40.txt is
    Total 400000000 bytes compressed to 79461853 bytes.
    Time 65043.98 sec, used 659 MB (691656451 bytes) of memory

    i wonder if using paq8sk32 -x15 -e1,english.dic on ts40.txt can it reach below 70.xxx.xxx ?

Page 5 of 5 FirstFirst ... 345

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •