Results 1 to 6 of 6

Thread: New rule for large text benchmark

  1. #1
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 779 Times in 486 Posts
    I am adding a new rule for the large text benchmark. As of today I will only publish benchmarks 30 days after the latest version of the program is published. Also, I only test the latest version. That means if you publish an update, then the clock is set back to 0.

    I have to do this because there are too many unstable releases (especially paq8o and lpaq) which are useless to users if there is an update every few days that breaks compatibility to get a bit more compression. Also, I want to encourage developers to do their own testing to save me work. This is an open benchmark so anyone can send me results, or better, post them to forums like this. I will still do some testing and post results to forums before the 30 days are up.

  2. #2
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,507
    Thanks
    742
    Thanked 665 Times in 359 Posts
    this will mean that too often updated programs will never be tested

    it's better to have rule that new versions aren't tested due 30 days since the last one

  3. #3
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    406
    Thanks
    155
    Thanked 235 Times in 127 Posts
    Well i have Celeron 2.0 GHz 512 MB ram PC (its good PC ). I can't test large files when it's about paq8o. It takes to long to test enwiki or even calgary corpus(options >6). So i hope others will make tests that needs more then my pc can offer....

    or somthing like that

    good rule
    KZo


  4. #4
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    I am totally d' I grant with Bulat is a rule that doesn't encourage very the development but only the back - compatibility with the past!

  5. #5
    Member
    Join Date
    May 2008
    Location
    Estonia
    Posts
    406
    Thanks
    155
    Thanked 235 Times in 127 Posts
    i someone releases some x compressor then you can never tell if its better then y compressor about same release time after 25 day's.
    KZo


  6. #6
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 779 Times in 486 Posts
    Quote Originally Posted by Bulat Ziganshin
    this will mean that too often updated programs will never be tested
    Well, they will be, but the results will just appear here instead of on my website until the software is stable, nobody can find any bugs, and nobody has ideas on how to make the speed or compression better. I have some ideas to improve the code myself, but unfortunately I cant get any work done if I spend all my time testing. Of course I have to follow my own rules too.

Similar Threads

  1. Text Detection
    By Simon Berger in forum Data Compression
    Replies: 15
    Last Post: 30th May 2009, 10:58
  2. LZSS with a large dictionary
    By encode in forum Data Compression
    Replies: 31
    Last Post: 31st July 2008, 22:15
  3. Large text benchmark
    By Matt Mahoney in forum Forum Archive
    Replies: 39
    Last Post: 13th January 2008, 01:57

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •