Activity Stream

Filter
Sort By Time Show
Recent Recent Popular Popular Anytime Anytime Last 24 Hours Last 24 Hours Last 7 Days Last 7 Days Last 30 Days Last 30 Days All All Photos Photos Forum Forums
  • a902cd23's Avatar
    Today, 13:35
    bcm2.exe c2047m e:\.mkv Compressing 'e:\.mkv': 2633235746 -> 2521863370 in 950.5 sec bcm1.exe -9 e:\.mkv Compressing e:\.mkv: 2633235746 -> 2522463312 in 871.8 sec Because windows caches the file, timing is not fair. But I guess v2 is a bit slower.
    3 replies | 30 view(s)
  • lz77's Avatar
    Today, 13:14
    Please add to "Test 1, text. Rapid. Open part" result for pglz.
    107 replies | 10856 view(s)
  • encode's Avatar
    Today, 12:55
    Usage: bcm c infile outfile Block size by default is 100 MB Memory usage is block size * 5 For a bigger block use e.g. bcm c2000m infile outfile For decompression: bcm d bcmfile outfile
    3 replies | 30 view(s)
  • fabiorug's Avatar
    Today, 12:39
    C:\Users\User\Documents\bcm200a1\bcm.exe C:\Users\User\Videos\fe.jpg C:\Users\User\Videos\fe.bcm It doesn't work. I tried to add " but neither, maybe I digited wrong extension
    3 replies | 30 view(s)
  • encode's Avatar
    Today, 12:00
    This is the strongest BCM ever! Enjoy new release! :_superman2: Sportman, please test it on ENWIK8/ENWIK9!
    3 replies | 30 view(s)
  • Ms1's Avatar
    Today, 11:57
    Test 1 text HCR 1 mcm 0.84 -x11 504 477 981 184,532,057 2 PPMonstr 1290 1294 2584 191,444,956 Test 2 images HCR 1 EMMA 852 876 1728 383,369,222 2 bmf 2.01 -S 851 735 1586 386,901,589 Test 3 mixed data HCR 1 PPMonstr 1965 1968 3933 106,587,129 2 PPMonstr var. J, feb 16 2006 -m1271 -o16 -r1 1162 1168 2330 118,674,642 Dmitry Shkarin just recompiled old source files. All this is like 15-20 years old. C'mon guys...
    107 replies | 10856 view(s)
  • Ms1's Avatar
    Today, 11:45
    1. Leaderboards are updated. We deleted most of reference lines in the tables for full (100%) tests. 2. There will be one more interim update in November, probably by Nov. 15. To be announced.
    10 replies | 2190 view(s)
  • Gotty's Avatar
    Today, 10:32
    Gotty replied to a thread paq8px in Data Compression
    Correct. ​What about the others?
    2183 replies | 582064 view(s)
  • suryakandau@yahoo.co.id's Avatar
    Today, 01:59
    Paq8pxd90fix1 - improve jpeg compression by adding 5 apm and 1 mixer context paq8pxdv90 -s8 test.jpg 2187172 paq8pxdv90fix1 -s8 test.jpg 2185794 paq8pxdv90 -s8 a10.jpg 623059 paq8pxdv90fix1 -s8 a10.jpg 622691y here is source code, binary file and batch script to compile inside the package. :_coffee:
    976 replies | 343213 view(s)
  • suryakandau@yahoo.co.id's Avatar
    Today, 00:54
    If iam not wrong the range of coef is 8bit
    2183 replies | 582064 view(s)
  • smjohn1's Avatar
    Yesterday, 23:41
    Nice to know there is effort in this direction. With more memory to use, a big global dictionary seems to be able to improve compression ratio (significantly).
    8 replies | 267 view(s)
  • Gotty's Avatar
    Yesterday, 23:21
    Gotty replied to a thread paq8px in Data Compression
    These are my questions that help you answering your question: What is the range of coef? What is the range of 5+2*(!comp)? What is the range of zu and zv? Can you answer them? (I believe, you can.)
    2183 replies | 582064 view(s)
  • suryakandau@yahoo.co.id's Avatar
    Yesterday, 21:52
    Why min(5+2*(!comp),zu+zv) is left shifted by 8 ? Not by 4 only ?
    2183 replies | 582064 view(s)
  • Mauro Vezzosi's Avatar
    Yesterday, 19:40
    I'm working veeery slowly on LZW, my goal is to improve the compression ratio ~regardless of speed. I hope to reach some good point (and keep the source code clean to publish it). My LZW version is symmetrical and decompression will be about as slow as the compression, I need to build the dictionary the same for both. I think it won't be as fast as other LZ77 implementations.
    8 replies | 267 view(s)
  • Gotty's Avatar
    Yesterday, 18:16
    Gotty replied to a thread paq8px in Data Compression
    Aha, I see. Here it is. m.set(coef | min(5+2*(!comp),zu+zv)<<8, 2048); You will be able to fix it yourself as soon as you do the homework. When doing development however, you'll need to validate any changes: for example this mixer context could be too large and thus waste memory or could be disadvantageous for some type of images. I did not test that. Fixing this line is not enough though. You have more problems. You don't need to rush with the fixing. It will not work on the long run if you expect me (or probably anyone else) to validate your changes, fix the code, run tests, merge, do a pull request. Please do your homework. You should not try anything until you can answer (at least!!!) those questions above.
    2183 replies | 582064 view(s)
  • smjohn1's Avatar
    Yesterday, 17:42
    Again, thx for great info. LZ78's dictionary is more global than LZ77's, so it might have potential to have better compression ratio. But of course, I am looking at speeds too. As of decompression speed, we certainly want as high as possible. In quite a lot applications, though, since decompression speed is much higher than compression speed, and if compression ( write ) and decompression ( read ) happen equally often, i.e., more symmetric, then decompression speed usually isn't a bottleneck, so a bit lower decompression speed is a good tradeoff for higher compression ratio. Now that LZW's patent should have expired ( am I correct? ), people may be able to start to re-examine it again. Of course, those are just my random thoughts. Thanks again.
    8 replies | 267 view(s)
  • suryakandau@yahoo.co.id's Avatar
    Yesterday, 16:34
    Yes..thank you..
    2183 replies | 582064 view(s)
  • Gotty's Avatar
    Yesterday, 16:30
    Gotty replied to a thread paq8px in Data Compression
    I'm still unsure what you mean. Are you asking me how to fix this code? m.set( coef | min(5+2*(!comp),zu+zv),1024);
    2183 replies | 582064 view(s)
  • suryakandau@yahoo.co.id's Avatar
    Yesterday, 16:28
    How should be my new context mixer ?
    2183 replies | 582064 view(s)
  • Gotty's Avatar
    Yesterday, 16:07
    Gotty replied to a thread paq8px in Data Compression
    I don't understand your question. Are you asking me about your new contexts or new contexts in general?
    2183 replies | 582064 view(s)
  • Shelwien's Avatar
    Yesterday, 15:49
    > So in your opinion, what is the best variation in the LZ78 family, if not LZW? Define "best". Compression-wise its probably GLZA currently. > Any LZ4 counterpart in LZ78 family? If you want something simple, it may be this: https://encode.su/threads/2983-Cracking-an-old-MS-DOS-game-s-compressed-file?p=57817&viewfull=1#post57817 As to speed potential, LZ78 is inherently slower than LZ77, because it has to do extra work during decoding, also large-alphabet entropy coding is much harder to do. > I am just wondering why there are so many LZ77 variations in lz-bench but not LZ78? LZW/LZ78 variants were pretty popular in 198x, then people started patenting them, and thus killed that branch of development. Its somewhat described here: https://oku.edu.mie-u.ac.jp/~okumura/compression/history.html Also lzbench is just a personal project of its developer (inikep): https://github.com/inikep/lzbench For example, here's another similar benchmark: https://github.com/powturbo/TurboBench with some different codecs. Or this: http://mattmahoney.net/dc/text.html - this one includes some LZW versions. > Is it because LZ78 has no advantage over LZ77, in compression ratio and/or speeds? Technically LZ78 includes LZ77 (I mean classes of compression algorithms, not original methods) - we can say that {token_type;distance;length} is just a method of dictionary index generation. Also there's a whole spectrum of intermediate algorithms, which attempt to reduce LZ77 redundancy (assigning many different codes to same strings): LZFG,ACB,LZP,ROLZ,even PPM can be considered one (ROLZ with unary length coding). I do think that LZ78 has potential to be stronger than LZ77 - we can even see that with GLZA. It may be not realized yet, but we cannot force people to work on it.
    8 replies | 267 view(s)
  • suryakandau@yahoo.co.id's Avatar
    Yesterday, 15:26
    so what is your explanation about them ?
    2183 replies | 582064 view(s)
  • Gotty's Avatar
    Yesterday, 15:09
    Gotty replied to a thread paq8px in Data Compression
    How about doing some homework? That is very-very essential. You need to do efforts, please.
    2183 replies | 582064 view(s)
  • suryakandau@yahoo.co.id's Avatar
    Yesterday, 15:07
    what is the use of new mixer context ? what is the use of new apm ?
    2183 replies | 582064 view(s)
  • suryakandau@yahoo.co.id's Avatar
    Yesterday, 14:37
    Please give some explanation about it...
    2183 replies | 582064 view(s)
  • suryakandau@yahoo.co.id's Avatar
    Yesterday, 14:35
    What is your email ? Maybe it is more ok to discuss with email...if I use git I think it more universalthan use this forum...
    2183 replies | 582064 view(s)
  • Gotty's Avatar
    Yesterday, 14:33
    Gotty replied to a thread paq8px in Data Compression
    I promised that we will help if you have any questions. So let's do it the other way around. I'll ask. Let's look at this single line, which is new: m.set( coef | min(5+2*(!comp),zu+zv),1024); What does 1024 mean on this line? What the "|" operator does exactly? What is the range of coef? What is the range of 5+2*(!comp)? What is the range of zu and zv? What do you need to adjust if you add a new mixer context?
    2183 replies | 582064 view(s)
  • Gotty's Avatar
    Yesterday, 14:26
    Gotty replied to a thread paq8px in Data Compression
    Could you please withdraw this version? Probably delete the attachment. Could you follow our guidelines, please? That is: 1) use git. 2) before posting any tweaks do understand how paq8px works.
    2183 replies | 582064 view(s)
  • suryakandau@yahoo.co.id's Avatar
    Yesterday, 10:23
    Paq8px197 - improve jpeg recompression Size Compressed Sec paq8pxd_v90 -s8 PH01046J.JPG 135611 107464 6 paq8px -8 PH01046J.JPG 135611 105624 7 paq8pxd_v90 -s8 20200708_105932.jpg 112038 81483 paq8px -8 f.jpg (darek corpus) 112038 81299 paq8pxd_v90 -s8 A10.jpg 842468 623059 25 paq8px -8 A10.jpg 842468 624267 36 paq8pxd_v90 -s8 test.JPG 3162196 2187172 85 paq8px -8 test.JPG 3162196 2194077 134 hhmm looks paq8px better than paq8pxd on small jpeg files...:eek::confused:
    2183 replies | 582064 view(s)
  • suryakandau@yahoo.co.id's Avatar
    Yesterday, 05:54
    @darek could you please this version to your DBA corpus ? Thank you!
    68 replies | 5480 view(s)
  • smjohn1's Avatar
    Yesterday, 04:12
    Thanks a lot again for lots of useful info. So in your opinion, what is the best variation in the LZ78 family, if not LZW? Any LZ4 counterpart in LZ78 family? I am just wondering why there are so many LZ77 variations in lz-bench but not LZ78? Is it because LZ78 has no advantage over LZ77, in compression ratio and/or speeds?
    8 replies | 267 view(s)
  • Shelwien's Avatar
    Yesterday, 02:46
    > what is a best quality open source package for standard LZW? Compress? https://github.com/andrew-aladev/lzws https://en.wikipedia.org/wiki/Compress#External_links https://sourceforge.net/p/giflib/code/ci/master/tree/ https://github.com/rouault/libtiff/tree/master/libtiff > And just curious, why is LZW not included in LZ-Bench or already but under a different package name? LZW had been patented for ~30 years, so people try to ignore it. Also compression ratio is nothing interesting. > efficiently represent index of words in the dictionary for better > compression ratio, which in turn may effect speeds. No one tried before? LZW is a specific algorithm, which is not especially good. There're various different implementations of LZ78 methods, like WRT text filters, or https://github.com/jrmuizel/GLZA
    8 replies | 267 view(s)
  • smjohn1's Avatar
    Yesterday, 00:00
    Thanks a lot. That thread is quite helpful. But besides lzw-ab, what is a best quality open source package for standard LZW? Compress? And just curious, why is LZW not included in LZ-Bench or already but under a different package name? I agree, LZ4-like implementation might not be better than LZ4 in speeds, but it might be worthy trying, as there is room in more efficiently represent index of words in the dictionary for better compression ratio, which in turn may effect speeds. No one tried before?
    8 replies | 267 view(s)
  • Shelwien's Avatar
    28th October 2020, 23:25
    There was this thread recently: https://encode.su/threads/3419-Are-there-LZW-LZ78-LZC-etc-variants-with-efficient-codeword-encoding As to speed, LZW without entropy coding may have similar complexity, but LZW still has to do more work normally (like dictionary updates), plus LZ4 is already optimized - has different handlers for some ranges of match lengths etc. The same would have to be done for LZW too.
    8 replies | 267 view(s)
  • smjohn1's Avatar
    28th October 2020, 22:09
    Could anyone recommend a high quality open source implementation of LZW? By high quality, I mean close to that of Deflate or LZ4. Also are there any LZ4 equivalent format of LZW, in terms of compression and decompression speeds? Thx in advance for any info.
    8 replies | 267 view(s)
  • suryakandau@yahoo.co.id's Avatar
    28th October 2020, 15:01
    Fp8sk31 - improve jpeg recompression port from paq8px195 - the compression and decompression speed still fast fp8sk31 -8 f.jpg 81782 bytes paq8pxdv89 -s8 f.jpg 81825 bytes paq8pxdv90 -s8 f.jpg 81483 bytes paq8pxv194 -8 f.jpg 81606 bytes paq8pxv196 -8 f.jpg 81359 bytes fp8sk31 -8 a10.jpg 627009 bytes paq8pxdv90 -s8 a10.jpg 623059 bytes paq8pxv196 -8 a10.jpg 624651 bytes
    68 replies | 5480 view(s)
  • Gotty's Avatar
    26th October 2020, 22:12
    Gotty replied to a thread paq8px in Data Compression
    These are the ones that cover some special cases: MJPEG 1) http://jjc.freeshell.org/turning_pages.html Turning Pages AVI 2) https://www.mainconcept.com/getting-started/samples.html Motion-JPEG, 1920x1080 3) http://www.doicamera.com/digital_video.htm Mjpeg sample video (22 mb) JPEGs with thumbnails https://encode.su/threads/342-paq8px?p=64174&viewfull=1#post64174 https://encode.su/threads/342-paq8px?p=64175&viewfull=1#post64175 https://encode.su/threads/342-paq8px?p=64176&viewfull=1#post64176 Several multipage TIFF with JPEGs http://156.35.94.8/tmp/ For plain single JPEG images you'll just need to look around in your pc or google. You'll need to find baseline JPEG files as paq8px does not support progressive JPEGs.
    2183 replies | 582064 view(s)
  • Darek's Avatar
    26th October 2020, 15:14
    Darek replied to a thread paq8px in Data Compression
    Here are two files: A10.jpg from Maximum Compression and F.JPG file from my testset.
    2183 replies | 582064 view(s)
  • Nania Francesco's Avatar
    26th October 2020, 12:25
    I have managed to change yours nationality! Sorry I'm late!
    204 replies | 130027 view(s)
  • suryakandau@yahoo.co.id's Avatar
    26th October 2020, 11:47
    May you give some JPEG files ? I don't have any files except test.jpg
    2183 replies | 582064 view(s)
  • Gotty's Avatar
    26th October 2020, 10:32
    Gotty replied to a thread paq8px in Data Compression
    I see. You should test your changes with many different files. We don''t build paq8px to compress a single test.jpg. It's a general compressor. Never add complexity or use more memory or slow down compression just for one file. If you don't test with multiple files, you will not be able to properly evaluate your changes. So please do.
    2183 replies | 582064 view(s)
  • suryakandau@yahoo.co.id's Avatar
    26th October 2020, 10:23
    When I add that context it can reduce test.jpg size
    2183 replies | 582064 view(s)
  • Gotty's Avatar
    26th October 2020, 10:17
    Gotty replied to a thread paq8px in Data Compression
    Yes. Almost. The "tweaks" in the last line are tweaks indeed. That is: parameter tuning = trying out different parameters and test them. In that regard it's the same as yours. However parameter tuning should be applied to parameters only. Your code contained changes of values (like 1024 to 2048 in m1->set) that are not tunable parameters = they must stay intact when the range of the context is unmodified (which is 1024 in this example). Knowing some background helps in parameter tweaking: you must know what you may tune and what shall stay fixed or be calculated. It's also important to know how to combine contexts. (Learning opportunity for you. ->) When it's about combining vectors, you may add them, average them, weight them. But when the contexts are simply independent values you should not add them. Your code (wrong): cxt = hash(++n, hc, advPred / 11 + (runPred), sSum2 >> 6); Fix: cxt = hash(++n, hc, advPred / 11, runPred, sSum2 >> 6); So this release is about fixing your code after I tested the added contexts more thoroughly. I removed some contexts of yours because they seemed to have no benefit. My question is: why did you add them? Did you have some specific files where they are beneficial?
    2183 replies | 582064 view(s)
  • suryakandau@yahoo.co.id's Avatar
    26th October 2020, 02:31
    It looks similar like I do, random forest too...:_haha2:
    2183 replies | 582064 view(s)
  • kaitz's Avatar
    26th October 2020, 00:42
    kaitz replied to a thread paq8px in Data Compression
    Size Compressed Sec paq8pxd_v90 -s8 PH01046J.JPG 135611 107464 6 paq8px -8 PH01046J.JPG 135611 105624 7 paq8pxd_v90 -s8 20200708_105932.jpg 3009379 2237601 83 paq8px -8 20200708_105932.jpg 3009379 2756675 748 (drops to default*) paq8pxd_v90 -s8 A10.jpg 842468 623059 25 paq8px -8 A10.jpg 842468 624651 36 paq8pxd_v90 -s8 test.JPG 3162196 2187172 85 paq8px -8 test.JPG 3162196 2194672 134 paq8pxd_v90 -s8 paq8px_v193_4_Corpuses.jpg 3340610 1418082 96 paq8px -8 paq8px_v193_4_Corpuses.jpg 3340610 1513474 142 paq8px -8 maxpc.pdf 14384698 8895217 1827 paq8pxd_v90 -s8 maxpc.pdf 14384698 8920647 1352 paq8px 2520 MB paq8pxd_v90 1978 MB (about 600MB is JPEG) *replace JASSERT((cPos & 63U) == 0) with: while (cPos &63U) cPos ++;
    2183 replies | 582064 view(s)
  • Gotty's Avatar
    25th October 2020, 23:39
    Gotty replied to a thread paq8px in Data Compression
    Fixes and improvements in jpeg model - fixed indexing (thank you, mpais!) - removed 2 contexts and 1 apm, added 1 context - hashing instead of direct contexts in apms (thank you, Kaitz!) - using abs for advPred and lcp in apms (thank you, Kaitz!) - reduced memory use in apms (-24 MB on level 8) - tweaks
    2183 replies | 582064 view(s)
  • Nania Francesco's Avatar
    25th October 2020, 22:53
    As soon as I can get my hands on the resource files I use for the construction of html pages ! I'm sorry I got the nation wrong!
    204 replies | 130027 view(s)
  • schnaader's Avatar
    25th October 2020, 21:15
    Thanks for adding Precomp to the benchmark! Could you change the country from RUS to GER? Some suggestions for options that might improve results for some of the tests: "-cl -intense" for summary, app1, app2, game1, game2, iso, raw, bin tests (intense is a bit slower, but detects headerless zlib streams) "-cl -lf+d1", "-cl -lf+d2" or "-cl -lf+d4" for wav test (enables lzma2 delta compression which will often lead to better results on audio)
    204 replies | 130027 view(s)
  • Darek's Avatar
    25th October 2020, 19:15
    Darek replied to a thread Paq8pxd dict in Data Compression
    At first my testset for paq8px_v90. Nice improvements. Still some bytes behind latest paq8px (about 100KB - without LSTM) but almost all files got some gains. There is only some loses for biggest 24bpp images - both of them loses about 400 bytes.
    976 replies | 343213 view(s)
  • anormal's Avatar
    25th October 2020, 18:08
    After some fixing and testing, I finished using this format: 5bits-number of tokens tokens... =12bits, 9bits index in dict+3bits length of match, encoded as 2 groups of 6bit literals...=6bits each one Each group of 6bits is finally encoded with a standard huffman encoder. I am happy as even very small strings, 4-5 chars are always compressed. Some stats, sizes are in bits: line:775 tokens:5(60) literals:25(150) orig:392 - naive6 : 294 - compressed no huffman:214 - HUFFMAN :202 line:776 tokens:3(36) literals:5(30) orig:136 - naive6 : 102 - compressed no huffman:70 - HUFFMAN :60 line:777 tokens:4(48) literals:11(66) orig:232 - naive6 : 174 - compressed no huffman:118 - HUFFMAN :110 line:778 tokens:13(156) literals:21(126) orig:592 - naive6 : 444 - compressed no huffman:286 - HUFFMAN :301 line:779 tokens:1(12) literals:11(66) orig:112 - naive6 : 84 - compressed no huffman:82 - HUFFMAN :69 line:780 tokens:2(24) literals:10(60) orig:136 - naive6 : 102 - compressed no huffman:88 - HUFFMAN :73 line:781 tokens:2(24) literals:3(18) orig:88 - naive6 : 66 - compressed no huffman:46 - HUFFMAN :46 line:782 tokens:2(24) literals:9(54) orig:128 - naive6 : 96 - compressed no huffman:82 - HUFFMAN :74 line:783 tokens:2(24) literals:14(84) orig:216 - naive6 : 162 - compressed no huffman:112 - HUFFMAN :92 line:784 tokens:6(72) literals:9(54) orig:240 - naive6 : 180 - compressed no huffman:130 - HUFFMAN :120 line:785 tokens:1(12) literals:5(30) orig:80 - naive6 : 60 - compressed no huffman:46 - HUFFMAN :42 line:786 tokens:1(12) literals:5(30) orig:80 - naive6 : 60 - compressed no huffman:46 - HUFFMAN :42 line:787 tokens:5(60) literals:14(84) orig:280 - naive6 : 210 - compressed no huffman:148 - HUFFMAN :139 line:788 tokens:4(48) literals:6(36) orig:168 - naive6 : 126 - compressed no huffman:88 - HUFFMAN :98 line:789 tokens:0(0) literals:5(30) orig:40 - naive6 : 30 - compressed no huffman:34 - HUFFMAN :28 Final compression no huffman:216246 / 356872 = ,6059 Final compression huffman:185306 / 356872 = ,5193 I'll try now to build a better dictionary that provides more matches and so better compression. What I am using now is simply inserting+merging ngrams sorted by frequency of use.
    7 replies | 450 view(s)
  • moisesmcardona's Avatar
    25th October 2020, 16:02
    moisesmcardona replied to a thread paq8px in Data Compression
    Honestly, I don't see where is the bullying part in this topic... Let's be good people and move on and continue delivering the best we do here :D
    2183 replies | 582064 view(s)
  • Nania Francesco's Avatar
    25th October 2020, 14:38
    I was able to test all the archivers and compressors again and also added Lzpm 0.18 and Precomp 0.4.7. I hope to release more updates soon! only from: http://heartofcomp.altervista.org/MOC/MOCA.htm
    204 replies | 130027 view(s)
  • suryakandau@yahoo.co.id's Avatar
    25th October 2020, 10:39
    2183 replies | 582064 view(s)
  • suryakandau@yahoo.co.id's Avatar
    25th October 2020, 06:14
    I believe that every people in this forum is not a child so they can know which is bullying and which is not bullying. Imagine if your children becomes bullying victim so what should you do ? Does this forum support bullying to another people ? I believe that the answer is doesn't.
    2183 replies | 582064 view(s)
  • Gotty's Avatar
    25th October 2020, 02:20
    Gotty replied to a thread paq8px in Data Compression
    It does not sound bullying to me. It is more like expressing anger or dislike because of your past. You did stupid things. <- It's also not bullying, it's a statement. Despite of all that you are welcome here. Read again our posts above and you'll see. For me your intention is clear in this thread: you would like to contribute (in contrast to your past when you wanted to have fame for yourself only). I take it as a sign of change and willingness. Let's see the present. "Blind" parameter tuning will not be enough to get far. As mpais stated it works but it's not very efficient. Note: please understand his motivation. By reading all his posts you will see that he would like to make/keep paq8px as high quality as possible. And any change that is not in that direction makes him feel uneasy. He expressed his fear that your contributions may not reach a high quality mark. And we all know that his fear has ground - I expressed the same in my earlier posts when I fixed bugs in your code. Let's look into the future then. Please, expect to spend 3-6 months just to understand the code base. I actually needed more time, so it may be more ;-) You will need to read the code line by line. Are you ready to do it? It's not an easy piece of software, you already know that. Also you will need to do proper testing before any contribution. What does it mean? - You need to test on many different files; - You need to test the effect of your changes one by one. I have the feeling that you add many new context at once and when the result is satisfying you don't even check which context was the one that actually helped. You must include only those changes that actually help and eliminate any changes that have no or little effect as with any unnecessary code the compression gets slower and slower; - You need to disable NDEBUG and run sanity tests to see if your changes trigger any asserts. Your contribution to the jpeg model was a good start. Nice compression gain but not so good memory usage and time. Although I don't have much time I'll see what I can do to make it a bit better in those areas.
    2183 replies | 582064 view(s)
  • suryakandau@yahoo.co.id's Avatar
    24th October 2020, 22:49
    as I said before it is not about programmers ethic anymore it is about bullying and it becomes human legal case now. Is this community support bullying another people ?
    2183 replies | 582064 view(s)
  • moisesmcardona's Avatar
    24th October 2020, 22:16
    Yes sir, attached are both native (AVX2) and standard (SSE2) builds, compiled with Multithreading support (-DMT)
    976 replies | 343213 view(s)
  • Darek's Avatar
    24th October 2020, 22:07
    Darek replied to a thread Paq8pxd dict in Data Compression
    If you can make build with paq8pxd v90 I could test it. Question - 4 corpuses are enough to test or I could take another files?
    976 replies | 343213 view(s)
  • Gotty's Avatar
    24th October 2020, 21:47
    Gotty replied to a thread paq8px in Data Compression
    The file has the same content, only line endings are different. Paq8px doesn't care, so the files are essentially the same. You can use either of them.
    2183 replies | 582064 view(s)
  • hexagone's Avatar
    24th October 2020, 21:35
    hexagone replied to a thread paq8px in Data Compression
    If you want to be part of a community, such comments are not helpful. Given your well documented history of mis-representations (that led to your disqualification from the GDCC), you should know better than questioning people's ethics and character.
    2183 replies | 582064 view(s)
  • Darek's Avatar
    24th October 2020, 20:21
    Darek replied to a thread paq8px in Data Compression
    Scores of my testset for paq8px_v185 - nice gain (150 bytes) for F.JPG file. :)
    2183 replies | 582064 view(s)
  • moisesmcardona's Avatar
    24th October 2020, 18:33
    Hey @kaitz, I went ahead and created a CMakeLists.txt file to allow compilation using CMake/Make. See my PR here: https://github.com/kaitz/paq8pxd/pull/12 :_yahoo2:
    976 replies | 343213 view(s)
  • anormal's Avatar
    24th October 2020, 17:43
    I don't think is useful out of this special case. Maybe it could be scaled in a similar way, for encoding a predefined set of strings, and when you need to decompress any string one at a time. But for modern processors/ram, you could use a much bigger dict, longest pointers in dictionary, etc. As I said you could you check other projects as femtozip or unishox, and Littlebit as posted before (I tested many, and usually shoco and smasz are suggested, but these I mentioned are much advanced and mature) I am going to try a GA to see if there are more interesting strategies while packing the dictionary, than the one I am using. If anyone could explain in few words as Z-Std algorithms for building dictionary works, cover and fastcover? (as I see in sources), I'd be happy :D @danlock, yes you are right, standard Sinclair Spectrum has 48KB (including screen ram), I think Vic-20 had 4KB? There are also many MSX systems with varied ram sizes, 16,32,64, etc, etc... Spectrum 128, has pageable banks, etc... Edit: I've just remembered I was looking for something similar years ago, while I was building a Chess png repo, and tried to compress each game apart in the DB. I'd try with pngs when I've tested more things, but with maybe a 1MB dict? It could be fun to explore this.
    7 replies | 450 view(s)
  • kaitz's Avatar
    24th October 2020, 15:33
    kaitz replied to a thread Paq8pxd dict in Data Compression
    Confirmed. I made GitHub issue for it. Will look into it in january. --- Also uploaded paq8pxd_v90, if someone wants to test. Cant upload large files now, so source only.
    976 replies | 343213 view(s)
  • suryakandau@yahoo.co.id's Avatar
    24th October 2020, 15:15
    This is not about programmers ethic anymore but it is about human legal case now. If gdcc choose you as the winner I think that is not a wise decision.
    2183 replies | 582064 view(s)
  • Darek's Avatar
    24th October 2020, 13:29
    Darek replied to a thread paq8px in Data Compression
    Scores of paq8px_v194 for my testset - there no big changes (especially on my jpeg file too :( )
    2183 replies | 582064 view(s)
  • Darek's Avatar
    24th October 2020, 13:26
    Darek replied to a thread paq8px in Data Compression
    Lenght of english.exp file is different (shorter) than latest issue. You need to swap this file for newest. However, from tests on my and Silesia files there no big impact of change this file.
    2183 replies | 582064 view(s)
  • suryakandau@yahoo.co.id's Avatar
    24th October 2020, 12:42
    I smell bullying effort here. That is not good for this forum to keep that. I know your Emma compressor is number one in gdcc image compression but your character is not good.
    2183 replies | 582064 view(s)
  • Gotty's Avatar
    24th October 2020, 12:12
    Gotty replied to a thread paq8px in Data Compression
    I've just came up with a name of this approach: "Random Forest" :D Yes, it's very clear he has no understanding.
    2183 replies | 582064 view(s)
  • mpais's Avatar
    24th October 2020, 11:09
    mpais replied to a thread paq8px in Data Compression
    Seems I'm late to the party as usual JpegModel.cpp, line 699: cxt = hash(++n, hc, advPred / 13, prevCoef2 / 20, prevCoefRs); advPred is And I'm guessing no one is testing his fpaq8 fork on 8 bit audio files. Almost 50% slower for about 0.22% to 0.28% improvement from 193fix2. 17 extra contexts and 13 extra daisy-chained SSE stages from a quick glance. Don't get me wrong, it's a valid approach, even if it's not very efficient, and the JPEG model is one of the lowest complexity models in paq8 so there's still some low hanging fruit, so it's probably a good choice for someone wanting to start helping development as it's one of the easiest to improve. And I don't really have the time right now to rewrite this model so I can't really put my money where my mouth is, hence you guys are free to take my opinion as you please. There is however the history of blatant disregard for proper attribution from Surya (and I'm not entirely convinced he and bwt are different users, but that's a different story), which it seems has even gotten him banned from the GDCC. I'll be the first to admit my coding skills are mediocre at best, and I usually make dumb mistakes that you so kindly fix :D, but this is just throwing stuff at the wall and seeing what sticks, without even understanding what one is doing. I get it, it's easy to just learn how to compile paq8, then tweak some numbers or add "hashes" (as he puts it) with different combinations of previous contexts, run it and see what effect it had. Oh, yeah, and strip the interface, change the program name, release the binary only and do wild claims... Surya, if you have changed your ways and really want to help, I welcome your efforts and I'm sure everyone here will help you learn. Tempting as it may be to just brute-force compression improvements, I suggest you try to first learn and code some simpler compression methods to hone your skills and get a better understanding of the field, instead of jumping head first into such a complex codebase.
    2183 replies | 582064 view(s)
More Activity