Page 1 of 8 123 ... LastLast
Results 1 to 30 of 215

Thread: CHK Hash Tool

  1. #1
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts

    Cool CHK Hash Tool

    CHK v1.20

    The first fully-featured release is here:

    http://compressme.net/



    The most important new features:
    • SHA1-Base32 and SHA384 calculation
    • "Add Folder" command
    • Ability to move columns, "sort arrow" in list view
    • Speed optimizations (CRC32 is more than 4 times faster)

  2. Thanks:

    Erich (5th November 2017)

  3. #2
    Member FatBit's Avatar
    Join Date
    Jan 2012
    Location
    Prague, CZ
    Posts
    190
    Thanks
    0
    Thanked 36 Times in 27 Posts
    Dear Mr. Encode,

    it looks like file is broken. I am not able to decompress. Can anyone confirm?

    Best regards,

    FatBit

  4. #3
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    Please re-download the file, check SHA1. I've tested the files on a few PCs and the iPad - everything seems fine

  5. #4
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 795 Times in 488 Posts
    32 bit download worked OK for me. Some timings on a 2.0 GHz T3200 on enwik9 in disk cache.

    chk 1.20 32 bit (timed with a watch)
    crc32 3 sec
    md5 9
    sha1 13
    sha256 22
    sha512 49
    keccak 94

    fsum 2.51 (timer global)
    crc32 5.0 sec
    md5 5.2
    sha1 5.6
    sha256 13.3
    sha512 142.6

    md5sum 1.22
    md5 6.3 sec

    zpaqd 6.24 (s command)
    sha1 10.6 sec

  6. Thanks:

    encode (19th April 2013)

  7. #5
    Member FatBit's Avatar
    Join Date
    Jan 2012
    Location
    Prague, CZ
    Posts
    190
    Thanks
    0
    Thanked 36 Times in 27 Posts
    Now is OK. Some "noise" was present on my "wire".

    FatBit

  8. #6
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    CHK v1.21 update:
    • Fixed annoying bug - Exception during system folders scan
    • Enabled MD4 (It was there actually)

    http://compressme.net/


  9. Thanks (3):

    Edison007 (3rd November 2013),Erich (5th November 2017),Matt Mahoney (4th November 2013)

  10. #7
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    BTW, if you have any ideas about CHK improvement, or in case if you're missing a feature or a hash type in CHK - let me know!



    My ideas for future improvements:
    • Do we need the RIPEMD160 hash? As a note, PGP standard includes SHA1, RIPEMD160 (As a non NSA alternative) and SHA2 (SHA256, SHA384, SHA512). RIPEMD160 is slower than SHA1
    • I'll remove column dragging. VCL is buggy as hell - each movement to user's freedom introduce a bunch of bugs that cannot be fixed. Currently, if you'll change a Windows theme - CHK will go nuts
    • Somehow visualize hashes and hash strength/security. Like NYC subway lines. A few ideas right here:
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	level0.png 
Views:	445 
Size:	47.6 KB 
ID:	2540   Click image for larger version. 

Name:	level1.png 
Views:	386 
Size:	50.6 KB 
ID:	2541   Click image for larger version. 

Name:	level2.png 
Views:	370 
Size:	51.6 KB 
ID:	2542   Click image for larger version. 

Name:	notsecure.png 
Views:	359 
Size:	50.2 KB 
ID:	2543   Click image for larger version. 

Name:	secure.png 
Views:	353 
Size:	51.1 KB 
ID:	2544  


  11. #8
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    CHK v1.30:
    • Removed Column Drag feature, which may lead to a program crash
    • Some GUI changes

    http://compressme.net/


  12. #9
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    Check out the concept art of "string hashing". I realized, this is the feature I'm really missing.
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	text0.png 
Views:	402 
Size:	21.8 KB 
ID:	2546   Click image for larger version. 

Name:	text1.png 
Views:	390 
Size:	26.3 KB 
ID:	2547   Click image for larger version. 

Name:	text2.png 
Views:	400 
Size:	21.9 KB 
ID:	2548  

  13. Thanks:

    Matt Mahoney (8th November 2013)

  14. #10
    Member FatBit's Avatar
    Join Date
    Jan 2012
    Location
    Prague, CZ
    Posts
    190
    Thanks
    0
    Thanked 36 Times in 27 Posts
    Dear Mr. Encode,

    thank you for your program, my recommendations are:

    1. Use non-proportional font - it looks like strings have different length.

    2. From user's point of view - something like program from www.clonespy.com. An ability to compare two directories against each and delete same files.

    3. From user's point of view - something like program from http://www.pepak.net/download/verify...egrity-souboru. Command line interface, many sources etc.

    Sincerely yours,

    FatBit

  15. #11
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    For Command-line interface have a look at: RHASH

    Anyway, most likely, I'll add the Whirlpool hash.
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	Whirlpool.png 
Views:	384 
Size:	21.9 KB 
ID:	2549  

  16. #12
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    Motivated by an extra popularity of my utility, I'm working on a major update. Special thanks to Dutch and Italian folks, by the way!
    And here's the brand new $100 shoes:
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	horizon.png 
Views:	448 
Size:	77.9 KB 
ID:	2550  

  17. Thanks:

    surfersat (13th November 2013)

  18. #13
    Member FatBit's Avatar
    Join Date
    Jan 2012
    Location
    Prague, CZ
    Posts
    190
    Thanks
    0
    Thanked 36 Times in 27 Posts
    Dear Mr. Encode,

    thank you for your suggestion. At least the ability to hide non equal files would be useful for better and faster multiplicity recognition.

    Sincerely yours,

    FatBit

  19. #14
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    The major update will be released soon - now it's under heavy testing!

    What's new in this upcoming release:
    • WHIRLPOOL hash
    • GUI improvements - 32-bit icons, "Refresh" command, new shortcuts
    • Some bug fixes


    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	whirlpool.png 
Views:	375 
Size:	51.0 KB 
ID:	2552  

  20. Thanks:

    surfersat (15th November 2013)

  21. #15
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    Well, looks like the final, standardized SHA3 will be different from what I'm using right now:
    Keccak News



    I think I should remove Keccak512 from the CHK. Next version will have a strong alternative - WHIRLPOOL hash (based on AES-like block cipher)

    Will extensively test BLAKE2 hash once again. The only thing I dislike about BLAKE2 is variety of hash versions (BLAKE2b, BLAKE2s, BLAKE2bp, BLAKE2sp). WinRAR 5 uses BLAKE2sp, other software use other BLAKE2 versions, and I'd like to use BLAKE2b - this is a mess for users who will try to verify hash of a file. On the other hand, BLAKE2 might be ideal for file deduplication.

    Anyway, what I can tell for sure - the SHAKE512 hash will be added after final standardization of SHA3.

    Nice read:
    SHA3: Past, Present and Future

  22. #16
    Member
    Join Date
    Jun 2013
    Location
    USA
    Posts
    98
    Thanks
    4
    Thanked 14 Times in 12 Posts
    The SHA3 thing is a total mess. There is a lot of arguing on the mailing list about the parameters. Personally, I'd leave it alone for the near future.

  23. #17
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    CHK v1.40 update:
    • Added WHIRLPOOL hash
    • Removed SHA3/Keccak512 hash. It's not yet standardized by NIST and the final version will be different from current Keccak512
    • GUI improvements - 32-bit icons, "Refresh" command, new shortcuts
    • Fixed "Open With... chk.exe" program crash

    http://compressme.net


  24. #18
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 795 Times in 488 Posts
    I think you mean http://csrc.nist.gov/groups/ST/hash/...esentation.pdf

    No mention of changes from the current standard. I haven't been following the mailing lists.

  25. #19
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    Please look at my post above

    In short, it will be faster and somewhat less secure (why??)

    c=512 as max (vs c=1024), different padding?, ...

  26. #20
    Member
    Join Date
    Jun 2013
    Location
    USA
    Posts
    98
    Thanks
    4
    Thanked 14 Times in 12 Posts
    The reason is probably because it's really slow in software, 2x slower than SHA256/512 with no major improvement in security. It may have been motivated by the fact that BLAKE as well as BLAKE2 are faster than SHA2 while providing higher security claims.

  27. #21
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 795 Times in 488 Posts
    > Please look at my post above

    I mean the link doesn't work. I got: Not found: The requested URL /groups/ST/hash/sha-3/.../kelsey_ches2013_presentation.pdf was not found on this server.
    (forum software bug?)

    > c=512 as max (vs c=1024),

    2^(c/2) = 256 bit security. Why would you ever need more than that?

    I think the reason for choosing Keccak was a completely different architecture that would be immune to a general break of MD style hashes. It would be good to have but I don't think people will use it as long as it is slower and SHA-2 is still secure. Maybe BLAKE2 will become a de-facto standard because of its speed, but it really hasn't gotten the same degree of attention from analysts as Keccak, yet.

    > different padding?

    Don't know. I can't see a good reason to change it.

  28. #22
    Member
    Join Date
    Jun 2013
    Location
    USA
    Posts
    98
    Thanks
    4
    Thanked 14 Times in 12 Posts
    I think the reason for choosing Keccak was a completely different architecture that would be immune to a general break of MD style hashes. It would be good to have but I don't think people will use it as long as it is slower and SHA-2 is still secure. Maybe BLAKE2 will become a de-facto standard because of its speed, but it really hasn't gotten the same degree of attention from analysts as Keccak, yet.


    The first part makes sense in theory. In practice maybe not.

    As for BLAKE2, it's based on BLAKE which received roughly the same amount of cryptanalysis during the SHA3 competition if not more. Many including myself thought that BLAKE would win :\.

    The main difference with BLAKE2 is the simplified permutation which brings it in line with the one used in the ChaCha20 stream cipher. Both are probably very secure.

    One other reason that Keccak was selected was probably because NIST has a history of choosing algorithms which are very fast in hardware and not so much in software(DES and AES being two examples).

  29. #23
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    Anyway, here's a screenshot from upcoming and final (of this year) version of CHK - v1.50, featuring RIPEMD160 and sexy, quite expensive icons from IconFactory (Designers of XBOX 360 and PKZIP Desktop interface):
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	chk150.png 
Views:	389 
Size:	55.6 KB 
ID:	2555  

  30. #24
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    The comparison. Please tell me what you think!
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	1.png 
Views:	397 
Size:	124.7 KB 
ID:	2556  

  31. #25
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,257
    Thanks
    307
    Thanked 795 Times in 488 Posts
    I dunno. I prefer a command line interface

  32. Thanks:

    encode (20th November 2013)

  33. #26
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    Soon to be released...
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	verify.png 
Views:	368 
Size:	57.2 KB 
ID:	2557  

  34. #27
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    What's new in CHK v1.50:
    • Added RIPEMD160
    • New icons, some GUI improvements & redesign

    http://compressme.net

    Enjoy new release!


  35. #28
    Member
    Join Date
    Jan 2014
    Location
    Russia
    Posts
    24
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Please, implement blake2sp in your CHK. It is already implemented in original b2sum from blake2.net, as well as in my rsc32 here:
    https://www.livebusinesschat.com/smf...p?topic=5504.0
    But these command-line only, so CHK with fine GUI is highly welcome.

    Although one thinks that blake2b to be faster,
    1) it produces crazy long output of 512-bits, rather for machines than for men;
    2) it does not fit to sse2, because 64*4=256, maybe for AVX only
    Last edited by persicum; 3rd February 2014 at 09:42. Reason: rsc32 update

  36. #29
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    4,010
    Thanks
    399
    Thanked 398 Times in 152 Posts
    The BLAKE2 is in my TODO list actually. The only reason I didn't added it right away is possible confusion about hash names - BKAKE2b,BKAKE2bp,BLAKE2s,BLAKE2sp produce different output. WinRAR calls it simply BLAKE2, other software uses different versions of that hash algorithm... I'd like to call it simply BLAKE2 too, but this may introduce some confusion. BTW, due to the same reason, I've removed SHA-3/Keccak512 from CHK as well. You see, SHA1 is SHA1 by any means. At that point, adding hashes like PGP signatures might be more useful. So anyways, I'll wait for some time to figure out what's the best and right direction for my software!

  37. #30
    Member
    Join Date
    Jan 2014
    Location
    Russia
    Posts
    24
    Thanks
    0
    Thanked 2 Times in 2 Posts
    The problem with different BLAKES is even worse. -))) All these four variants support arbitrary length output. So, for instance, there is not only 32-byte blake2sp, but also 1-byte,2-byte, … 16-byte… until full length 32-byte. My RSC32 supports 256-bit output as well as 128-bit output and they are different from each other…
    So you should not only implement four different BLAKES, but also introduce a field for desirable output length.
    The other issue is although BLAKERS propose their BLAKES as “even faster than MD5”, AVX is required to achieve this speed for 64-bit, or at least SSE4.1;

    BTW, I found that CHK is quite slow on my i5 WinXP32. What environment do you use to create it?
    I prefer an old fashion inline shit-asm, which is typically 30% slower than a code produced by modern optimizing C++ compilers. So my RSC32 sometime noticeably slower than RHash binaries.
    Unfortunately, I found with a wall clock that CHK is even slower –(((. One Gig file was in RAM cache.
    CHK
    CRC32 -1s
    MD5 – 8 s
    SHA1 – 10 s
    SHA256 – 17 s

    RSC32
    CRC32 – 1s
    MD5 – 3s
    SHA1 – 3s
    B2s – 3s
    B2sp – 2s
    Tiger – 5s
    SHA256 – 11s

    RHash
    MD5 – 2s
    SHA1 – 7s
    Tiger – 9s
    SHA256 – 7s
    SHA3-256 - 35s



    Unfortunately, I could not implement b2sp to use all my cores well – it is just slightly faster than ordinary b2s, in spite of my attempts to synchronize nodes. Because it seems you don’t like just recompile ready-to-use source codes, it is interesting how you implement b2sp
    Last edited by persicum; 1st February 2014 at 19:29. Reason: RHash bench

Page 1 of 8 123 ... LastLast

Similar Threads

  1. pcompress, a deduplication/compression utility
    By moinakg in forum Data Compression
    Replies: 160
    Last Post: 1st July 2020, 00:40
  2. Hash / Checksum algorithm considerations
    By Cyan in forum Data Compression
    Replies: 61
    Last Post: 16th June 2017, 00:28
  3. CHK v1.10 is here!
    By encode in forum Data Compression
    Replies: 21
    Last Post: 30th December 2012, 18:20
  4. CHK v1.03 is here!
    By encode in forum Data Compression
    Replies: 58
    Last Post: 2nd December 2012, 00:00
  5. Remote diff utility
    By Shelwien in forum Data Compression
    Replies: 2
    Last Post: 6th September 2009, 15:37

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •