Page 3 of 8 FirstFirst 12345 ... LastLast
Results 61 to 90 of 215

Thread: CHK Hash Tool

  1. #61
    Member
    Join Date
    Mar 2008
    Location
    Minsk
    Posts
    9
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Got it. So I think that some kind of progress indicator is too obvious feature to request it too.)
    Last edited by nemoW; 18th June 2014 at 00:50.

  2. #62
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Yep! Take into account that CHK is not a commercial project - it's a tool developed in spare time. No spare time = no development...

  3. #63
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Quote Originally Posted by Mangix View Post
    interesting note:

    I ran MinGW's strip utility against the 64-bit chk.exe and reduced the size.

    Original: 8,671,602 bytes

    Stripped: 6,760,960 bytes

    Figured I'd post this since this is a forum about making files smaller .
    BTW, can you tell me the exact command-line parameters and MinGW version number?

  4. #64
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    The upcoming CHK v1.70:
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	chk170.png 
Views:	293 
Size:	67.5 KB 
ID:	3036  

  5. #65
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    CHK v1.70 is here: compressme.net

    What's new:
    • Added Tiger hash (No TTH this time)
    • Now CHK keeps all settings, including window size and width of each column
    • Smaller 64-bit executable (Big thanks to Mangix)
    • Fixed bug with empty Typename for some weird system files
    • Changed output format (Removed '*' character before filename)
    • Some code optimizations


  6. #66
    Member
    Join Date
    Feb 2013
    Location
    San Diego
    Posts
    1,057
    Thanks
    54
    Thanked 71 Times in 55 Posts
    Some suggestions:

    1. Display multiple kinds of hashes at the same time
    2. Update columns incrementally while computation is in progress (rather than display a blank window)
    3. Eliminate or reduce flicker when updating the status bar
    4. Make menus responsive during long computations
    5. Provide a clean way to terminate while computations are in progress (in case the user inadvertently adds more files than they really care about and gets stuck in an overly-long computation)

    Most are really just applying polish to the UI for the sake of aesthetics (at least as far as I care). #1, however, would increase the basic utility and I'd consider it highest priority (it also carries some risk of overcomplicating the UI, though).

    #3 should be low-hanging fruit.

    #5 is part of #4 and if you don't want to go all the way to #4, at least take care of #5.

    #2 will reduce the time to display useful information and should be fairly well worth the effort.

    Extra credit:

    6. Provide richer export capabilities: e.g. csv/tsv
    7. Integrate with the right-click menu in Windows 7 (and later)
    8. Utilize multiple cores/cpus
    9. Make a Chrome application

    Sorry if some of these have been discussed previously. I didn't read the whole thread before posting.

    As a point of reference, I'm using the GNU coreutils versions of these utilities. They have CLIs and their interfaces are pretty spare, but along with that come some benefits that are basically free. One is that the outputs of previous runs don't disappear, because you can easily scroll up in the terminal and see the entire log of what you've done. As a *nix user, this is the sort of feature that seems extremely basic. You might want to familiarize yourself with using these tools just for the sake of knowing the competition.

    Don't take my critiques the wrong way. That it's a good tool is obvious from the feedback you've already gotten. You've clearly found an unmet need and scratched an itch -- that's a job well done. Consider this a bag of ideas to draw from if you really want to kill it.
    Last edited by nburns; 5th November 2014 at 01:01.

  7. Thanks:

    encode (5th November 2014)

  8. #67
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Thanks for your feedback!
    I'm collecting ideas and opinions indeed. As a quick note, before adding something to the CHK, I'm carefully testing a new idea/feature - many stuff I have implemented so far was not added to the program at all. A brilliant idea, at the first sight, may do a serious mischief - making CHK unreadable, too complicated, slow, etc. etc. In other words, I'm desperately avoiding "Feature Creep", which can kill the entire program. Anyway, many useful features was not added due to my lack of spare time only...

  9. #68
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts

    CHK v1.72

    Okay guys, a new version is here:

    http://compressme.net/#chk

    The theme of a new release is "accelerated CRCs"

    CRC16/CRC32 are slice-by-8, CRC64 is slice-by-4 - a highly optimized implementations!

    Plus, extra nail polish like some small bug fixes and code optimizations!

    Enjoy!

  10. #69
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts

    CHK v1.73

    A quick update this time! Now you CAN close the program during file processing!

    http://compressme.net/#chk

  11. #70
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    An upcoming CHK v1.75
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	chk175.png 
Views:	257 
Size:	34.4 KB 
ID:	3325  

  12. #71
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Well, next version will be v1.80 - a major update this time.
    Too many improvements to list, but the main changes are:
    • Added optimized Adler32 checksum
    • Now you can save to SFV format
    • Added "Delete" (Move to Recycle Bin) command
    • Speed optimization of MD4/ED2K, MD5, SHA1/SHA1-Base32, SHA256/SHA224 and SHA512/384. Double checked each line of a code, re-optimized for 64-bit build
    • Massive GUI speedup, especially if you deal with thousands of files
    • Flicker-free status bar
    • Actual file error message instead of a mysterious "?" sign
    • File-count feature
    • Some GUI polish and improvements

    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	chk180.png 
Views:	238 
Size:	34.5 KB 
ID:	3339  

  13. Thanks:

    nburns (9th January 2015)

  14. #72
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    CHK v1.80 is here:
    http://compressme.net/

    Enjoy new release!

  15. #73
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    A brief description of CHKs output

    The latest version of CHK (v1.80) introduces new output formats, so it's time to explain what's new!

    "Copy Hash" command (Ctrl+C)
    Copies a hash into Windows Clipboard in following format:
    <Hash>
    if single file selected, and:
    <Hash> *<FileName>
    <Hash> *<FileName>
    for multiple files. Please note that <filename> contains no path.

    "Save As..." command (Ctrl+S)
    Saves all hashes to a file. All hash files are UTF-8 encoded - which look like standard ANSI text in case of English only filenames and if filenames contain non English characters (e.g. Russian, Chinese filenames), UTF-8 encoding is used instead. As a note not all hash tools may support UTF-8 encoded files, but at least SFV Ninja (GUI) and RHASH (Command-line) fully support CHK's output. All line-breaks are in Windows format - CR+LF. All filenames contain relative path (Relative to a saved hash file).

    Each hash has its own default output format:
    CRC32 - SFV File (*.sfv)
    MD5 - MD5 File (*.md5)
    SHA1 - SHA1 File (*.sha1)
    SHA256 - SHA256 File (*.sha256)
    And Text File (*.txt) for all other hashes.

    As a note, MD5/SHA1/SHA256 files have the same format as Text File, but with different extensions.

    Text File (Compatible with GNU utilities such as MD5SUM, SHA1SUM, etc.):
    ; Hash: <HashName>
    ;
    <Hash> *<FileName>
    <Hash> *<FileName>
    ...
    ; <Error> *<FileName>
    ...
    ; <FileCount> file(s)

    SFV File (Simple File Verification, compatible with QuickSFV):
    <FileName> <Hash>
    <FileName> <Hash>
    ...
    Files with no hashes (Access error) are skipped.

    Comma-Separated Values (You may import these files to e.g. Microsoft Excel):
    "<FileName>",<Size>,<Hash>
    "<FileName>",<Size>,<Hash>
    ...
    All filenames are quoted, since any filename may contain commas.
    Files with no hashes (Access error) are skipped.

    Hope this helps!

  16. #74
    Member
    Join Date
    Dec 2014
    Location
    narnia
    Posts
    2
    Thanks
    0
    Thanked 1 Time in 1 Post
    Thanks encode. This tool is very useful. If I may request a feature: BLAKE2 as an optional hash. It's an extremely fast algorithm. Definitely worth a look.

    https://blake2.net/

    P.S. BLAKE2 =BLAKE2b

  17. Thanks:

    encode (17th January 2015)

  18. #75
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Thanks for your feedback!

    The only problem with BLAKE2 is possible confusion with what kind of exact BLAKE2 variant was used. WinRAR (RAR5) now uses BLAKE2sp. Some checksum tool called the "checksum" uses BLAKE2s. The main build of CHK is 64-bit, everything was (re)optimized for 64-bit compile, so BLAKE2b is perfect fit.
    A few things to consider:
    Is BLAKE2b hash string too long to deal with? As a note SHA256 is by far much more popular than SHA512. Even SHA512/256 exists (truncated SHA512, just as SHA384. BTW, SHA512/256 was implemented, but not included in current CHK releases yet)
    Seems that BLAKE2s is more popular right now. But CHK is from the data compression world, and hence:
    Adler32 (Somewhere in ZLib)
    CRC16 (LHA/LZH)
    CRC32 (ZIP, RAR, others)
    CRC64 (7-Zip, XZ)
    Should I consider BLAKE2sp (used by RAR5) instead?


  19. #76
    Member
    Join Date
    Dec 2014
    Location
    narnia
    Posts
    2
    Thanks
    0
    Thanked 1 Time in 1 Post
    The developers of BLAKE2 state that BLAKE2b is the default algorithm. By my own tests I can confirm that BLAKE2b is significantly faster than BLAKE2s. However, it is definitely worth asking whether the 128 character output is too long for general purposes. The community seems to be leaning towards BLAKE2s.

    In the end, it really depends on the program when deciding which algorithm to use: 64/32 bits, multiple/single core processing, etc.

    Yet, we absolutely need a BLAKE2 standard algorithm regardless of the fragmented implementations throughout this community. If I was a betting man I'd put my money on BLAKE2b, but who knows?

  20. #77
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    So, let's wait and see!

    Meanwhile, a quick update - CHK v1.81:
    • Fixed freezes during processing of large files
    • WinRAR-styled ListView
    • Changed the default hash to SHA256, changed columns default widths

    http://compressme.net/

    Recently found on the web that some users may stop using CHK just because it freezes on large files... Bingo!


  21. Thanks:

    Fallon (19th January 2015)

  22. #78
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts

    CHK v1.82

    • Added progress indicator (% done) for large files (>20MB)
    • Major speed-up of ALL cryptographic hash functions (I've found a bottleneck in my secure hash framework)

    http://compressme.net/


  23. #79
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    A memo for phuture releases
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	hashlist.png 
Views:	228 
Size:	9.5 KB 
ID:	3369  

  24. #80
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    i just tried slicing-by-16 crc32 and it shows good improvement: https://github.com/Bulat-Ziganshin/Crc32

    Code:
    C:\Testing\Crc32>Crc32_vc2013_x64.exe
      8 bytes at once: CRC=00EE2DAA, 0.590s, 1736.712 MB/s
     16 bytes at once: CRC=00EE2DAA, 0.419s, 2441.144 MB/s
    
    C:\Testing\Crc32>Crc32_gcc492_x64.exe
     8 bytes at once: CRC=00EE2DAA, 0.541s, 1892.791 MB/s
    16 bytes at once: CRC=00EE2DAA, 0.303s, 3379.538 MB/s
    2*16 bytes at once: CRC=00EE2DAA, 0.273s, 3750.916 MB/s
    Last edited by Bulat Ziganshin; 26th January 2015 at 03:38.

  25. Thanks:

    encode (26th January 2015)

  26. #81
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Yep, but it also depends on CPUs cache size. So, right now, 8KB lookup-table is pretty adequate. (And hence CRC32 is slice-by-8 and CRC64 is slice-by-4)

  27. #82
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,511
    Thanks
    746
    Thanked 668 Times in 361 Posts
    i can't recall cpus with L1$ < 24 KB. btw, stephan answered me with his run:

    Code:
    GCC 4.7.2 (g++ -mtune=native -O3 Crc32.cpp -o crc32) 64 bit CentOS
    Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
    
    
     88 bytes at once: CRC=00EE2DAA, 0.500s, 2048.000 MB/s
      1 byte  at once: CRC=00EE2DAA, 2.470s, 414.575 MB/s
    +  4 bytes at once: CRC=17560371, 0.950s, 1077.895 MB/s
    +2*4 bytes at once: CRC=17560371, 0.930s, 1101.075 MB/s
    +4*4 bytes at once: CRC=17560371, 0.890s, 1150.562 MB/s
    +  8 bytes at once: CRC=17560371, 0.560s, 1828.571 MB/s
    +2*8 bytes at once: CRC=17560371, 0.480s, 2133.333 MB/s
    +4*8 bytes at once: CRC=17560371, 0.450s, 2275.556 MB/s
      4 bytes at once: CRC=00EE2DAA, 0.960s, 1066.667 MB/s
    2*4 bytes at once: CRC=00EE2DAA, 0.940s, 1089.362 MB/s
    4*4 bytes at once: CRC=00EE2DAA, 0.940s, 1089.362 MB/s
      8 bytes at once: CRC=00EE2DAA, 0.520s, 1969.231 MB/s
     16 bytes at once: CRC=00EE2DAA, 0.380s, 2694.737 MB/s
    2*16 bytes at once: CRC=00EE2DAA, 0.360s, 2844.444 MB/s
    2*8 bytes at once: CRC=00EE2DAA, 0.480s, 2133.333 MB/s
    4*8 bytes at once: CRC=00EE2DAA, 0.480s, 2133.333 MB/s
    chunked        : CRC=00EE2DAA, 0.480s, 2133.333 MB/s
    and my answer, if you are interested:

    if you are interested in even more efficient algorithm, the key is concatenation of CRC values. shortly speaking, you can compute CRC of N+M+K+... bytes by calculating CRC of each block and then combining them. details are briefly described in [1] (see figure 3 and description at pages 5/6) with more math details in crc-doc.1.0.pdf [2]

    [1] http://www.intel.com/content/dam/www...tion-paper.pdf
    [2] https://code.google.com/p/crcutil/
    of course, for you, Ilya, it may be easier to just incorporate the crcutil library
    Last edited by Bulat Ziganshin; 26th January 2015 at 14:36.

  28. Thanks:

    encode (26th January 2015)

  29. #83
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    For testing purposes and evaluation, redesigned the GUI using Visual Studio and .NET Framework. Not really happy with the results anyway...
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	net.png 
Views:	212 
Size:	9.6 KB 
ID:	3374  

  30. #84
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Testing BLAKE2s...
    Code:
    C:\blake2\Release>timer blake2 enwik9
    
    Timer 3.01  Copyright (c) 2002-2003 Igor Pavlov  2003-07-10
    BLAKE2s test-bed by encode
    MD5=E206C3450AC99950DF65BF70EF61A12D
    Elapsed time: 1.718s
    
    Kernel Time  =     0.125 = 00:00:00.125 =   7%
    User Time    =     1.593 = 00:00:01.593 =  91%
    Process Time =     1.718 = 00:00:01.718 =  99%
    Global Time  =     1.734 = 00:00:01.734 = 100%
    
    C:\blake2\Release>timer b2sum32 -a blake2s enwik9
    
    Timer 3.01  Copyright (c) 2002-2003 Igor Pavlov  2003-07-10
    39c71fb47fdb8cbab5e81c2f3830eec9f94f1823481a8540326d2499a605243f enwik9
    
    Kernel Time  =     0.265 = 00:00:00.265 =  16%
    User Time    =     1.359 = 00:00:01.359 =  83%
    Process Time =     1.625 = 00:00:01.625 = 100%
    Global Time  =     1.625 = 00:00:01.625 = 100%
    
    C:\blake2\Release>timer b2sum64 -a blake2s enwik9
    
    Timer 3.01  Copyright (c) 2002-2003 Igor Pavlov  2003-07-10
    39c71fb47fdb8cbab5e81c2f3830eec9f94f1823481a8540326d2499a605243f enwik9
    
    Kernel Time  =     0.109 = 00:00:00.109 =   7%
    User Time    =     1.375 = 00:00:01.375 =  92%
    Process Time =     1.484 = 00:00:01.484 = 100%
    Global Time  =     1.484 = 00:00:01.484 = 100%
    The MD5 code is from CHK Checksum Utility (Compiled using Visual Studio 2013, 32-bit build). Right now, my dummy BLAKE2s implementation takes 4 seconds here...

  31. #85
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    BTW, why not?
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	chk_tw.png 
Views:	223 
Size:	9.8 KB 
ID:	3383  

  32. #86
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    CHK v1.85 "Golden Sample" - the final version in a series of quick updates is here!

    http://compressme.net

    What's new:
    • Added SHA512/256 hash, removed CRC16,SHA224,SHA384 hashes - to save some space for future hash additions and keep the look more cleaner
    • Some GUI-related bug fixes
    • Moved in-code strings to CHK's resources (to "Hint" property of elements, mostly) - to simplify Third-Party localization (Yes, I'm talking to you Chiahong Hong)

    Enjoy new release!

  33. Thanks:

    Fallon (3rd February 2015)

  34. #87
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    For a Chinese version of CHK, have a look at:
    http://www.azofreeware.com/2013/11/chk-150-hash.html
    Special thanks to Chiahong Hong!

  35. #88
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    878
    Thanks
    51
    Thanked 106 Times in 84 Posts
    would it be possible to right click intergrate it into windows like quicksfv ?
    and have one cheskum files with several hashes on the same file(s) ?

  36. #89
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    681
    Thanks
    38
    Thanked 168 Times in 84 Posts
    Maybe this one is what you're looking for.
    http://www.nirsoft.net/utils/hash_my_files.html

  37. #90
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,984
    Thanks
    377
    Thanked 352 Times in 140 Posts
    Well, "Right click" and "Open with..." -> CHK will solve this. Later, will add a feature to add a registry entry for this.
    Several hashes for a single file might be a mess - already tested a few ideas. However, two columns - one with simple checksum like Adler32/CRC32 + a second one with a secure hash might work fine here.

Page 3 of 8 FirstFirst 12345 ... LastLast

Similar Threads

  1. pcompress, a deduplication/compression utility
    By moinakg in forum Data Compression
    Replies: 156
    Last Post: 17th October 2019, 04:11
  2. Hash / Checksum algorithm considerations
    By Cyan in forum Data Compression
    Replies: 61
    Last Post: 16th June 2017, 01:28
  3. CHK v1.10 is here!
    By encode in forum Data Compression
    Replies: 21
    Last Post: 30th December 2012, 19:20
  4. CHK v1.03 is here!
    By encode in forum Data Compression
    Replies: 58
    Last Post: 2nd December 2012, 01:00
  5. Remote diff utility
    By Shelwien in forum Data Compression
    Replies: 2
    Last Post: 6th September 2009, 16:37

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •