Results 1 to 22 of 22

Thread: brunsli jpeg recompressor

  1. #1
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    846
    Thanks
    242
    Thanked 309 Times in 184 Posts
    Quote Originally Posted by Shelwien View Post
    Lepton is probably fastest atm (with MT and AVX).
    https://github.com/google/brunsli is about 2.5x faster than lepton (single-threaded), but compresses 1 % worse.

  2. #2
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,942
    Thanks
    291
    Thanked 1,286 Times in 728 Posts
    Here I patched it to allow output filename and compiled for windows: http://nishi.dreamhosters.com/u/brunsli_v0.rar
    Seems to work.

    842468 A10.jpg
    680816 brunsli
    688273 lepton
    672236 lepton-slow

  3. #3
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,942
    Thanks
    291
    Thanked 1,286 Times in 728 Posts
    http://nishi.dreamhosters.com/u/brunsli_v1.rar
    - Added patches to compile with IntelC/windows
    - Merged cbrunsli and dbrunsli to a single program
    - Added stdin/stdout support

    Usage:
    Code:
     brunsli c A10.jpg A10.brn
     brunsli d A10.brn A10_unp.jpg
     cat A10.jpg | brunsli c - - | cat >A10.brn
     cat A10.brn | brunsli d - - | cat >A10_unp.jpg
    Added brunsli to 7zdll/.pa format. Compression is both faster and better than normal lepton (still worse than lepton2 aka lepton-slow),
    but decompression is somehow a little slower.

    Code:
      0.172s:  7z.exe a -ms=off -m0=lepton 1.pa a10.jpg
      0.063s:  7z.exe t 1.pa
    
      0.110s:  7z.exe a -ms=off -m0=brunsli 1.pa a10.jpg
      0.094s:  7z.exe t 1.pa

  4. Thanks (2):

    Bulat Ziganshin (8th June 2019),hunman (7th June 2019)

  5. #4
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    846
    Thanks
    242
    Thanked 309 Times in 184 Posts
    Quote Originally Posted by Shelwien View Post
    but decompression is somehow a little slower.
    Perhaps multithreading?

  6. Thanks:

    Shelwien (10th June 2019)

  7. #5
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    615
    Thanks
    255
    Thanked 242 Times in 121 Posts
    I just created a branch of brunsli on GitHub that removes the brotli dependency. Apart from the smaller binaries, this has the advantage of not compressing metadata (the only thing brotli is used for in brunsli) which is useful for using brunsli together with other compressors or for deduplication of metadata between JPEGs.

    Here are results for a strange file I found in an eBook that has not much image data (10x20 pixels) but lots of metadata (some ICC color profile):
    Code:
                                        size (bytes)        encoding time (ms)  notes
      original                          566,486
      Precomp 0.4.7 -cn                 486,517             2000                here we can see that packJPG compresses metadata, too
      Precomp -t+                       298,250             400                 not using packJPG helps for this file
      brunsli:                          363,960             200                 better than packJPG in both speed and size...
      brunsli + Precomp                 364,073             300                 ...but nothing left to optimize
      brunsli-nobrotli:                 566,165             60                  only image data is compressed, so not much difference
      brunsli-nobrotli + Precomp:       298,378             500                 metadata compressed by lzma2
    Attached are Visual Studio 2019 64 bit binaries (note: if you need best performance, try to compile with clang) and the mentioned tested JPG file.
    Renamed the binaries to "Xbrunsli-nobrotli.exe" to reduce confusion and conflicts with existing brunsli binaries.
    Attached Images Attached Images  
    Attached Files Attached Files
    Last edited by schnaader; 11th October 2019 at 17:01.
    http://schnaader.info
    Damn kids. They're all alike.

  8. Thanks (8):

    Bulat Ziganshin (11th October 2019),comp1 (11th October 2019),Gonzalo (14th October 2019),Jyrki Alakuijala (10th October 2019),maadjordan (12th October 2019),Mike (10th October 2019),Shelwien (10th October 2019),Stephan Busch (10th October 2019)

  9. #6
    Member
    Join Date
    May 2008
    Location
    Kuwait
    Posts
    353
    Thanks
    37
    Thanked 38 Times in 23 Posts
    I tried 7-zip with mfilter and the result is much less (2,667 bytes)

  10. #7
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,942
    Thanks
    291
    Thanked 1,286 Times in 728 Posts
    mfilter can recompress jpeg thumbnails, maybe because of that?

  11. #8
    Member
    Join Date
    Aug 2016
    Location
    Russia
    Posts
    108
    Thanks
    6
    Thanked 70 Times in 37 Posts
    MFilter uses the following additional optimizations for jcaron.jpg:

    1) Compresses jpeg thumbnail in Exif segment
    2) Compresses jpeg thumbnail in Photoshop segment
    3) Deletes "well known" ICC profile. MFilter knows several well known ICC profiles and can delete and restore such ICC profiles on the fly.

  12. Thanks (5):

    Bulat Ziganshin (12th October 2019),Gonzalo (14th October 2019),maadjordan (13th October 2019),schnaader (12th October 2019),Shelwien (12th October 2019)

  13. #9
    Member
    Join Date
    May 2008
    Location
    Kuwait
    Posts
    353
    Thanks
    37
    Thanked 38 Times in 23 Posts
    item #3 is like Ecm did with dumped cds which could explain why Mfilter size is large and this should get benefit with compressing few jpg files and lose gain on compressing tens of jpg files.

  14. #10
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    615
    Thanks
    255
    Thanked 242 Times in 121 Posts
    Nice trick! Can also be confirmed using SREP:

    Code:
    MFilter7z.64.dll.srep	1,200,440  // srep MFilter7z.64.dll
    jcaron.jpg.srep		  536,446  // srep jcaron.jpg
    MFilter_jcaron.dat.srep 1,215,822  // copy /b MFilter7z.64.dll + jcaron.jpg MFilter_jcaron.dat
    After searching a bit, I found a site from Adobe with download links for their typical ICC profiles and together with a string from jcaron.jpg ("U.S. Web Coated (SWOP) v2"), the specific profile can be found:

    Code:
    USWebCoatedSWOP.icc       557,168
    USWebCoatedSWOP.icc.srep  531,192
    USWeb_jcaron.dat.srep     541,982
    By the way, the mentioned thumbnail recompression would also be possible with the modified brunsli version:

    Code:
    cover.jpg                   201,988 // file with 2 thumbnails
    cover.jpg.brn               152,636 // unmodified brunsli treats thumbnails as metadata...
    cover.jpg.brn.pcf           152,665 // ...so precomp finds nothing afterwards
    cover.jpg.brn-nobrot        163,750 // modified brunsli-nobrotli
    cover.jpg.brn-nobrot.pcf_cn 162,093 // precomp -cn => 2/2 JPG streams (the thumbnails)
    cover.jpg.brn-nobrot.pcf    152,653 // but it doesn't really help compared to unmodified brunsli on this file
    The thumbnail streams are small compared to the whole file (5,157 bytes each) and completely identical, so the second one gets deduplicated by unmodified brunsli as well as by lzma2 on the modified brunsli.

    Unfortunately, brunsli has no streaming support (processes whole JPEGs only), so you can't apply it a second time to recompress thumbnails. Another modified version would be needed that detects and processes thumbnails in the metadata.
    http://schnaader.info
    Damn kids. They're all alike.

  15. Thanks (2):

    maadjordan (13th October 2019),Stephan Busch (13th October 2019)

  16. #11
    Member
    Join Date
    May 2008
    Location
    Kuwait
    Posts
    353
    Thanks
    37
    Thanked 38 Times in 23 Posts
    as always Shnaader, you keep amazing me.

  17. #12
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    846
    Thanks
    242
    Thanked 309 Times in 184 Posts
    Quote Originally Posted by schnaader View Post
    ... brunsli has no streaming support ...
    Brunsli the lib, no.

    Brunsli the file format, yes. Brunsli has a fixed two way progressive layout. 8x8 subsampled whole image first, followed by the sequential AC.

  18. #13
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    615
    Thanks
    255
    Thanked 242 Times in 121 Posts
    Quote Originally Posted by Jyrki Alakuijala View Post
    Brunsli the lib, no.

    Brunsli the file format, yes. Brunsli has a fixed two way progressive layout. 8x8 subsampled whole image first, followed by the sequential AC.
    Sorry, "streaming support" was an unclear term, here. I meant the capability to detect JPG streams inside other files (so, "embedded support" would be a better term), e.g. JPG in RAW files, game containers or .tar files. Or, as in the case above, a file processed by brunsli-nobrotli that has the main image data compressed, but two JPG streams for the thumbnails left that are "embedded" in the .brn-nobrot file (part of the copied metadata).
    http://schnaader.info
    Damn kids. They're all alike.

  19. #14
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,942
    Thanks
    291
    Thanked 1,286 Times in 728 Posts
    http://nishi.dreamhosters.com/u/brunsli_v2a_dll.rar

    Applied schnaader's patch to my dll kit... can be replaced in mfilter (smaller dll because of removed brotli).
    Doesn't affect compressed size though, I guess mfilter doesn't feed metainfo to brunsli at all.

  20. Thanks:

    Mike (1st December 2019)

  21. #15
    Member
    Join Date
    May 2008
    Location
    Kuwait
    Posts
    353
    Thanks
    37
    Thanked 38 Times in 23 Posts
    but yours lose some compression but can unpack both safely.

  22. #16
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,942
    Thanks
    291
    Thanked 1,286 Times in 728 Posts
    That's the idea... Brunsli normally uses brotli library to compress jpeg metainfo, while this version leaves metainfo uncompressed.
    In theory this can improve solid compression of recompressed jpegs.

  23. #17
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    846
    Thanks
    242
    Thanked 309 Times in 184 Posts
    Brunsli is currently discussed on hacker news: https://news.ycombinator.com/item?id=22456764

  24. Thanks:

    Bulat Ziganshin (2nd March 2020)

  25. #18
    Member
    Join Date
    Mar 2012
    Location
    Paris
    Posts
    38
    Thanks
    11
    Thanked 3 Times in 3 Posts
    Is there a standalone Windows x64 binary of Brunsli I could use to archive thousands of jpeg?

    It can be done through Imagemagick too, but "Requires the brunsli delegate library." Where the library can be found?

    Thank you for your help

  26. #19
    Member
    Join Date
    May 2009
    Location
    France
    Posts
    99
    Thanks
    13
    Thanked 75 Times in 45 Posts

  27. Thanks:

    jimbow (19th March 2020)

  28. #20
    Member
    Join Date
    Mar 2012
    Location
    Paris
    Posts
    38
    Thanks
    11
    Thanked 3 Times in 3 Posts
    Thank you, I managed to build it in VS2019, although the configuration is x64-Debug, how can I switch to release x64?

  29. #21
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    615
    Thanks
    255
    Thanked 242 Times in 121 Posts
    Note that https://ci.appveyor.com/project/eust...uild/artifacts also contains so-called "Build artifacts" (in this case executables and libraries), so there should be no need to build yourself (unless you are curious or want to modify something).
    http://schnaader.info
    Damn kids. They're all alike.

  30. Thanks:

    jimbow (19th March 2020)

  31. #22
    Member
    Join Date
    Mar 2012
    Location
    Paris
    Posts
    38
    Thanks
    11
    Thanked 3 Times in 3 Posts
    Is there any viewer for brunsli encoded files? (doing in-RAM decompression)

  32. Thanks:

    necros (Yesterday)

Similar Threads

  1. JPEG issues a draft call for a JPEG reference software
    By thorfdbg in forum Data Compression
    Replies: 11
    Last Post: 19th April 2017, 16:18
  2. lzma recompressor
    By Shelwien in forum Data Compression
    Replies: 33
    Last Post: 25th February 2016, 22:40
  3. JPEG XT Demo software available on jpeg.org
    By thorfdbg in forum Data Compression
    Replies: 40
    Last Post: 16th September 2015, 15:30
  4. Replies: 9
    Last Post: 11th June 2015, 23:28
  5. Replies: 0
    Last Post: 6th February 2015, 05:57

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •