Results 1 to 20 of 20

Thread: e8c posts from GDC discussion

  1. #1
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts

    e8c posts from GDC discussion

    T2, Rapid.
    Modified IZ.
    Сам доделать не осилю.
    Attached Files Attached Files

  2. #2
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts
    Fix (in attachment).
    ​Обидно, "уплыл косарь".
    Attached Files Attached Files
    • File Type: c x.c (8.8 KB, 254 views)

  3. #3
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts
    ​Несколько слов о тайм-лимитах для T2.

    Мне нравится, как команда WebP определяет "скоростные режимы": не "fast" и "slow", а "capture mode" и "delivery/storage mode".


    Предположим, необходим кодек для покадрового lossless-сжатия видео, захваченного с экрана во время игры. У геймера 8-ядерный проц, из которых только 2 ядра могут быть отданы под real-time сжатие. Видео 1080@60 - это 375 MB/s, то есть минимальная однопоточная производительность кодека = 200 MB/s.


    В то же время, победителем в категории "T2, Rapid" может стать асимметричный кодек с соотношением "enc / dec" == "36 s / 4 s" == "28 MB/s / 250 MB/s", который useless для описанного выше варианта применения.

  4. #4
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts
    Submission to T2, compiled executable only.
    Attached Files Attached Files

  5. #5
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts
    .
    Attached Files Attached Files
    Dvizh must go Dvizh, no open source - no Dvizh.

  6. #6
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts
    Multithreaded GROOT_FORCE, linear scalability.
    Attached Files Attached Files
    Last edited by e8c; 20th October 2020 at 12:03. Reason: added tiles.c
    Dvizh must go Dvizh, no open source - no Dvizh.

  7. #7
    Member
    Join Date
    Aug 2014
    Location
    Argentina
    Posts
    548
    Thanks
    240
    Thanked 94 Times in 74 Posts
    All of these progs seem overly-tuned to the sample file. They work remarkably well on it but not so much on pretty much any other PNM...

  8. #8
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts
    Algorithm Errors as Art:

    https://app.box.com/s/gtm698mi8ns8adv62i1s57wgcuj0lab1
    Attached Files Attached Files
    Last edited by e8c; 25th October 2020 at 23:45. Reason: added __int128.c
    Dvizh must go Dvizh, no open source - no Dvizh.

  9. #9
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts
    Encoder selects predictor using statistical inference.
    Attached Files Attached Files
    Dvizh must go Dvizh, no open source - no Dvizh.

  10. #10
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts

    Test using Termux


    https://wiki.termux.com/wiki/Interna...ternal_storage

    Code:
    $ ./build.sh
    ...
    $ ls -l
    total 133232
    -rwx------ 1 u0_a219 u0_a219       237 Nov 13 21:37 build.sh
    -rwx------ 1 u0_a219 u0_a219     18288 Nov 13 21:39 guess_1
    -rwx------ 1 u0_a219 u0_a219     18288 Nov 13 21:39 guess_2
    -rwx------ 1 u0_a219 u0_a219     18288 Nov 13 21:39 guess_4
    -rw------- 1 u0_a219 u0_a219     21018 Nov 13 21:36 guess_with_timer.c
    -rwx------ 1 u0_a219 u0_a219       294 Nov 13 21:36 run.sh
    drwx------ 2 u0_a219 u0_a219      4096 Nov 12 13:35 storage
    -rw------- 1 u0_a219 u0_a219 136323089 Nov 12 14:12 uncompressed.ppm
    SM-T820 (APQ8096, 2x Kryo HP 2.15 GHz + 2x Kryo LP 1.6 GHz):

    Code:
    $ ./run.sh
    encode,   1 thread :    29 MPx/s
    decode,   1 thread :    33 MPx/s
    encode,   2 threads:    59 MPx/s
    decode,   2 threads:    67 MPx/s
    encode,   4 threads:    81 MPx/s
    decode,   4 threads:    98 MPx/s
    ​Планов развивать кодек у меня нет. Вероятно, это последнее сообщение в данной теме. Лицензии на представленный здесь код не будет (и передачи его в "общественную собственность" тоже).
    Attached Files Attached Files
    Last edited by e8c; 23rd November 2020 at 21:32. Reason: finalization
    Dvizh must go Dvizh, no open source - no Dvizh.

  11. #11
    Member
    Join Date
    Apr 2015
    Location
    Greece
    Posts
    124
    Thanks
    42
    Thanked 33 Times in 22 Posts
    e8c you are probably the most open person in gdcc . I will probably open source agiannis_image after gdcc.

  12. #12
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts
    Quote Originally Posted by algorithm View Post
    I will probably open source agiannis_image
    Try [to do] it in RFC form with title "Adding a new compression method to PNG": JPEG XL is too slow, WebP 2 limits the image size, both cannot completely replace PNG.
    Dvizh must go Dvizh, no open source - no Dvizh.

  13. #13
    Member
    Join Date
    Apr 2015
    Location
    Greece
    Posts
    124
    Thanks
    42
    Thanked 33 Times in 22 Posts
    Indeed e8c. Judging JPEG XL lossless from gdcc results, it is bad. agiannis_image is a very simple codec. It is strange that Alex Rhatushnyak helped in JPEG XL, yet JPEG XL is much worse than QLIC2. I don't understand what's the point of gaining few % while being many times slower.

  14. #14
    Member
    Join Date
    Jun 2015
    Location
    Switzerland
    Posts
    967
    Thanks
    264
    Thanked 347 Times in 219 Posts
    Quote Originally Posted by algorithm View Post
    Indeed e8c. Judging JPEG XL lossless from gdcc results, it is bad. agiannis_image is a very simple codec. It is strange that Alex Rhatushnyak helped in JPEG XL, yet JPEG XL is much worse than QLIC2. I don't understand what's the point of gaining few % while being many times slower.
    JPEG XL uses decision trees for context modeling. When we get a bit further with the freeze, we can change decision trees to be of a form that is easy to represent as LUTs and speed them up substantially. Also, the predictors of the lossless coding are computed inefficiently at the moment. I'm expecting a ~3x speed up in decoding for lossless.

  15. #15
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts

    Artificial Image

    guess ~ 0.9 MB
    Kvick ~ 1.1 MB
    agiannis_image ~ 1.7 MB
    pglz ~ 2.5 MB
    Attached Files Attached Files
    Dvizh must go Dvizh, no open source - no Dvizh.

  16. #16
    Member
    Join Date
    Feb 2020
    Location
    Area 51
    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by e8c View Post
    guess ~ 0.9 MB
    Kvick ~ 1.1 MB
    agiannis_image ~ 1.7 MB
    pglz ~ 2.5 MB
    150kb with decode rate ~150mb/s at single core of modern cpu
    Guess who?

  17. #17
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    251
    Thanks
    49
    Thanked 107 Times in 54 Posts
    152868 bytes -- BMF 2.01 -s -q9 -- it is essentially a collection of codecs, with smart detection & switching, per-image afaiu
    197737 bytes -- JPEG-XL 0.0.1-6b5144cb -q 100 and the default -s 7
    160018 bytes -- same as above, but with -s 9

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  18. #18
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts
    https://docs.google.com/spreadsheets...#gid=598115372

    CLIC: 970'720'188 -133 MB

    All other subsets:

    Code:
    Art             604'043'656     -13 MB
    Aw              739'756'548     -69 MB
    ComixArt        744'836'572     +41 MB
    FanArt          558'182'420     -27 MB
    Fractals      1'428'366'392     +55 MB
    Games         2'445'693'860     -74 MB
    ITAP          1'343'411'024     -78 MB
    LowPoly         781'223'752      -9 MB
    LPCB          1'272'103'344    -128 MB
    Manga           640'241'792    +171 MB
    PixelArt        166'549'012    +106 MB
    Pixiv           509'755'764     -65 MB
    SpecArt         886'676'696     -10 MB
    ​Total: 13'091'561'020 (w/o transparency layer)
    Better than 2 entries on the diagram: https://encode.su/threads/3544-JXL-r...ll=1#post67989
    Attached Files Attached Files
    Last edited by e8c; 15th January 2021 at 06:11. Reason: tested agiannis_image ("a_run.tar.xz")
    Dvizh must go Dvizh, no open source - no Dvizh.

  19. #19
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts

    About lossy JPG for screenshots in Android 8+ on Samsung devices

    Click image for larger version. 

Name:	Vatolin_met_Kadyrov.jpg 
Views:	21 
Size:	105.8 KB 
ID:	8242
    Code:
    $ ls -l
    ​total 404
    -rw-rw-r-- 1 abc abc 166924 янв  5 01:08 Vatolin_met_Kadyrov.guess
    -rwxrwx--- 1 abc abc 108289 янв  5 01:02 Vatolin_met_Kadyrov.jpg
    -rw-rw-r-- 1 abc abc 133429 янв  5 01:07 Vatolin_met_Kadyrov.png

    There is only one reason why swapping PNGs for JPGs makes sense: PNG is slow.
    I remember that saving 3 MPx screenshots took more than 1 second - it's uncomfortable: instead of interacting with the screenshot, I had to observe a message about incomplete saving the image.
    I think that if "guess" becomes part of the PNG, there will be no need to use the lossy format.


    Looking at the listing above, it is worth remembering that ".jpg" is source: ".png" and ".guess" store jpg-compression artifacts, without these artifacts compression could probably be better.
    Dvizh must go Dvizh, no open source - no Dvizh.

  20. #20
    Member
    Join Date
    Jul 2018
    Location
    Russia
    Posts
    28
    Thanks
    0
    Thanked 3 Times in 2 Posts

    First step in development of MODE_2

    ​Special processing if tile completely grayscale, and same algorithm as with "-1" if not.

    Code:
    $ ./gray -2 Bennu_Grayscale.ppm Bennu_Grayscale.gray
    encode,   2 threads:   118 MPx/s
    $ ./gray -d Bennu_Grayscale.gray tmp.ppm
    decode,   2 threads:   293 MPx/s
    $ ls -l
    -rwxrwx--- 1 root vboxsf  101907455 янв 17 00:08 Bennu_Grayscale.emma
    -rwxrwx--- 1 root vboxsf  126415072 янв 18 21:37 Bennu_Grayscale.gray
    -rwxrwx--- 1 root vboxsf  105285138 янв 16 23:46 Bennu_Grayscale.lea
    -rwxrwx--- 1 root vboxsf  128446880 янв 16 19:26 Bennu_Grayscale.png
    -rwxrwx--- 1 root vboxsf  732538945 янв 16 19:24 Bennu_Grayscale.ppm
    New total for MCIS (Moon Citizen's Image Set) v1.20: 12'745'684'464
    Attached Files Attached Files
    Dvizh must go Dvizh, no open source - no Dvizh.

Similar Threads

  1. GDC Competition: Discussions
    By Ms1 in forum Data Compression
    Replies: 146
    Last Post: 2nd January 2021, 16:47
  2. GDC Competition: Notices
    By Ms1 in forum Data Compression
    Replies: 15
    Last Post: 15th December 2020, 19:58
  3. C as default for data compression?; Disturbing deletion of posts here
    By JamesWasil in forum The Off-Topic Lounge
    Replies: 18
    Last Post: 13th February 2019, 22:56
  4. Forum BUG: Posts don't have newlines when using NoScript
    By m^2 in forum The Off-Topic Lounge
    Replies: 6
    Last Post: 24th January 2012, 21:18
  5. RSS feeds to track new posts
    By schngrg in forum The Off-Topic Lounge
    Replies: 0
    Last Post: 28th June 2011, 18:27

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •