Results 1 to 9 of 9

Thread: Data decompression on in-memory kernel

  1. #1
    Member
    Join Date
    Jan 2009
    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Data decompression on in-memory kernel

    Hello all,

    I am writing my own kernel and was wondering if I could be guided to a compression algorithm that does a great job for decompressing a kernel image in-memory. Compression time/memory usage is not so important as the decompression components. I suppose what I would be looking for is an algorithm that is high speed and low memory overhead yet still meaningfully compressed. I apologize if I am too vague or if I should not have posted this question here.

    Regards,
    Creg

  2. #2
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by cregd View Post
    Hello all,

    I am writing my own kernel and was wondering if I could be guided to a compression algorithm that does a great job for decompressing a kernel image in-memory. Compression time/memory usage is not so important as the decompression components. I suppose what I would be looking for is an algorithm that is high speed and low memory overhead yet still meaningfully compressed. I apologize if I am too vague or if I should not have posted this question here.

    Regards,
    Creg
    LZSS by encode (not open source yet), LZO, LZOP (not open source).
    Also Tornado 7/8/9 c1/c2/c3 has very efficient decompression, but memory requirements are higher.

  3. #3
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,348
    Thanks
    212
    Thanked 1,012 Times in 537 Posts
    Its probably more a matter of code preprocessing than compression algorithm.
    There're much better things than E8.

  4. #4
    Member
    Join Date
    Jan 2009
    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts
    m^2: Thanks for your reply. Having source code is a must for this to work nicely. LZSS and Tornado look pretty cool, it would be nice if I could take a look at the source in LZSS so that I could see what the implications would be on my code base.

    Shelwien: I am sorry but I am quite unskilled in this area, can you please elaborate?

  5. #5
    Member
    Join Date
    May 2008
    Location
    Germany
    Posts
    410
    Thanks
    37
    Thanked 60 Times in 37 Posts

    @m^2 "LZOP is opensource"

    http://www.oberhumer.com/opensource/lzo/

    30 Apr 2008:

    LZO 2.03 has been released,
    featuring major speedups for 64-bit architectures like AMD64,
    minor overall speedups, portability ...

    http://www.lzop.org/

    sadly to say

    the last official version of LZOP is Version 1.02 rc1 from 25 Jul 2005

    for the actual LZO 2.03 (30 Apr 200
    there is actually not a complementary LZOP - binary

  6. #6
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,982
    Thanks
    377
    Thanked 351 Times in 139 Posts
    Quote Originally Posted by cregd View Post
    it would be nice if I could take a look at the source in LZSS so that I could see what the implications would be on my code base.
    The source is available now:
    http://encode.su/forum/attachment.ph...3&d=1233048710


  7. #7

  8. #8
    Programmer toffer's Avatar
    Join Date
    May 2008
    Location
    Erfurt, Germany
    Posts
    587
    Thanks
    0
    Thanked 0 Times in 0 Posts
    I can be wrong - but i've actually never seen an E8E9 applied to kernel images - only standard gzip and maybe bzip.

    E8E9 is a transformation for x86 machine code, which converts relative jumps/calls addresses to absolute. Imagine on at offset 1000 there's a relative jump with offset +200 and at offset 1500 there's a relative jump to -300. These relative addresses are different, but encode the same location. This way you translate them to absolute offsets: 1200. Hence there are more similar patterns and it improves compression.

    Maybe one could apply simple statistical compression? I don't know if it is convenient, since you need additional memory and some more CPU time. I don't think that CPU power is a problem nowdays.

  9. #9
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,611
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by joerg View Post
    http://www.oberhumer.com/opensource/lzo/

    30 Apr 2008:

    LZO 2.03 has been released,
    featuring major speedups for 64-bit architectures like AMD64,
    minor overall speedups, portability ...

    http://www.lzop.org/

    sadly to say

    the last official version of LZOP is Version 1.02 rc1 from 25 Jul 2005

    for the actual LZO 2.03 (30 Apr 200
    there is actually not a complementary LZOP - binary
    Thanks for correction, it's a nice news for me.

Similar Threads

  1. CLI memory teste needed
    By SvenBent in forum The Off-Topic Lounge
    Replies: 7
    Last Post: 21st April 2010, 09:06
  2. Memory Limit?
    By Earl Colby Pottinger in forum Data Compression
    Replies: 28
    Last Post: 11th April 2010, 04:59
  3. IBM Active Memory Expansion
    By m^2 in forum Data Compression
    Replies: 0
    Last Post: 17th February 2010, 01:15
  4. 2G+ memory blocks
    By Shelwien in forum Data Compression
    Replies: 0
    Last Post: 6th March 2009, 03:13
  5. Nanozip decompression data troubles
    By SvenBent in forum Data Compression
    Replies: 11
    Last Post: 12th January 2009, 23:25

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •