Results 1 to 5 of 5

Thread: Unidentified Compression used in 1997 Westwood PC Game "Lands of Lore 2"

  1. #1
    Member
    Join Date
    Nov 2020
    Location
    Germany
    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Question Unidentified Compression used in 1997 Westwood PC Game "Lands of Lore 2"

    Hello Compression Experts,

    I'm a developer desperately trying to figure out a compresion method used in an old Westwood PC game (Lands of Lore II and III from 1997/1999). I'm not very familiar with compression, and having tried for the last couple of days, I am stuck for now and hoping that maybe someone's trained eye can spot the employed method better than myself - or even knows the method from their own experience.

    Some of Westwood's common algorithms used during that time are well documented (e.g. "Format80" aka. "Westwood LCW" or "Format 2", an LZW variation). The former is actually used in the game, too, but their "main" archives appears to be compressed using a method that is not the standard LCW. It is quite possibly a variant (they are indeed famous forusing variations of their file formats and algorithms…)

    I've been able to obtain a decoded copy for some of the game files (but not for all, that's why the algorithm is still needed), which should be helpful in determining the compression. The files are custom-format resource archives containing textures and sprite graphics in various formats. I've been able to understand the file structure when uncompressed, but not the compression. Below is a screenshot of both compressed (top) and uncompressed (bottom). I have attached the complete (sample) file.

    Click image for larger version. 

Name:	compressed.png 
Views:	30 
Size:	68.8 KB 
ID:	8083


    Click image for larger version. 

Name:	uncompressed.png 
Views:	23 
Size:	24.7 KB 
ID:	8084


    I've discovered that certain literals to copy to the output are prefixed by "00 xy", where xy + 18 is the length of the literal. Some smaller literals are prefixed by"0x", where x + 3 is the length instead. There also seem to be copy instructions, occuring e.g. at offset 36 and 52, but I was unable to determine their pattern. The compresion seems to be command-based to me, similar to"Format 80", but I was unable to determine their structure.


    If anyone has an idea, I would be deeply grateful for any advice!

    Best,
    Michael
    Attached Files Attached Files

  2. #2
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    4,013
    Thanks
    303
    Thanked 1,328 Times in 759 Posts
    First 32 bits look like uncompressed data size.
    After that could be compressed size?
    Literal data match starts at 000A - "00 08" could be either LZSS flags or some descriptor of literal run (of length 1A).
    001A: 5C 02 04 referenced "00 00 A2" string at offset 0002?

    Could be a good idea to compare with known compression algorithm (eg. LZ4) with literal runs and matches of length 3+?
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	cmp1a.png 
Views:	26 
Size:	500.6 KB 
ID:	8087   Click image for larger version. 

Name:	cmp_vs_lz4.png 
Views:	17 
Size:	743.0 KB 
ID:	8088  

  3. #3
    Member
    Join Date
    Nov 2020
    Location
    Germany
    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Thanks Shelwien for your answer!
    I think you're right about the 2 x 4b before the actual compressed data. The first is indeed the uncompressed length, though the second does not seem to match the compressed size.
    I agree about the literals, though some lengths match a +3 size (e.g. offset 0x26), some +18 (e.g. 0x9), and some indicate even one character (e.g. 0x32).
    I also agree about 0x1A (5C 02 [04]) being a reference ("copy 3 bytes"), though I did not consider a pointer to offset 0. In this case, it works, but then the refernce at 0x34 (5C 01 [03]), which probably copies 3 bytes from 0x33, does not fit into the scheme. Relative offsets - in this case 24 and 16 - did not work out either.

    In any case, it's a good start and I'll definitely check out LZ4 and similars (though LZ4 is probably too new to be it I guess). Thanks again

  4. #4
    Member deus-ex's Avatar
    Join Date
    Aug 2019
    Location
    Germany
    Posts
    4
    Thanks
    2
    Thanked 2 Times in 2 Posts
    @micTronic

    I searched the net a bit for you and the following links may be usefull to you:

    Westwood SHP Format (Lands of Lore)
    http://www.shikadi.net/moddingwiki/W...Lands_of_Lore)


    View/Export graphics, animations and other data from Dune2, Legend of Kyrandia, Eye of Beholder, Lands of Lore 1
    https://sourceforge.net/projects/westpak2/


    Last available version of Westpak2
    http://www.andy2.net/archives/WestPak2_070a-exp.exe

  5. #5
    Member
    Join Date
    Nov 2020
    Location
    Germany
    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Thanks for your input too, @deus-ex! I've read about various Westwood formats, but the ones I found documented (including SHP, WSA, VQA, MIX,...) use different compression than the one I encountered.
    Likewise, all available tools (including Westpak, Engine file converter, XCC_Mixer, and others) seem to not recognize that particular format. Nevertheless, I will keep looking for such references too.

Similar Threads

  1. sony PS5 comes with an inbuild "Kraken compression" ...
    By joerg in forum Data Compression
    Replies: 8
    Last Post: 23rd March 2020, 21:42
  2. Replies: 3
    Last Post: 8th April 2018, 02:12
  3. new compressor LZNA = "LZ-nibbled-ANS" - "Oodle 1.45"
    By joerg in forum Data Compression
    Replies: 22
    Last Post: 19th February 2018, 05:50
  4. Replies: 7
    Last Post: 4th January 2016, 15:06
  5. Replies: 6
    Last Post: 24th April 2012, 14:50

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •