Activity Stream

EDIT: Due to the latest switch from deflate in favor of Zlib, the algorithm manages to SURPASS PNG in most cases. Hello, I'm a newbie to compression and this forum, so please be patient. I recently wrote an image compression algorithm and put a prototype on GitHub. I've written a long explanation there, but in order to make it more comfortable here's the explanation: Image Compression Algorithm A new image compression algorithm. In this release version, the algorithm performs worse than PNG in most cases. In fact, the only image, where the algorithm outperforms PNG is the white void of img_3.png. However, the algorithm produces just slightly larger files then PNG. For example, img_2.png is about 12.8 MB, the resulting binary is 12.9 MB. How the system works Clustering The first step in the system is clustering the pixels. This happens in 5 Dimensions, with R, G, B, x and y of every Pixel. X & Y are normed over 255 in order to have a balance between the color values and the pixel position. This might offer a possible improvement. In the current settings a Kmeans is used to define 3 dominant clusters. More clusters are possible, but the calculation time increases rapidly with an increasing number of clusters. The encoding supports up to 255 clusters, but this is probably overkill. After defining the clusters, we calculate a cluster map, that removes the color values and just displays belonging to a cluster. A visualization of this would look like this: Grid In the next step we lay a grid on top of the cluster map. The chunks of the grids are not fixed size. They vary in size near the edges. For every grid, we check if all pixels in a grid belong to the same cluster. If this is given, the pixel is calculated relative, otherwise absolute. The gird contains for every chunk a value that determines the cluster or that the chunk has to be calculated absolute. Here is an illustration of this grid map. Every white pixel, symbolizes an absolute chunk. Calculating Lists In this step we, finally calculate the pixel values that are later written into the file. Every chunk is calculated according to the grid’s perception of absolute or relative value. Every chucks pixel values are added to a super list of relative or absolute pixel values. The pixel values are calculated in wiggly lines. Every cluster has a minimum pixel value. This value is according to the minimum R, G, B value in that chunk. The resulting pixel value is an addition of this chunk value and the encoded pixel value. Flatten and Byte conversion The grid, the cluster colors, the lines are converted in Vectors of u8 and then converted into bytes. Deflate Grid and lines bytes representations are compressed with the deflate algorithm. This should achieve the compression and provides an opportunity to optimization. Write File The resulting binary is just a list of the relevant compressed objects. Advantages compared to PNG Because of the grid, it's possible to load just specific chunks without loading the entire image. With further improvements it might be possible to surpass PNG in compression rate, but I can't prove that. Disadvantages compared to PNG Because of the clusterisation it takes quite long to calculate a result. It might be possible to improve that, although this would probably require to abolish Kmeans for another clustering algorithm. One solution to that could be a neuronal net. Help and Ideas are much appreciated, especially contributions on GitHub Thanks for your time! :)

I'm reverse engineering a version of Asobo Studio's DPC archive format used in the PC release of the game FUEL (2009). I am able to unwrap the first "layer" of the format by breaking the archive down into the files described in the DPC header using a modified version of this MexScript. However these extracted files appear to be compressed with a custom LZ variant. Some games released before FUEL (CT Special Forces: Fire for effect, Ratatouille, and Wall-E) each used a slightly different LZ variant than the previous release so I am expecting FUEL to use something similar to those. @Shelwien has provided a series of unLZ_rhys scripts in previous posts (linked at the bottom) but none of them seam to properly decompress the files I extracted. I have attached a selection of extracted files that appear to be compressed and contain a small amount of text near the beginning. They all follow a similar pattern to the one in this image: Which closely resembles the compressed files from the previous posts. In theory this should only require a small modification to the unLZ_rhys tool but unfortunately I cannot seem to figure out the header layout/mask for this new version of the format. Any help with how to modify the tool or advice in general would be greatly appreciated. If you need more samples or the original DPC files I can provide them. https://encode.su/threads/3147-Reverse-Engineering-Custom-LZ-Variant https://encode.su/threads/3526-Asobo-s-Ratatouille-DPC-Data-Compression