Page 2 of 7 FirstFirst 1234 ... LastLast
Results 31 to 60 of 191

Thread: Precomp 0.4

  1. #31
    Member
    Join Date
    May 2008
    Location
    Kuwait
    Posts
    353
    Thanks
    37
    Thanked 38 Times in 23 Posts
    Quote Originally Posted by nanoflooder View Post
    ABBYY uses their own zlib library called AbbyyZlib.dll to compress the Lingvo dictionaries. Unfortunately, it has very different export functions, and the produced file structure seems to be different because Precomp can't decompress it even in -brute mode. Could you please look at it and make a verdict on whether or not is it possible to make something out of this? Thanks in advance
    I have attached the library and a sample dictionary (I've chosen a very small one to make it able to be compressed in -brute mode).
    after i hex-view the file i found this:

    deflate 1.2.3 Copyright 1995-2005 Jean-loup Gailly

    inflate 1.2.3 Copyright 1995-2005 Mark Adler

    I.n.c.\.Z.l.i.b.F.i.l.e...h.......\.S.r.c.\.Z.l.i. b.F.i.l.e...c.p.p

    and strange that zlib developers where not mentioned for copyright. May be they but it in the main program..

  2. #32
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    615
    Thanks
    255
    Thanked 242 Times in 121 Posts
    Quote Originally Posted by mhajicek View Post
    These PNG files (unzip first) after processing with precomp 0.4 are not restored back correctly from .pcf . Can send you more if needed, but others are bigger (~1MB each).
    Thanks for these files. This seems to be a special bug in processing multi PNGs (PNGs where the zLib data is split into multiple chunks), perhaps related to penalty bytes.
    I'll try to fix it - if you have additional images where this happens, I can send you a test version afterwards so you can test if the bug is fixed for all of them.
    http://schnaader.info
    Damn kids. They're all alike.

  3. #33
    Member
    Join Date
    Mar 2009
    Location
    Prague, CZ
    Posts
    62
    Thanks
    32
    Thanked 7 Times in 7 Posts
    Sure, ready for testing
    (for its almost the only thing i can do to help the development because of my poor programming skills)

  4. #34
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    615
    Thanks
    255
    Thanked 242 Times in 121 Posts
    I managed to fix both bugs Michal reported.

    It seems the crash happened because ZLIB1.DLL isn't used anymore. It looks like I can't compile zLib myself without getting this crashes, even switching compiler optimizations off or compiling ZLIB1.DLL myself didn't help.
    So in the next version, ZLIB1.DLL will be needed again until I found out what happens here and how to fix it.

    The PNG recompression errors were errors in the multi PNG code. Penalty bytes weren't spread correctly across multiple chunks and sometimes the last chunk length wasn't stored. Those cases occur rarely, so that is why this bug hasn't yet been found in an earlier version.
    I send a PM with a developer version to Michal so he can test it on the other PNGs he has. If everything runs fine, I'm looking forward to release Precomp 0.4.1 next week.
    http://schnaader.info
    Damn kids. They're all alike.

  5. #35
    Member
    Join Date
    May 2008
    Location
    brazil
    Posts
    163
    Thanks
    0
    Thanked 3 Times in 3 Posts
    Hello

    If possible ,add a plugin interface in the next version and modularise all precompression algos.


    (And pleeease , open the source someday)

    Thanks

  6. #36
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    920
    Thanks
    57
    Thanked 113 Times in 90 Posts
    will precomp 4.1 have multi file support ( *.* ) and maybe be able to work on same numbers of files as cores in parallel ?

  7. #37
    Member
    Join Date
    Aug 2008
    Location
    Saint Petersburg, Russia
    Posts
    215
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Why would you need the multi-file support in...Precomp? It's so much more useful with batch files

    For example: you want to try to decompress all the files and
    - delete the .pcf if the file hasn't been decompressed;
    - delete the original file if it has;
    - do this in current directory and all its subdirectories.

    Code:
    @echo off
    @dir /a-d /b /s>"%temp%\precomp.txt"
    @cd ..
    @pause
    @echo Decompressed:
    @for /f "usebackq delims=" %%f in ("%temp%\precomp.txt") do (
    	rem HidCon.exe might be useful here in case you
    	rem don't want to see a lot of blinking windows
    	@start /wait cmd /c precomp -slow -o"%%f.pcf" "%%f"
    	@if errorlevel 2 (
    		@del "%%f.pcf"
    	) else (
    		@del "%%f"
    		@echo   %%f
    	)
    )
    @del "%temp%\precomp.txt"
    Then, when you're finished, you would like to compress all the files back, wouldn't you?
    Code:
    @echo off
    @dir /a-d /b /s *.pcf>"%temp%\precomp.txt"
    @echo Compressed:
    @for /f "usebackq delims=" %%f in ("%temp%\precomp.txt") do (
    	@start /wait precomp -r -o"%%~dpnf" "%%f"
    	@echo   %%f
    	@del "%%f"
    )
    @del "%temp%\precomp.txt"
    I personally use [similar to] this code inside some of my current distributions. It really does help to save some weight!

  8. #38
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    615
    Thanks
    255
    Thanked 242 Times in 121 Posts
    Quote Originally Posted by lunaris View Post
    Hello

    If possible ,add a plugin interface in the next version and modularise all precompression algos.

    (And pleeease , open the source someday)

    Thanks
    Opening the source and some kind of interface to add "plug-in" support for own file formats (this is what you meant, isn't it?) will happen when Precomp gets beta, which is not that far away as we're getting close to 0.5 which will be the first beta version.

    Quote Originally Posted by SvenBent View Post
    will precomp 4.1 have multi file support ( *.* ) and maybe be able to work on same numbers of files as cores in parallel ?
    I'm pretty sure Precomp 0.4.1 won't support both these features as it will be available soon to fix some 0.4 bugs and crashes, but I want to implement multiple file support next, so it's likely this will be added in 0.4.2. By the way, nanoflooder has a good point here, the latest version returns errorlevels, so it's possible to glue this together yourself. I also think that there are command-line tools to spread commands across different CPU cores, so even that should be possible using batch files.
    http://schnaader.info
    Damn kids. They're all alike.

  9. #39
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    920
    Thanks
    57
    Thanked 113 Times in 90 Posts
    the problem with running the file multiple times by batch is purely performance

    is not noticeable on big file. several thousands jpegs the constant loading of precomp does take up a considerable amount of time.


    also i wan to be able to do it in several subfoldes as weel but i cannot use sweep for th
    sweep'ing af start command make screw up the "" arodun variabel and thne it doesn work with filenames with space
    e.g

    sweep start for %i in (*.jpg) do precomp "%i"

    will not work even thought just
    start for %i in (*.jpg) do precomp "%i" works perfectly in each directory


    so instead it would be nice to be able to
    sweep start precomp *.*

  10. #40
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    688
    Thanks
    41
    Thanked 173 Times in 88 Posts
    Quote Originally Posted by schnaader View Post
    Hi!
    Finally, Precomp 0.4 is out....
    Schnaader, just thank you !!!
    I would like to report not the bug but little imprecision. Seems that PreComp stores the name of processed file in low-case. I mean if I use precomp TEST.DAT I got test.dat after recompression. It's not critical but mandatory for some of my tasks. Of course I can use -r -o solution but anyway.

    Quote Originally Posted by schnaader View Post
    ... but I hope I can add support for more stream types like CAB in the next versions ...
    It will be just great. As far as I know CAB format used inside of MSI files so make them smaller in distributive will be nice.

    Quote Originally Posted by nanoflooder View Post
    ABBYY uses their own zlib library called AbbyyZlib.dll to compress the Lingvo dictionaries. Unfortunately, it has very different export functions, and the produced file structure seems to be different because Precomp can't decompress it even in -brute mode.
    You'll not believe me but I thought about it just a few days ago. I suppose DSL Compiler applies not only zlib compression but also some kind of encryption on resulting lsd dics. By the way what Lingvo version you're using? I can't find AbbyyZlib.dll file in my Lingvo 12.

  11. #41
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    615
    Thanks
    255
    Thanked 242 Times in 121 Posts
    Quote Originally Posted by Skymmer View Post
    Schnaader, just thank you !!!
    I would like to report not the bug but little imprecision. Seems that PreComp stores the name of processed file in low-case. I mean if I use precomp TEST.DAT I got test.dat after recompression. It's not critical but mandatory for some of my tasks. Of course I can use -r -o solution but anyway.
    Conversion to lower case was done to prevent differences in archive when only the filename case differs. This should be indeed corrected, the best behaviour would be to not use the case of the input file parameter but to get the correct case from the file system instead. This will also be important for upcoming Linux versions.
    http://schnaader.info
    Damn kids. They're all alike.

  12. #42
    Member
    Join Date
    Mar 2009
    Location
    Prague, CZ
    Posts
    62
    Thanks
    32
    Thanked 7 Times in 7 Posts
    To nanoflooder:
    Thank you for the posted batch code. Useful, simple, inspired me to learn a bit more about it.
    btw tempfile not necessary afaik

  13. #43
    Member
    Join Date
    Apr 2009
    Location
    Beirut,Lebanon
    Posts
    39
    Thanks
    0
    Thanked 0 Times in 0 Posts

    How to recompress?

    Hi all i just have this problem whenever i precompres a file it always give the same size whatever i use Winrar & 7zip all zip formats i tried nothing seems to work plz can some one post me whats the ideal way to use or the best software and how to precompres it with the best way i would apreciate the help....

  14. #44
    Member
    Join Date
    May 2008
    Location
    Antwerp , country:Belgium , W.Europe
    Posts
    487
    Thanks
    1
    Thanked 3 Times in 3 Posts
    Quote Originally Posted by SolVanDyck View Post
    Hi all i just have this problem whenever i precompres a file it always give the same size whatever i use Winrar & 7zip all zip formats i tried nothing seems to work plz can some one post me whats the ideal way to use or the best software and how to precompres it with the best way i would apreciate the help....
    Try for example a .PDF file and compare the following :
    a.pdf -> Winrar -> a.pdf.rar (1)
    a.pdf -> Precomp -> a.pdf.pcf -> Winrar -> a.pdf.pcf.rar (2)

    compare the size of (1) and (2).
    You could do the same using 7zip instead of Winrar.

  15. #45
    Member
    Join Date
    Apr 2009
    Location
    Beirut,Lebanon
    Posts
    39
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Stil the same size nothing has changed

  16. #46
    Member Fu Siyuan's Avatar
    Join Date
    Apr 2009
    Location
    Mountain View, CA, US
    Posts
    176
    Thanks
    10
    Thanked 17 Times in 2 Posts

    Thumbs up

    Quote Originally Posted by SolVanDyck View Post
    Stil the same size nothing has changed
    I just tried the program and it is amazing!

    Lossless_Compression_Statistical_Model.pdf -- 3.99M
    convert it to xxxxxxxxxxxxxxxxxxxxxxxx.pcf -- 9.04M

    pdf--> my compressor (csc2) 732k
    pcf--> my compressor (csc2) 375k (half reduced)

    but the WinRAR works bad on this file. pdf--> rar 709k, pcf---> 698k. I think the small dictionary size (4M) of WinRAR must be the main problem.

  17. #47
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    688
    Thanks
    41
    Thanked 173 Times in 88 Posts
    Quote Originally Posted by SolVanDyck View Post
    Stil the same size nothing has changed
    First of all can I give advise ? When you ask for help try to be verbose as you can. What type of file you're trying to pre-compress ? What's the feedback from the program ?

    In your case, did you try -slow switch ?
    Maybe PreComp writes: "None of the given compression and memory levels could be used.
    There will be no gain compressing the output file." ?
    If so, then PreComp just can't deal with such file.

  18. #48
    Member
    Join Date
    Apr 2009
    Location
    Beirut,Lebanon
    Posts
    39
    Thanks
    0
    Thanked 0 Times in 0 Posts

    The Problem is!

    I have 3 maps file of a game and i need to recompress them in what the best format do i need to compress them .rar .7z .zip etc...? which ideal u advise me to chose and i need to compress it to maximum? and what the best way to recompress it? and if u could show the command line with the switch that would be prefect and please write the command line in the simplest form so i could copy paste it to a batch file es: precomp.exe filename etc..

  19. #49
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    409
    Thanks
    0
    Thanked 5 Times in 5 Posts
    I guess you don't know very much about precompression and the data you have.
    Precomp extracts zip, gzip, bz2 and some other streams inside a file and reconstructs them on the other way.
    It absolutely won't work on every random file and is no magic weapon everywhere. If you want help you have to give us more details.
    Use precomp and the -slow switch. Then look what the output says. Dunno whats hard there. If the output says: There will be no gain compressing the output file as Skymmer pointed out it's easy to understand. If it says something like 0/20 streams recompressed there is a small chance that this can be changed in later versions.

  20. #50
    Member
    Join Date
    Apr 2009
    Location
    Beirut,Lebanon
    Posts
    39
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Just dont get it.

    I just want to know how can i use it arent i suppose to compress a file than add it to precomp and when it finishes it size is bigger than the old one then i cmpress it with a more powerful archive than than deflate like freearc and what software to use for deflate people im just confused please help :? :? :?

  21. #51
    Tester
    Black_Fox's Avatar
    Join Date
    May 2008
    Location
    [CZE] Czechia
    Posts
    471
    Thanks
    26
    Thanked 9 Times in 8 Posts
    You don't compress in the beginning, you are supposed to take a file, give it to precomp and compress it after that. (Also you're supposed to search first and ask only if you can't find relevant info yourself, but I guess you'll figure out this one in time )
    I am... Black_Fox... my discontinued benchmark
    "No one involved in computers would ever say that a certain amount of memory is enough for all time? I keep bumping into that silly quotation attributed to me that says 640K of memory is enough. There's never a citation; the quotation just floats like a rumor, repeated again and again." -- Bill Gates

  22. #52
    Member
    Join Date
    Aug 2008
    Location
    Saint Petersburg, Russia
    Posts
    215
    Thanks
    0
    Thanked 0 Times in 0 Posts
    SolVanDyck, maybe this will help:
    Imagine a bunch of files, including dll's and exe's with embedded pictures, jpg's, png's and so on - when you try to compress them with an archiver, it doesn't compress all the precompressed data (such as jpg, png etc.), though the compression ratio of the algorythms used in those files is significantly worse than in most of the today's archivers - it's deflate, gzip, bzip2, instead of lzma, ppmd, proper bwt, you know. So what precomp does is decompresses all that precompressed data, so you can perform superior compression using your own, better archiver, and then afterwards decompress it and recompress all the jpg, gif, png back!

    Compression: .jpg (contains metadata + deflated picture) -> [precomp] .jpg.pcf (contains metadata + decompressed picture) -> [7-Zip (for example)] .jpg.pcf.7z (contains well compressed metadata and the original picture)
    Decompression: .jpg.pcf.7z -> [7-Zip] .jpg.pcf -> [precomp] .jpg

  23. #53
    Member Fu Siyuan's Avatar
    Join Date
    Apr 2009
    Location
    Mountain View, CA, US
    Posts
    176
    Thanks
    10
    Thanked 17 Times in 2 Posts

    Smile

    Quote Originally Posted by nanoflooder View Post
    SolVanDyck, maybe this will help:
    Imagine a bunch of files, including dll's and exe's with embedded pictures, jpg's, png's and so on - when you try to compress them with an archiver, it doesn't compress all the precompressed data (such as jpg, png etc.), though the compression ratio of the algorythms used in those files is significantly worse than in most of the today's archivers - it's deflate, gzip, bzip2, instead of lzma, ppmd, proper bwt, you know. So what precomp does is decompresses all that precompressed data, so you can perform superior compression using your own, better archiver, and then afterwards decompress it and recompress all the jpg, gif, png back!

    Compression: .jpg (contains metadata + deflated picture) -> [precomp] .jpg.pcf (contains metadata + decompressed picture) -> [7-Zip (for example)] .jpg.pcf.7z (contains well compressed metadata and the original picture)
    Decompression: .jpg.pcf.7z -> [7-Zip] .jpg.pcf -> [precomp] .jpg
    Thank you nanoflooder. it helps me,too. I finally understand what it is used for. It give u a chance to recompress the data with a better algorithm while replacing the old compression method used in the already compressed files.

  24. #54
    Member
    Join Date
    Aug 2008
    Location
    Saint Petersburg, Russia
    Posts
    215
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Yup, exactly.

  25. #55
    Member
    Join Date
    Apr 2009
    Location
    Beirut,Lebanon
    Posts
    39
    Thanks
    0
    Thanked 0 Times in 0 Posts
    thanks nanoflooder allot really but i need the steps only i know it recompresses
    but i re compress a zipped file or any file bear with me please..

  26. #56
    Member
    Join Date
    Aug 2008
    Location
    Saint Petersburg, Russia
    Posts
    215
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Let's say you got c:\file.zip to recompress
    Code:
    c:\>precomp -slow -ofile.zip.pcf file.zip
    You will get another file, c:\file.zip.pcf, which is a decompressed zip in precomp format. Now you can compress it better
    Code:
    c:\>7z a -mx9 file.zip.pcf.7z file.zip.pcf
    The .7z file will weigh much less than the original .zip. So you give the file to your friend along with precomp.exe and packjpg_dll.dll, and to decompress it back you just run
    Code:
    c:\>7z x file.zip.pcf.7z
    c:\>precomp -r -ofile.zip file.zip.pcf
    c:\>del file.zip.*

  27. #57
    Member
    Join Date
    Apr 2009
    Location
    Beirut,Lebanon
    Posts
    39
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Thanks allot nanoflooder that explains allot man and i even tried it
    but it gave a message "non of the given data could be compressed...."
    what should i do do i need file larger that 1 GB to work? can i use at the finale compression freearc because i like at allot and if so give the command of freearc
    to compress to the maximum.

  28. #58
    Member
    Join Date
    Aug 2008
    Location
    Saint Petersburg, Russia
    Posts
    215
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by SolVanDyck View Post
    but it gave a message "non of the given data could be compressed...."
    ...so why didn't you read that aloud earlier?
    what should i do do i need file larger that 1 GB to work?
    Nono, Precomp isn't a magic tool - if it doesn't detect anything it can recompress in a file, it won't do that, regardless of the file size.
    can i use at the finale compression freearc because i like at allot and if so give the command of freearc to compress to the maximum.
    Okay, I allow you to use FreeArc. But only once! Then use gzip forever.

  29. #59
    Member
    Join Date
    Apr 2009
    Location
    Beirut,Lebanon
    Posts
    39
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Gzip what is gzip do u have it download link...

  30. #60
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,

    Quote Originally Posted by SolVanDyck View Post
    Gzip what is gzip do u have it download link...
    I suggest to have a cup of coffee or tea (a BIG one...) and read a site or two...

    - http://datacompression.info/
    - http://www.maximumcompression.com/programs.php
    - http://dir.yahoo.com/Computers_and_I...s/compression/
    - http://en.wikipedia.org/wiki/File_compression
    - ftp://ftp.elf.stuba.sk/pub/pc/pack/ <= should provide some programs of interest

    Just to mention a few...
    Be sure to have enough coffee / tea

    Best regards!

Page 2 of 7 FirstFirst 1234 ... LastLast

Similar Threads

  1. Precomp (and Precomp Comfort) in 315 kb
    By Yuri Grille. in forum Data Compression
    Replies: 2
    Last Post: 1st April 2009, 19:40
  2. Precomp 0.3.8
    By schnaader in forum Data Compression
    Replies: 116
    Last Post: 6th March 2009, 09:37
  3. Precomp 0.3.5 is out!
    By squxe in forum Forum Archive
    Replies: 1
    Last Post: 20th August 2007, 14:55
  4. Precomp 0.3.3 is out!
    By squxe in forum Forum Archive
    Replies: 1
    Last Post: 20th July 2007, 17:27

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •