Page 2 of 2 FirstFirst 12
Results 31 to 42 of 42

Thread: LzTurbo: The fastest now more faster

  1. #31
    Member
    Join Date
    Mar 2017
    Location
    Kashmir
    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts
    @Anyone Who Can Help
    How can I make Lzturbo Show progress in inno...
    How can i use lzturbo stdin stdout..Thanks

  2. #32
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,912
    Thanks
    291
    Thanked 1,272 Times in 719 Posts

  3. #33
    Member
    Join Date
    May 2017
    Location
    Sealand
    Posts
    15
    Thanks
    7
    Thanked 2 Times in 2 Posts
    I've tried many commands for lzt with stdin stdout in the arc.ini but i just couldnt find it. It'd be great if someone could send me the lzt arc.ini commands with stdout stdin support.

  4. #34
    Member
    Join Date
    May 2017
    Location
    Hungary
    Posts
    10
    Thanks
    49
    Thanked 3 Times in 3 Posts
    Stdout in windows binary is buggy!
    The author told me.

  5. Thanks:

    Chirantan (24th September 2017)

  6. #35
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    981
    Thanks
    96
    Thanked 396 Times in 276 Posts
    I tried Lzturbo 1.2 with enwik10 but decompress file is only 1,073,741,824 bytes.

    227,112,649 bytes, 2,342.967 sec. - 10.514 sec., lzturbo -49 -p1 (v1.2)

    With compression I see a row:
    VirtualAlloc failed

    No other errors.

    OS page file is disabled.

  7. #36
    Member CompressMaster's Avatar
    Join Date
    Jun 2018
    Location
    Lovinobana, Slovakia
    Posts
    189
    Thanks
    52
    Thanked 13 Times in 13 Posts
    Quote Originally Posted by Sportman View Post
    OS page file is disabled.
    That´s the problem. Lzturbo 1.2 probably needs more RAM than you have available. Enable pagefile and see if it helps. Or upgrade your RAM. But compression will be slooooow, though.

  8. #37
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    981
    Thanks
    96
    Thanked 396 Times in 276 Posts
    Quote Originally Posted by CompressMaster View Post
    Lzturbo 1.2 probably needs more RAM than you have available.
    There is 32GB and 30GB+ memory available.

  9. #38
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    565
    Thanks
    67
    Thanked 199 Times in 147 Posts
    Try restarting your system.

  10. #39
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    981
    Thanks
    96
    Thanked 396 Times in 276 Posts
    Quote Originally Posted by dnd View Post
    Try restarting your system.
    It didn't help, but I found the problem.

    It looks like that Lzturbo 1.2 (-49) use system TEMP path to write something while compressing, but this TEMP path is at an other much slower storage device and had less then 10GB free space. I increased the temp space to 10GB+ (still at the slow storage device) and now it work without errors:

    2,028,356,329 bytes, 8,212.226 sec. - 33.829 sec., lzturbo -49 -p0 (v1.2)

    Lzturbo 1.2 -32 option hadn't this problem.

  11. Thanks:

    dnd (20th January 2020)

  12. #40
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    565
    Thanks
    67
    Thanked 199 Times in 147 Posts
    LzTurbo is not using any temporary files.
    The issue has to do with "VirtualAlloc" on windows.
    It's possible that the Virtual Memory Size is set to 16GB or lower on your system.
    If you want, you can change the settings in "Computer Properties/Advanced System Settings/Performance/Advanced/Virtual Memory"
    It can be other solution possible:https://github.com/nicehash/NiceHash...ved/issues/652

    Please make sure that decompression is done from the faster drive.
    Last edited by dnd; 20th January 2020 at 12:44.

  13. #41
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    981
    Thanks
    96
    Thanked 396 Times in 276 Posts
    Quote Originally Posted by dnd View Post
    It's possible that the Virtual Memory Size is set to 16GB or lower on your system.
    Page file was in all tests disabled because I wanted an error instead of using a page file (memory swapping).

    Strange enough Lzturbo 1.2 -49 only use 2,089,672KB memory during compression and it looks like the same for decompression, so no swapping expected.

    Maybe Windows use a lot of memory for caching (when reading a 10GB file and writing a 2GB+ file) and when the page file is disabled use the temp folder. I need to research this more.

    For now I enabled the page file and moved the temp folder both at/to the fast fast storage device.

    Quote Originally Posted by dnd View Post
    Please make sure that decompression is done from the faster drive.
    Compression and decompression are always done at the same fast storage device.

    Update 1:
    I tested with a 30GB fixed page file and no temp file space and got the same "VirtualAlloc failed" error with Lzturbo 1.2 by options -19, -29, -39 and -49 only.

    Update 2:
    I tested with a 30GB fixed page file and 32GB temp file space and got the same "VirtualAlloc failed" error with Lzturbo 1.2 by options -19, -29, -39 and -49 only.
    Last edited by Sportman; 21st January 2020 at 03:58.

  14. #42
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    565
    Thanks
    67
    Thanked 199 Times in 147 Posts
    Strange enough Lzturbo 1.2 -49 only use 2,089,672KB memory

    LzTurbo is using a custom memory mapped allocator (based on
    VirtualAlloc).
    It can use up to ~16-17 GB for 1GB block (option -b1000).
    You must look at the virtual memory used.

Page 2 of 2 FirstFirst 12

Similar Threads

  1. Faster permutation inverse algorithm?
    By nburns in forum The Off-Topic Lounge
    Replies: 7
    Last Post: 21st July 2013, 11:28
  2. Computing the BWTS faster
    By nburns in forum Data Compression
    Replies: 4
    Last Post: 20th July 2013, 20:17
  3. Replies: 1
    Last Post: 21st June 2011, 11:48
  4. GCC 4.6 faster than previous GCC, faster than LLVM
    By willvarfar in forum The Off-Topic Lounge
    Replies: 2
    Last Post: 15th November 2010, 15:09
  5. Faster PAQ7/8
    By LovePimple in forum Forum Archive
    Replies: 6
    Last Post: 8th February 2007, 13:04

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •