[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Lzip-bug] On Windows unpacking does not use all cores
From: |
Romano |
Subject: |
[Lzip-bug] On Windows unpacking does not use all cores |
Date: |
Mon, 26 Feb 2018 19:57:13 +0100 |
User-agent: |
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 |
Greetings Antonio, I have an issue with utilizing all 4 cores of my CPU
during unpacking, it seems to only use around 25-43% of my 4 core cpu in
any case, even if using stdio.
Yesterday I compiled latest lzlib 1.9 and plzip 1.7 under cygwin(also
latest, just installed yesterday) on Windows 7 and as a 64bit both. It
compiled without any errors or warning and tool also work fine. During
packing it is able to utilize all CPU cores to 100% so multithreading
works. Same with testing using -t flag. Howerver, when I actually try to
unpack with -d it never even peak above 50%. This despite -n option,
even if I double the number to -n 8. On parallel, until now I always
used FreeArc's internal 4x4:lzma which always fully utilized my cpu and
it shows as during unpacking without io limitation it could reach ~200Mib/s.
I am aware of blocks concept as well, tool also did not utilized all CPU
with smaller -B block and big enough file, and I know for sure its not
my HDD limiting it because first it is quicker than output and FreeArc
can still utilize its max, but also because it does not utilize full CPU
even if using stdin/stdout.
I tried it alone as well as in conjuction with FreeArc as I am
considering its own 32bit lzma to replace with plzip, if I can fix this
problem. I never liked xz and 7zip and their lzma2 preciselly because it
doesnt support unpacking on more threads. I see this tool as finally
something with better vision. For reference, if you are familiar with
FreeArc here is my setup:
[External compressor:plzip]
default = -s 64Mi -m 273 -B 128Mi
packcmd = plzip.exe -c -n 4 {options} - - <stdin> <stdout>
unpackcmd = plzip.exe -c -n 8 -d - - <stdin> <stdout>
^Decompress seem to ignore -n option, here as well as on its own command
line. Thesting 8+Gib archive with this option(using "test archive" under
FreeArc gui) would result on using only 25-40% cpu again, while FA
internal 4x4:lzma would peak CPU fully. I also include compiled binaries
if it help. But they were compiled with -march=native -O2 on Haswell, so
need avx2 cpu. I can compile another one for you if you cannot run this.
I hope this help you and thanks for a great tool you did.
Best regards, Romano
(PS: Also, I dont know if this is a bug in plzip, but if I compress
under Freearc and click "cancel", first 3 threads eventually cancel as
they should but last one remain with CPU at 25% forever - thus plzip
never quit.)
plzip_compiled.zip
Description: Binary data
- [Lzip-bug] On Windows unpacking does not use all cores,
Romano <=