Thread: LZTURBO 0.9 parallel compressor

New version : lzturbo 0.9 parallel compressor
at http://consultant-berater.de/lzturbo

- improved & faster compression.
- compress/decompress multiple files in parallel.
- compress/decompress directories recursively.
- compress/decompress as easy as the copy command. No archive files needed.

Please compare lzturbo to thor, slug, quicklz. Specially on newer hardware (like core 2 duo).
Use switches as recommended in lzturbo.txt file.

Please read lzturbo.txt

Enjoy!

Thanks Hamid!

Quick test...

Compression settings: -11

A10.jpg > 842,504
AcroRd32.exe > 2,095,707
english.dic > 1,648,598
FlashMX.pdf > 4,341,107
FP.LOG > 2,032,838
MSO97.DLL > 2,649,965
ohs.doc > 1,107,412
rafale.bmp > 1,912,757
vcfiu.hlp > 1,142,726
world95.txt > 1,243,243

Total = 19,016,857 bytes


Compression settings: -59

A10.jpg > 839,071
AcroRd32.exe > 1,417,442
english.dic > 687,644
FlashMX.pdf > 3,736,621
FP.LOG > 789,021
MSO97.DLL > 1,810,448
ohs.doc > 833,243
rafale.bmp > 1,006,476
vcfiu.hlp > 677,425
world95.txt > 576,470

Total = 12,373,861 bytes

Another quick test...

Test machine: Intel PIII (Coppermine) @750 MHz, 512 MB RAM, Windows 2000 Pro SP4

Test File: ENWIK9 (1,000,000,000 bytes)

Timed with AcuTimer v1.2


Compression settings: -11

Compressed size: 430,453,227 bytes
Elapsed Time: 00:02:05.970 (125.970 Seconds)


Compression settings: -12

Compressed size: 386,091,908 bytes
Elapsed Time: 00:02:49.249 (169.249 Seconds)

Thanks LovePimple!

http://cs.fit.edu/~mmahoney/compression/text.html#2328

Quick Test:

Eve Online installation:

uncompressed 3.037.826.415 Bytes

7z -md=128m -mfb=273 -ms=on -mx=9
compressed 1.204.469.019 Bytes
Kernel Time = 20.546 = 00:00:20.546 = 0%
User Time = 4742.406 = 01:19:02.406 = 160%
Process Time = 4762.953 = 01:19:22.953 = 161%
Global Time = 2946.204 = 00:49:06.204 = 100%

LZTurbo -59 -r
compressed 1.427.313.134 Bytes
Kernel Time = 64.968 = 00:01:04.968 = 0%
User Time = 13569.484 = 03:46:09.484 = 198%
Process Time = 13634.453 = 03:47:14.453 = 199%
Global Time = 6832.703 = 01:53:52.703 = 100%

Thanks Matt!
Thanks thometal !

thometal
Can you report decompression time, please.

7z decompressions time:
Kernel Time = 8.843 = 00:00:08.843 = 2%
User Time = 194.953 = 00:03:14.953 = 62%
Process Time = 203.796 = 00:03:23.796 = 65%
Global Time = 312.359 = 00:05:12.359 = 100%

LZTurbo will i post later

LZTurbo decompressions time:

Kernel Time = 22.640 = 00:00:22.640 = 5%
User Time = 586.765 = 00:09:46.765 = 143%
Process Time = 609.406 = 00:10:09.406 = 148%
Global Time = 409.657 = 00:06:49.657 = 100%

enwik9: AMD X2-3800+ (Dual core)
__________________________size____Global comp._time in sec.
7z_-md=128m_-mfb=273___206938137__32m13
___-ms=on_-mx=9
lzturbo_______-59________232701587___9m33

thor_0.96a_e4___________314092324___1m05
thor_0.96a_e5___________315611172___1m27
lzturbo_______-43________318502927_____35
gzip_-9_________________322591995___1m41

lzturbo_______-41________361432918_____24.82
quicklz1.4b6_3___________397439311___1m20

lzturbo_______-21________401221587_____25.37
slug1.1_________________404250979_____26.0
quicklz1.4b6_2___________406053574_____30.38
lzturbo_______-11________430453227_____30.52
quicklz1.4b6_1___________466309381_____32.4
thor_0.96a_e1___________488397982_____32.6

donotdisturb
use CODE tag

<div class="jscript"><pre>
enwik9: AMD X2-3800+ (Dual core)
size Global comp. time in sec.
7z -md=128m -mfb=273 206938137 32m13
-ms=on -mx=9
lzturbo -59 232701587 9m33
lzturbo -44 297641351 1m21

thor 0.96a e4 314092324 1m05
thor 0.96a e5 315611172 1m27
lzturbo -43 318502927 35
gzip -9 322591995 1m41

lzturbo -41 361432918 24.82
quicklz1.4b6 3 397439311 1m20

lzturbo -21 401221587 25.37
slug1.1 404250979 26.0
quicklz1.4b6 2 406053574 30.38
lzturbo -11 430453227 30.52
quicklz1.4b6 1 466309381 32.4
thor 0.96a e1 488397982 32.6
</pre></div>

donotdisturb
By the way, can you describe the -59 algorithm? How do you search matches (hash chains, binary tree, ...), how do you encode LZ output, and so on.

i&#039;m curious about 7- zip resluts in lzma mode with 1 gb dictionary

The MC SFC summary shows a total of 12,381,859 bytes rather than the 12,373,861 bytes I got using the -59 setting.

http://www.maximumcompression.com/data/summary_sf.php

Originally Posted by encode

how do you encode LZ output, and so on

it seems that he used coders from tornado for -1*/-2*/-4* modes

Originally Posted by Bulat Ziganshin

it seems that he used coders from tornado for -1*/-2*/-4* modes

Indeed suspicious compressor - too many mirrored features from TORNADO...

i don&#039;t think that there are many people who can develop good huf/ari encoder, so it&#039;s not surprising