Compression speed benchmark

[Bulat Ziganshin]

i just tried my development version (it is why this service popular). i plan to improve binary files compression and then release it

[Sportman]

thanks for decompression times! but ive send today lot of jobs and they are not appeared here.

Yes I added decompression time and compare OK (file size only) at http://www.metacompressor.com/uploads.aspx where results of succeeded jobs are listed.

[donotdisturb]

Pareto Frontier (Decompression process time). Testfile 1 from http://www.metacompressor.com/uploads.aspx.

<div class="jscript"><pre>
SIZE CTM CPTM DTM DPTM
399,933,429 204 13:18.974 26 04.196 lzturbo 0.92 | -19
374,203,844 212 13:50.440 25 04.570 lzturbo 0.92 | -29
357,376,380 675 11:14.517 20 05.288 cabarc | -m lzx:21 N
267,109,958 240 03:57.433 33 07.659 tornado 0.4a | -11
266,715,072 541 08:58.999 33 08.392 tornado 0.3 | -11
263,651,514 987 16:21.745 32 08.486 tornado 0.3 | -12
248,744,915 688 15:25.569 35 17.097 7za 4.57 | -t7z -mx=9
236,817,357 266 04:28.368 240 03m46.435 arc 0.40 | -m6
232,225,857 292 04:42.861 257 04m02.550 arc 0.40 | -m7
183,236,508 1392 23:10.000 1413 23m32.823 paq9a 9a |
</pre></div>


Note: lzturbo is using 4 cores at ~100%, so the process time must
be (normally) divided by 4, making lzturbo the first general-purpose compressor that can decompress > 1gb/s (with proper i/o).
- Tornado (mode -11 & -12) is using > 1,5 gb ram

[Bulat Ziganshin]

tornado -7 mode uses 64megs for decompression and compress better than cabarc. but your rating is based on time, not memreqs.

you not wrote how many memory reqd for 4-threaded -19 compression - it&#039;s also 1.5gb, yes?

[Nania Francesco]

I wanted to signal that have launched different times for the TEST 5 Ringses 1.3 but not from the result! does it perhaps go to Crash?
Please Help me!

[donotdisturb]

Bulat Ziganshin
lzturbo is using maximal ~230mb per core for compression in mode
-11. (also <1gb for this test). The difference is: this mode can
also be used with if a users have < 512mb ram (use lzturbo in single core mode) but to use tornado -11 & -12 mode you must have 2gb ram installed.
This is only a simple note. memory usage is also important to report.

[donotdisturb]

sorry i mean lzturbo mode -19 & -29 (all modes with level 9)

[Sportman]

I wanted to signal that have launched different times for the TEST 5 Ringses 1.3 but not from the result! does it perhaps go to Crash?
Please Help me!

They where all successfully submitted but If a process run for more then one hour and compression is not finished the process is canceled, there is no feedback for this yet.

[Sportman]

I added peak memory used during compression and peak memory used during decompression http://www.metacompressor.com/uploads.aspx

[Sportman]

I wanted to signal

I saw you submitted some new tests, after 9,970 seconds (2 hours 46 min) I manual canceled Rings 1.4 test version busy compressing test file5, it had already created a 47,700,410,368 bytes output file!

Maybe an idea to add that when the output file size is xx% bigger then the input file size to stop compression and exit the application with an application error.

Other tests succeeded and had all improved compression ratio.

[Nania Francesco]

Ok! Thanks !

[Christian]

Just wanted to say big thanks, sportman! Your live-testing environment is awesome. Just a couple of minutes after the request all results were already online.

[Piotr Tarsa]

can someone test bwt based compressors (eg. dark) with big block size? matt on ltcb for unknown reason uses block sizes that are 1/2 1/3 1/4 of enwik9 size instead of using max possible dictionary size (block size should be maximal unless file isn&#039;t homogenous, in such cases it should be divided to maximal homogenous blocks).

i tried testing dark but it failed.

[Sportman]

i tried testing dark but it failed.

Thats because Dark v0.51 with p-b390mf give error "Not enough memory!"

There is only 2GB memory in test system.

[Black_Fox]

big block size

5N is too much for his testing system BBB should, on the other hand, handle whole enwik9 even on 2GB system.