Squeeze Chart 2007

[encode]

cool, I will test Quad 1.11 asap

QUAD 1.11 is still under development and isnt available for public yet.

The fixed benchmark is online.

Thank you!

[Stephan Busch]

New benchmark is online including latest Quad versions..
Quad 1.11 hash 2 has a very very nice speed improvement, Ilia.

[Stephan Busch]

Here you go guys:

Quad 1.12
FreHook 0.2
CCM 1.20c

results are available now on my benchmark site

[Christian]

If you plan to test 1.20d, could you then please remove the results for 1.20c?

And as always, thank you very much for your great work!

[Stephan Busch]

Results are online of: CCM 1.20d(x) and CCMTest

[Black_Fox]

Results are online of: CCM 1.20d(x) and CCMTest

Nice! CCMTest seems still powerful enough (with exception of CD image )
Are you going to test PAQ8K in spite of its sloweness?

[Christian]

Thanks a bunch, Stephan!

Nice! CCMTest seems still powerful enough (with exception of CD image )

Im quite sure that I can fix the CD Image very easily (-100M) - at least I hope so.

[Stephan Busch]

Nope, Black_Fox, I won't test PAQ8k because it's too slow. Same with PAQ8HPx. Those 'speeds' are both highly unpractical and further improvements are too less.

[Stephan Busch]

Results of CCMTEST2 are now online

[Christian]

Thanks Stephan!

Hmm, CD Image isn't totally fixed - but I can live with that. Memory usage for CCMTEST2 is 41M. So, it's still 'slim'.

[Stephan Busch]

New main chart includes latest CCM 1.20f(x)

It is better and faster - very well done

[Matt Mahoney]

Nope, Black_Fox, I wont test PAQ8k because its too slow. Same with PAQ8HPx. Those speeds are both highly unpractical and further improvements are too less.

Actually PAQ8HPx are all faster than PAQ8L. But they are specialized for the Hutter prize and only compress well on English text.

I agree about PAQ8K being too slow (3x slower than PAQ8L). maximumcompression.com wont test it either. Thats why I didnt include those changes in PAQ8L. My goal was to keep the same speed as PAQ8JD. However, PAQ8K does get better compression on binary files like .exe than PAQ8L.

[Stephan Busch]

Yes, PAQ8K is too slow. I would test PAQ8HPx, but it is in deed for text files only. The lack of bitmap, jpeg and binary strength would
lead to still bad compression (compared to PAQ8L) on my testsets.
Nevertheless, I do honour and respect A. Ratushnyak's brilliant work.
And of course all the inventors and contributors that have made PAQ
to what it is now

[Stephan Busch]

Newest CCM 1.20g is included in newest benchmark.
BTW: I changed the index.html of my website. Is the site
displayed correctly on your systems? With Opera it seems to be ok, but I had problems to display it on IE7..

[Christian]

Thank you very much Stephan!

Btw. the website looks fine using Opera.

[Stephan Busch]

Hi folks,

I just wanted to pass a line in order to give a life sign.
All my Operating Systems were deleted and replaced by 64bit Vista;
unfortunately, this brought some network card driver problems and so I couldn't get online.. Besides that, dozens of applications had to be installed..
However; latest QuickLZ is already tested; Slug to follow today.
The Dolby Digital waveform was replaced by a more sophisticated extended waveform and an incompressible image was added to my lossless bitmap cmp tab.. All results will be published within the next 24 hours.. Thanks beforehand for your patience..


Cheers,
Stephan

[Christian]

Good to hear from you again. I'm looking forward to your next update!

[LovePimple]

Thanks for keeping us informed, Stephan!

[Stephan Busch]

Hi guys,

updated benchmark is now online. New entries are:

* CCM 1.21
* LZPM 0.03
* QuickLZ 1.30 beta 3
* Slug 1.1b
* Thor 0.94
* Tornado 0.1

Slug 1.1b is the fastest compressor of my testsets; LZPM has a huge performance increase; CCM 1.21 is 500 sec slower than CCM 1.20g; Thor is slightly better than 0.93; QuickLZ beta 3 is slightly
better but 13 sec slower than beta 1; Tornado 0.1 reached moderate 154. place and when selecting blocksize of 256m instead of 100m it performs worse on some testsets.

Some timings for compression are also included: PAQ1, Squeez 5.60, WinRK (ROLZ1) ans MSoftware Library. So far, CCM is the fastest compressor of the white PAQ class.

My website is now also reachable via www.squeezechart.com.

Regards,
Stephan

[Black_Fox]

Thanks Stephan!

CCM 1.21 is 500 sec slower than CCM 1.20g

While having exactly the same output?? Oh...

And congratulations to Chris for fastest compressor!

[Bulat Ziganshin]

"FreeARC 0.36 -md912MB -sGroupin" seems very stramge cmdline. i recommend you to use -m7 or -m6p with external ppmonstr.exe; also arc.groups should be in the same dir as arc.exe

last column at http://squeezechart.freehost.ag/main.html seems broken

[Bulat Ziganshin]

also, ppmvc was updated to 1.2 (he was fixed bug with my testset, so it's possible that your testset was be fixed too)

[Bulat Ziganshin]

also, can you please move encoding time to the left (close to totalsize)?

[Stephan Busch]

I moved encoding time to the left. Thanks for the suggestion.
Do you think I should also move the architecture columns to the right?

The right FreeARC switch is just -m7 (without memory switch and without grouping switch?

Thanks for the PPMVC hint - I will test it asap.

[encode]

updated benchmark is now online.

Thank you!

[Bulat Ziganshin]

I moved encoding time to the left.

thank you, it should be more readable (although main.html isnt yet updated)

Do you think I should also move the architecture columns to the right?

no, i dont think so. this info may be useful for archiver selection. i may only suggest exchanging "bit depth" and OS columns

The right FreeARC switch is just -m7 (without memory switch and without grouping switch?

yes. btw, -sGROUPING in help means description of solid groups (blocks) and has absolutely the same syntax as in rar. say, -se10m100f means grouping by extension, 10 mb or 100 files - whatever will be smaller

[Stephan Busch]

The main.html will be updated this evening. I am currently running
again FreeARC 0.36 with -m7 switch. If I understood that switch,
it calls ppmonstr - are the files stored in a solid archive and grouped together by extension? This mode seems to be far better than the former used switch, but I still miss TTA and bitmap encoder.
Those might be also interesting in an 7-ZipMOD. I also rerun the tests with WinRK 1.1.3 (PPMZ) - it is indeed as slow as PAQ..
BTW: Are you planning to give FreeArc the jpeg recompression feature? What I like most with FreeArc is the recursive feature so that I can take the unTARed testsets with simply putting Freearc files in my host dir C:Testsets and run it from there like tihs:

D:TESTSETS>timer arc a -m7 -r 3d.arc D:TESTSETSTEST_3DGame*.*

Timer 3.01 Copyright (c) 2002-2003 Igor Pavlov 2003-07-10
ARC 0.36 creating archive: 3d.arc
Compressed 2.287 files, 237.816.639 => 78.076.853 bytes. Ratio 32.8%
Directory 70.967 => 23.436 bytes. Ratio 33.0%
Compression time 324.58 secs, speed 733 kb/s. Total 414.77 secs
Kernel Time = 4.968 = 00:00:04.968 = 1%
User Time = 326.421 = 00:05:26.421 = 75%
Process Time = 331.390 = 00:05:31.390 = 76%
Global Time = 431.516 = 00:07:11.516 = 100%

.. most cmp must be copied into testset dir and started from there
which is not as user-friendly. Well, PPMonstr is used and one could state that this is an invention of someone else, but you seem to have made it capable to recurse dirs, to compress more than one file in an archive and to sort files by extension while original ppmonstr only compress single files (which will require to TAR them before compression) and lacks sort&solid features.

[Bulat Ziganshin]

.. most cmp must be copied into testset dir and started from there
which is not as user-friendly.

you can put compressors in directory mentioned in PATH or use fullpath to compressor:

D:TESTSETS> timer c:/testset/arc a c:/testset/arc a 3d.arc -r

freearc has 2 main lines of modes: asymmetric -m2x .. -m7x and symmetric: -m2 .. -m7. symmetric modes has faster and better compression while asymmetric modes has faster decompression with less memory reqs. it is like differences between uharcs -m3 (aymmetric) and -mz/-mx (symmetric) modes. because your goal is better and faster compression, there is no meaning in testing -m*x modes

also, there are two experimental modes involving ppmonstr.exe - -m5p and -m6p (numbers define memory usage, 1gb for any -m7*, 512mb for any -m6* and so on). so, best compression will be either in -m7 or -m6p mode

Well, PPMonstr is used and one could state that this is an invention of someone else

the whole idea of my program is providing good archiver interface to existing great compression libraries. you may not know, but arc now includes 7 algorithms - ppmd, lzma, grzip, bcj, lzp, dict and rep, of those only last two are my own. my part is to provide file handling, sorting, grouping, preprocessing and so on. in particular, you can see the following:

D:/Temp>arc a a -m6p --display
ARC 0.36 creating archive: a.arc using exe+rep:256mb:512+lzma:32mb:max:bt4:128, $obj => rep:256mb:512+lzma:32mb:max:bt4:128, $text => dict:64mb:l8192:m400:s100+lzp:64mb:85%:64:h22+pmm: 16:392808kb

here ppmonstr mentioned as "pmm". as you can see, data are splitted into text part and the rest, text data are preprocessed with dict and lzp and only then goes to ppmonstr. my own tests show that at the last end we got rather fast and good compression, much better than with ppmonstr alone. otoh, durilca includes its own aalgorithms of improving compression compared to bare ppmostr engine, so it may be very interesting comparison

I am currently running
again FreeARC 0.36 with -m7 switch. If I understood that switch,
it calls ppmonstr - are the files stored in a solid archive and grouped together by extension?

no, its a -m6p that calls ppmonstr. and of course, data sent to ppmonstr are preporocessed in the same way as data sent to ppmd in -m6 mode. ppmonstr just got large datafiles after sorting, splitting into blocks and preprocessing with dict+lzp. data are sorted by arc.groups, extension and so on and then splitted into 64mb chunks (it improves efficiency of dict preprocessor)

but I still miss TTA and bitmap encoder.
Those might be also interesting in an 7-ZipMOD

tta is finished, i dont plan bitmap encoder for the next version. Igor is aware of all my work (i send him letter each time i publish something) but its hard to convince him to use something, i prefer to just inform him

Are you planning to give FreeArc the jpeg recompression feature?

it will be great, but im far from understanding details of this recompression. it seems that i will implement table, executable and text preprocessors first. i still search for good bmp compression library, now jpeg-2000 is my best hit (there is free jpeg-2000 library)

[encode]

it will be great, but im far from understanding details of this recompression.

You can use PAQ7 or modified PAQ7 specially for JPG files!

[Bulat Ziganshin]

i think it is just too slow for my users