What's wrong with my testing script?

[m^2]

The problem are inconsistent results, sometimes there's ~2-3s lag added.

The test set is a snapshot of my currently developed programs, mostly text + version histories (=archives), additionally some exes. Total-1199 files, 25 803 208 bytes, quite a lot of repeated files (build scripts, license etc.).

I did 4 tests because of mistakes in archivers' switches.
1:

"qlz -1"               15037169  0.359 
"qlz -0"               14742577  0.406 
"qlz -2"               14568944  0.547 
"qlz -3"               14095700  1.000 
"FreeArc -m0"          25819371  1.532 
"FreeArc -m1"          13161354  1.578 
"FreeArc -mquick"      14476888  1.906 
"FreeArc -m1 skip"     11459499  2.281 
"FreeArc -mquick skip" 12759342  2.907 
"rar -m1"              13713984  3.125 
"FreeArc -m2"          10030764  3.141 
"FreeArc -m3"           9453453  7.266 
"FreeArc -m4"           9195635 11.313 
"FreeArc -m5"           9145910 16.359 
"FreeArc -m6"           9144054 16.610 
"FreeArc -m7"           9143300 16.391 
"FreeArc -m8"           9143129 16.219 
"FreeArc -m9"           9143123 16.219 
"rar -m2"              13175017 14.953

2:

"qlz -1"               15037169 0.375 
"qlz -0"               14742577 0.469 
"qlz -2"               14568944 0.562 
"qlz -3"               14095700 1.015 
"FreeArc -m0"          25819371 1.609 
"FreeArc -m1"          13161354 1.609 
"FreeArc -m1 skip"     11459499 2.250 
"FreeArc -mquick"      14476888 2.453 
"FreeArc -mquick skip" 12759342 2.907 
"rar -m1"              13713984 3.031 
"FreeArc -m2"          10030764 3.093 
"7z -mx=2"             11941046 6.969 
"7z -mx=1"             11941046 7.062 
"FreeArc -m3"           9453453 7.406 
"FreeArc -m4"           9195635 10.219 
"7z -mx=3"              9988324 10.938 
"rar -m2"              13175017 11.484 
"FreeArc -m9"           9143123 12.812 
"FreeArc -m7"           9143300 12.953 
"FreeArc -m8"           9143129 16.343 
"FreeArc -m5"           9145910 16.656 
"FreeArc -m6"           9144054 16.829

3:

"qlz -1"               15037169  0.375 
"qlz -0"               14742577  0.406 
"qlz -2"               14568944  0.547 
"qlz -3"               14095700  0.969 
"FreeArc -mquick"      14476888  1.266 
"FreeArc -m0"          25819371  1.610 
"FreeArc -m1"          13161354  1.750 
"FreeArc -m1 skip"     11459499  2.297 
"FreeArc -m2"          10030764  3.125 
"rar -m1"              13713984  3.078 
"FreeArc -mquick skip" 12759342  3.531 
"7z -mx=1"             11941046  7.062 
"7z -mx=2"             11941046  7.094 
"FreeArc -m3"           9453453  7.687 
"7z -mx=3"              9988324 10.266 
"FreeArc -m4"           9195635 11.047 
"FreeArc -m8"           9143129 12.891 
"FreeArc -m6"           9144054 13.063 
"rar -m2"              13175017 13.594 
"FreeArc -m5"           9145910 14.907 
"FreeArc -m7"           9143300 16.125 
"FreeArc -m9"           9143123 16.500

4:

"qlz -1"               15037169 0.407 
"qlz -0"               14742577 0.469 
"qlz -2"               14568944 0.609 
"NanoZip -cf"          12587260 1.000 
"qlz -3"               14095700 1.015 
"NanoZip -cn"          25857726 1.031 
"FreeArc -m0"          25819371 1.469 
"NanoZip -cF"          10845954 1.484 
"FreeArc -m1"          13161354 1.641 
"FreeArc -mquick"      14476888 1.719 
"NanoZip -cd"          10184506 1.938 
"FreeArc -m1 skip"     11459499 2.344 
"FreeArc -mquick skip" 12759342 2.984 
"FreeArc -m2"          10030764 3.078 
"rar -m1"              13713984 3.093 
"NanoZip -cD"           9816465 3.594 
"7z -mx=2"             11941046 6.984 
"7z -mx=1"             11941046 7.125 
"FreeArc -m3"           9453453 7.297 
"7z -mx=3"              9988324 10.047 
"FreeArc -m4"           9195635 10.296 
"rar -m2"              13175017 11.687 
"FreeArc -m7"           9143300 12.735 
"FreeArc -m9"           9143123 12.859 
"FreeArc -m5"           9145910 12.938 
"FreeArc -m6"           9144054 13.313 
"FreeArc -m8"           9143129 16.969 
"NanoZip -co"           9779789 22.672 
"NanoZip -cO"           9064783 84.453 
"NanoZip -cc"           8789846 128.969

The problem occurs usually with FreeArc -m5..-m9, but also with rar -m2
BTW FreeArc does great, but NanoZip...

[nanoflooder]

Did you also include memory switches for each run? Like, -mXXXXm for NZ?

[m^2]

No.

[nanoflooder]

Well, did you do that on purpose? Because that may be the key to your problem. I don't remember how much memory does NZ use by default, but it seems like it's far from much enough for it to show good compression results.

[m^2]

Well, did you do that on purpose? Because that may be the key to your problem. I don't remember how much memory does NZ use by default, but it seems like it's far from much enough for it to show good compression results.

So you think that NZ results are poor? Because I find them extraordinary. cf cF cd and cD are give the smallest results when compared to equally fast / faster compressors. If you say that it may be even faster, I'll try to tweak it later.

And what I call problem is that tests don't give repeatable results.

[Intrinsic]

Some variance is to be expected. But even a slight bit of disk access from other tasks can screw the results. I'm guessing nothing else is running when these tests are in progress? ie not even moving the mouse etc? i turn off eveything that isn't needed, esp virus checkers as they'll hurt the performance a lot potentially. I even have an absolute bare bones install of XP so i know nothing will interfere with my tests, most services disabled, no stupid tasks like crappy sound driver apps that system tray etc. Only thing i'll have running is Directory Opus which will be running my test scripts and ATI Tray Tools. All visual crap turned off(set to best performance in the system properties, no swap file(disk thrashing will shaft your results if it starts paging anything out)

[pat357]

So you think that NZ results are poor? Because I find them extraordinary. cf cF cd and cD are give the smallest results when compared to equally fast / faster compressors. If you say that it may be even faster, I'll try to tweak it later.

For highly redundant data you might try :

nz a -cD m1.5g ...  (1,5Gb memory, you can try even higher settings)
arc a -mexe+delta+lzma:1gb:ht4:128:mc128 (also try higher mc-values, like mc256 or mc512 and higher fb-values : 192 even 273 instead of 128))

And what I call problem is that tests don't give repeatable results.

Are these "wall" times or process times ?
Better use process times, measured by Igor's TIMER.
How much RAM does your system have ?
What is the largest available consistent memory block you have before you start testing FA ?

[m^2]

Did some testing today.
Machine: Pentium D 805@2.66 Ghz, 1 GB RAM, XP x64.

"qlz -1"                                     15037169   0.359
"qlz -0"                                     14742577   0.390
"4x4 1"                                      17555999   0.438 
"thor e1"                                    15002518   0.453 
"4x4 2"                                      14160760   0.500 
"qlz -2"                                     14568944   0.547
"4x4 3"                                      13164904   0.547 
"thor e2"                                    14127036   0.672 
"NanoZip -cn"                                25857726   0.656
"4x4 1 -p128"                                17555999   0.781 
"thor e3"                                    12622556   0.844 
"NanoZip -cf"                                12587260   0.922
"NanoZip -cf -m700m"                         12587260   0.938 
"qlz -3"                                     14095700   0.969 
"slug"                                       12092726   1.110  
"FreeArc -m0"                                25819371   1.141 
"4x4 4"                                      10988868   1.437 
"NanoZip -cF"                                10845954   1.438  
"NanoZip -cF -m700m"                         10845954   1.438 
"FreeArc -m1"                                13161354   1.453   
"FreeArc -mquick"                            14476888   1.656 
"NanoZip -cd -m700m"                         10184506   1.860 
"NanoZip -cd"                                10184506   1.890 
"4x4 5"                                      10351629   1.937 
"FreeArc -m1 -ms"                            11459499   2.203  
"thor e5"                                    13112840   2.375 
"Tornado c2 x1"                              11343994   2.438 
"Tornado c1 x1"                              12615810   2.453 
"Tornado c4 x1"                              10300248   2.766 
"Tornado c3 x1"                              10475534   2.813 
"FreeArc -mquick -ms"                        12759342   2.844  
"rar -m1"                                    13713984   2.984  
"FreeArc -m2"                                10030764   3.016  
"thor e4"                                    10584124   3.219 
"4x4 6"                                      10169667   3.351 
"PAQ8p -0"                                   25860111   3.406
"XCOPY"                                      25819371   3.469
"NanoZip -cD -m700m"                          9816465   3.500 
"NanoZip -cD"                                 9816465   3.516   
"Tornado c1 x2"                              12696681   3.750 
"Tornado c2 x2"                              11402122   3.813 
"Tornado c3 x2"                              10519150   4.157 
"Tornado c4 x2"                              10345153   4.172 
"4x4 1 -p1024"                               17555999   4.234 
"4x4 7"                                      10030889   5.500 
"7z -mx=2"                                   11941046   6.969  
"7z -mx=1"                                   11941046   6.985
"FreeArc -m3"                                 9453453   7.266  
"4x4 2 -p1024"                               14160760   7.891 
"4x4 1t"                                     11057086   8.172 
"4x4 3 -p1024"                               13164904   8.329 
"4x4 8"                                       9666401   8.875 
"bliz f"                                     10846485   9.235 
"7z -mx=3"                                    9988324  10.015 
"FreeArc -m4"                                 9195635  10.219 
"7z -mx=4"                                    9988324  10.312 
"4x4 3t"                                     10957522  11.079
"7z -mx=6"                                    9464474  11.312 
"rar -m2"                                    13175017  11.312 
"7z -mx=5"                                    9464474  11.547 
"4x4 4 -p1024"                               10988732  11.562 
"CTXf -mq"                                   12968143  11.625 
"bliz c"                                     10806183  11.797 
"CTXf -mf"                                   12763662  12.188 
"FreeArc -m5"                                 9145910  12.641 
"FreeArc -m7"                                 9143300  12.735 
"FreeArc -m9"                                 9143123  12.812 
"FreeArc -m8"                                 9143129  12.891
"rar -m3"                                    12933414  12.953 
"7z -mx=8"                                    9396262  13.000 
"FreeArc -m6"                                 9144054  13.063 
"7z -mx=7"                                    9441597  13.078 
"m1"                                         13441371  13.547
"rar -m4"                                    12747964  13.750 
"4x4 2t"                                     10943913  14.015 
"CTXf -mq -b"                                11394873  14.125 
"rar -m5"                                    12730870  14.172 
"4x4 5 -p1024"                               10352191  14.406 
"4x4 4t"                                     10494034  14.579 
"CTXf -mf -b"                                11018242  14.750 
"4x4 9"                                       9397328  16.500 
"4x4 10"                                      9377747  20.546 
"NanoZip -co"                                 9779789  22.672 
"ccm 0"                                       9371713  23.469 
"SBC -m1 -b63 -ha"                           11797160  24.750
"rzm"                                         9139279  43.437 
"FreeArc -mexe+delta+lzma:1gb:ht4:128:mc128"  9218779  67.703 
"FreeArc -mexe+delta+lzma:1gb:ht4:192:mc128"  9216363  69.734 
"NanoZip -cO"                                 9064783  84.453 
"FreeArc -mexe+delta+lzma:1gb:ht4:128:mc256"  9224229 124.032 
"FreeArc -mexe+delta+lzma:1gb:ht4:192:mc256"  9222238 126.563 
"NanoZip -cc"                                 8789846 128.969 
"FreeArc -mexe+delta+lzma:1gb:ht4:128:mc512"  9259001 237.906 
"FreeArc -mexe+delta+lzma:1gb:ht4:192:mc512"  9258881 241.531 
"PAQ8p -1"                                   10232338 633.125 
"PAQ8p -2"                                    9955560 646.235 
"PAQ8p -3"                                    9063162 651.203 

4x4 0.2a
7z 4.57
Blizzard 0.24b
ccm 1.30c
FreeArc 0.5a-230608
m1 0.0
NanoZip 0.04
qlz 1.31
rar 3.80b4
RZM 0.07h
SBC 0.970
slug 1.27b
THOR 0.96 alpha
Tornado 0.4

All tests done at least 3 times, the best result showed(*).
Compilation of the best results:

"qlz -1"               15037169   0.359
"qlz -0"               14742577   0.390
"4x4 2"                14160760   0.500 
"4x4 3"                13164904   0.547
"thor e3"              12622556   0.844 
"NanoZip -cf"          12587260   0.922
"slug"                 12092726   1.110   
"4x4 4"                10988868   1.437 
"NanoZip -cF"          10845954   1.438   
"NanoZip -cd -m700m"   10184506   1.860  
"FreeArc -m2"          10030764   3.016  
"NanoZip -cD -m700m"    9816465   3.500  
"FreeArc -m3"           9453453   7.266  
"FreeArc -m4"           9195635  10.219  
"FreeArc -m5"           9145910  12.641
"FreeArc -m7"           9143300  12.735 
"FreeArc -m9"           9143123  12.812 
"rzm"                   9139279  43.437 
"NanoZip -cO"           9064783  84.453 
"NanoZip -cc"           8789846 128.969

You may notice some unusual entries like "4x4 2 -p1024" - compression with 1024 threads. I wanted to see how does it affect compression ratio...and it has almost no effect, does 4x4 use slicing data into p parts?

Some variance is to be expected. But even a slight bit of disk access from other tasks can screw the results. I'm guessing nothing else is running when these tests are in progress? ie not even moving the mouse etc? i turn off eveything that isn't needed, esp virus checkers as they'll hurt the performance a lot potentially. I even have an absolute bare bones install of XP so i know nothing will interfere with my tests, most services disabled, no stupid tasks like crappy sound driver apps that system tray etc. Only thing i'll have running is Directory Opus which will be running my test scripts and ATI Tray Tools. All visual crap turned off(set to best performance in the system properties, no swap file(disk thrashing will shaft your results if it starts paging anything out)

There was no AV or other services that could look at archivers' hands, nothing CPU intensive and nothing that I expect to write to disk, but the system wasn't clean. Firefox, Miranda, Total Commander, TextPad (no saves/autosaves during tests). I was moving my mouse.
I know that some variance is usual, but this is really a lot, I really don't think that's it. But I have nothing to loose, I'll try a clean OS, maybe tomorrow.

For highly redundant data you might try :

nz a -cD m1.5g ...  (1,5Gb memory, you can try even higher settings)
arc a -mexe+delta+lzma:1gb:ht4:128:mc128 (also try higher mc-values, like mc256 or mc512 and higher fb-values : 192 even 273 instead of 128))

Are these "wall" times or process times ?
Better use process times, measured by Igor's TIMER.
How much RAM does your system have ?
What is the largest available consistent memory block you have before you start testing FA ?

LZMA doesn't work well here...too much text, most of the rest is not compressible anyway.

I didn't test fb 273, I tested 276 instead (mistake ) and got errors. It's way too slow to be practical and not expected to perform well anyway, so I'll skip this.
Nanozip picks memory well, results are exactly the same.

What are "wall" times? I use Igor's timer.
How can I know what is the largest consistent memory block in my system ?? And how does it matter? Isn't Windows' virtual memory smart enough to allocate inconsistent blocks? All I can tell is that I have ~600-700 MB free and I remember seeing one archiver claiming to allocate ~800MB.

(*)Arc -mquick has one result 1.266, seven ~1.7-1.8 and one 2.4xx. I guess something was wrong with the extreme times and picked the second best.

Thank you all for suggestions.

[m^2]

There was so much to write that I forgot to ask about one effect that I don't understand.
With all tests, my batches start with compressing data twice with qlz, just to fill system file cache. However, I noticed that if I run XCOPY on the test set, speed of archiving without compression is greatly reduced. With compression - no effect at all.

[Intrinsic]

Just remember that any activity that takes focus from a task will draw cpu power away from it, even if it runs on one core while you use another. So i just set stuff running and leave the comp todo it's thing which i watch TV

And yep it's good you run it at least twice, i think most of us do that too, or even just dump it out to /dev/null(or windows equiv) once then kick it off for real.

All I can tell is that I have ~600-700 MB free and I remember seeing one archiver claiming to allocate ~800MB.

If that's the case it'll seriously effect the processing time, as it'll page stuff out of memory to make room. Before i got my new comp i was running at the same amount of free mem on one of my more "busy"(read: full of crap) XP installs, so i only let(where possible) a method use 512mb at max to reducing disk thrashing from paging.

[m^2]

Just remember that any activity that takes focus from a task will draw cpu power away from it, even if it runs on one core while you use another. So i just set stuff running and leave the comp todo it's thing which i watch TV

I really don't believe that such light usage has measurable impact on single threaded compressors, unless there's some HDD usage that I don't know about. But, as I said, I'll do some tests. My dog is gonna like it.

If that's the case it'll seriously effect the processing time, as it'll page stuff out of memory to make room. Before i got my new comp i was running at the same amount of free mem on one of my more "busy"(read: full of crap) XP installs, so i only let(where possible) a method use 512mb at max to reducing disk thrashing from paging.

Good point, I'll care more in the future.
The archiver which used this memory was CCM with -6, (-7, 1.2 GB works too), not benchmarked, probably nothing on the list is affected.

[m^2]

Some FreeArc tests (I did it yesterday, not clean machine yet..):

"FreeArc -m0"                                25819371   1.141 
  
"FreeArc -m1 -ms"                            11459499   2.203
"FreeArc -m1 -ma-2 -ms"                      11459499   2.125 
"FreeArc -m1 -ma- -ms"                       11543101   2.140  
"FreeArc -m1"                                13161354   1.453 
"FreeArc -m1 -ma-2"                          13161354   1.406 
"FreeArc -m1 -ma-"                           13214237   1.437   

"FreeArc -m2"                                10030764   3.016 
"FreeArc -m2 -ma-2 -ms"                      10084297   3.719 
"FreeArc -m2 -ms"                            10107256   4.093 
"FreeArc -m2 -ma-2"                          10121896   2.969 
"FreeArc -m2 -ma- -ms"                       10168296   3.734   
"FreeArc -m2 -ma-"                           10203831   2.984  

"FreeArc -m3 -ms"                             9445514   8.453 
"FreeArc -m3"                                 9453453   7.266 
"FreeArc -m3 -ma-2 -ms"                       9500774   8.532 
"FreeArc -m3 -ma- -ms"                        9507177   8.547 
"FreeArc -m3 -ma-2"                           9519822   7.781 
"FreeArc -m3 -ma-"                            9525659   7.843  

"FreeArc -m4"                                 9195635  10.219 
"FreeArc -m4 -ms"                             9210841  10.188 
"FreeArc -m4 -ma-2 -ms"                       9272311  10.875 
"FreeArc -m4 -ma-2"                           9272529  10.844 
"FreeArc -m4 -ma- -ms"                        9277863  10.877
"FreeArc -m4 -ma-"                            9278056  10.844  

"FreeArc -m5 -ms"                             9135677  13.141 
"FreeArc -m5"                                 9145910  12.641 
"FreeArc -m5 -ma-2"                           9172643  12.140
"FreeArc -m5 -ma- -ms"                        9174156  12.344
"FreeArc -m5 -ma-2 -ms"                       9177111  12.281 
"FreeArc -m5 -ma-"                            9178013  12.360
 
"FreeArc -m6 -ms"                             9133877  13.188 
"FreeArc -m6"                                 9144054  13.063
"FreeArc -m6 -ma-2"                           9170657  13.906 
"FreeArc -m6 -ma- -ms"                        9172243  12.657
"FreeArc -m6 -ma-2 -ms"                       9175141  12.593
"FreeArc -m6 -ma-"                            9176091  14.171

"FreeArc -m7 -ms"                             9133065  13.297 
"FreeArc -m7"                                 9143300  12.735 
"FreeArc -m7 -ma-2"                           9169918  14.391 
"FreeArc -m7 -ma- -ms"                        9171488  12.515
"FreeArc -m7 -ma-2 -ms"                       9174396  12.938 
"FreeArc -m7 -ma-"                            9175335  14.375 

"FreeArc -m8 -ms"                             9132916  13.500 
"FreeArc -m8"                                 9143129  12.891
"FreeArc -m8 -ma-2"                           9169755  14.375 
"FreeArc -m8 -ma- -ms"                        9171332  12.547
"FreeArc -m8 -ma-2 -ms"                       9174233  13.281 
"FreeArc -m8 -ma-"                            9175177  14.203 

"FreeArc -m9 -ms"                             9132900  13.203 
"FreeArc -m9"                                 9143123  12.812 
"FreeArc -m9 -ma-2"                           9169761  14.187 
"FreeArc -m9 -ma- -ms"                        9171322  12.610
"FreeArc -m9 -ma-2 -ms"                       9174243  13.312 
"FreeArc -m9 -ma-"                            9175162  14.172 

"FreeArc -mx -ms"                             9132900  13.266 
"FreeArc -mx"                                 9144779  19.610 
"FreeArc -mx -ma-2"                           9169761  14.188
"FreeArc -mx -ma- -ms"                        9171322  12.625
"FreeArc -mx -ma-2 -ms"                       9174243  13.453 
"FreeArc -mx -ma-"                            9175162  14.203 

"FreeArc -max -ms"                            9049012  20.593 
"FreeArc -max"                                9060380  22.125
"FreeArc -max -ma-2"                          9085152  20.860  
"FreeArc -max -ma- -ms"                       9088168  19.985
"FreeArc -max -ma-2 -ms"                      9089205  19.703 
"FreeArc -max -ma-"                           9091297  21.156 

"FreeArc -mexe+delta+lzma:1gb:ht4:128:mc128"  9218779  67.703 
"FreeArc -mexe+delta+lzma:1gb:ht4:128:mc256"  9224229 124.032 
"FreeArc -mexe+delta+lzma:1gb:ht4:128:mc512"  9259001 237.906 
"FreeArc -mexe+delta+lzma:1gb:ht4:192:mc128"  9216363  69.734 
"FreeArc -mexe+delta+lzma:1gb:ht4:192:mc256"  9222238 126.563 
"FreeArc -mexe+delta+lzma:1gb:ht4:192:mc512"  9258881 241.531 

"FreeArc -mquick"                            14476888   1.656 
"FreeArc -mquick -ms"                        12759342   2.844

I merged old and new results.
Notation: -ma-2 is -ma- switch with a tweaked arc.groups (general tweaks, not for this particular test set) that I posted in the FreeArc topic.
Conclusions:
- -ms improves compression ratio here, except for -ma-2 in modes 5+. That's because -ma-2 correctly detects some sqx archives that happen to be compressible further.
- -ma- and -ma-2 are equally fast and ~0.5s faster than detection by content.
- -ma-2 has slightly better compression ratio than -ma- (<1%), automatic detection works best, but the difference is slight too.
- in faster modes -ms slows compression down. In slower - speeds up.
- -m5..9/x are almost the same.

Updated list of the most efficient programs / modes:

"qlz -1"                15037169   0.359
"qlz -0"                14742577   0.390
"4x4 2"                 14160760   0.500 
"4x4 3"                 13164904   0.547
"thor e3"               12622556   0.844 
"NanoZip -cf"           12587260   0.922
"slug"                  12092726   1.110   
"4x4 4"                 10988868   1.437 
"NanoZip -cF"           10845954   1.438   
"NanoZip -cd -m700m"    10184506   1.860  
"FreeArc -m2 -ma-2"     10121896   2.969 
"FreeArc -m2"           10030764   3.016  
"NanoZip -cD -m700m"     9816465   3.500  
"FreeArc -m3"            9453453   7.266 
"FreeArc -m3 -ms"        9445514   8.453  
"FreeArc -m4 -ms"        9210841  10.188 
"FreeArc -m4"            9195635  10.219 
"FreeArc -m5 -ma-2"      9172643  12.140  
"FreeArc -m7 -ma- -ms"   9171488  12.515
"FreeArc -m8 -ma- -ms"   9171332  12.547 
"FreeArc -m5"            9145910  12.641
"FreeArc -m7"            9143300  12.735 
"FreeArc -m9"            9143123  12.812
"FreeArc -m5 -ms"        9135677  13.141  
"FreeArc -m6 -ms"        9133877  13.188
"FreeArc -m9 -ms"        9132900  13.203 
"FreeArc -max -ma-2 -ms" 9089205  19.703 
"FreeArc -max -ma- -ms"  9088168  19.985
"FreeArc -max -ms"       9049012  20.593  
"NanoZip -cc"            8789846 128.969

[Intrinsic]

Yeah i only used CCM with the 274Mb used option(-4 i think?) Otherwise it'll grind away heh.

[m^2]

Clean machine didn't help :/
I did most tests 3 more times and additionally some more:

"CCM 0"       9371713  22.641 
"CCM 1"       9293769  23.266 
"CCM 2"       9185369  23.563 
"CCM 3"       9162288  24.172
"CCM 4"       9148813  24.781 
"CCM 5"       9140366  25.422 
"CCM 6"       9135885  26.094
"CCMX 0"      9372462  26.250  
"CCMX 1"      9290740  27.094 
"CCMX 2"      9179047  27.984 
"CCMX 3"      9152385  29.063 
"CCMX 4"      9134386  29.672 
"CCMX 5"      9121118  30.329 
"CCMX 6"      9112756  31.000 

"SQX -m1"    10322898   8.672 
"SQX -m2"    10303146   9.250 
"SQX -m3"    10296038   9.843 
"SQX -m4"    10112262  20.578 
"SQX -m5"     9925270  24.843 
"SQX -mx"     9968430  23.406 

"BriefLZ"    15652257   1.422 

"lzop -1 -F" 15009233   0.640 
"lzop -1"    15009645   0.672 
"lzop -2 -F" 15035957   0.703 
"lzop -2"    15036369   0.656 
"lzop -3 -F" 15035957   0.640 
"lzop -3"    15036369   0.656 
"lzop -4 -F" 15035957   0.625 
"lzop -4"    15036369   0.671 
"lzop -5 -F" 15035957   0.641
"lzop -5"    15036369   0.703 
"lzop -6 -F" 15035957   0.625 
"lzop -6"    15036369   0.719
"lzop -7 -F" 13757820   4.890 
"lzop -7"    13757820   4.968 
"lzop -8 -F" 13720012   8.907 
"lzop -8"    13720012   9.000 
"lzop -9 -F" 13715518  10.844 
"lzop -9"    13715518  10.921

[pat357]

Nanozip picks memory well, results are exactly the same.

With only 1GB RAM, NZ with -m1.5g has no sense of course
NZ will probably use all the available RAM on your system, limited to the largest available consistent memory block. (see below)

What are "wall" times?

Wall times are like the "global time" displayed by Igor's timer.

How can I know what is the largest consistent memory block in my system ??

Look at http://forall.ru-board.com/egor23/on...al_Memory.html
There is a tool "memo2g". thats can show you this value.
(page is in Russian, you can use Google to translate (like I did )

And how does it matter? Isn't Windows' virtual memory smart enough to allocate inconsistent blocks?

Unfortunately windows doesn't do this well.
Lzma with large dictionary (or in case of NZ, BWT with larger blocks) will handle redundant data (duplicate or similar files in this test) much better.
The maximum value for the dictionary is however limited by the largest available consistent memory block. If your memory gets fragmented, the max dictionary will decrease and you might get worse compression for redundant data. (like your test-data)
To see the influence of a larger dict on redundant data, you could try the following :
try to compress 2 identical files from about 100Mb ( a 100Mb .zip, rar or any compressed file will show this effect even better)
use your favorite archivers to compress these 2 files

QLZ
RAR -m2 .. -m5 
7z 
arc a -mlzma:1gb:ht4:16:mc8
arc m2 m3  and max
NZ -cd .. cO with m50m ,m200m, m300m
CCM 3 and 5

How did RAR perform compared to the other ?
What is the ratio form NZ -m50m compared to m300m ?

[m^2]

Wall times are like the "global time" displayed by Igor's timer.

Process time measures only CPU time, right? I wanted it to be a real life test that takes into account HDD handling too and therefore - I measured global times.

Look at http://forall.ru-board.com/egor23/on...al_Memory.html
There is a tool "memo2g". thats can show you this value.
(page is in Russian, you can use Google to translate (like I did )

Unfortunately windows doesn't do this well.

Sadly, google (altavista too) barely knows russian..and this coupled with my not so good english makes this article hard to understand. But I guess that the issue is not inability to allocate non continuous block of physical memory but lack of free continuous block in process' address space, right?

Lzma with large dictionary (or in case of NZ, BWT with larger blocks) will handle redundant data (duplicate or similar files in this test) much better.
The maximum value for the dictionary is however limited by the largest available consistent memory block. If your memory gets fragmented, the max dictionary will decrease and you might get worse compression for redundant data. (like your test-data)
To see the influence of a larger dict on redundant data, you could try the following :
try to compress 2 identical files from about 100Mb ( a 100Mb .zip, rar or any compressed file will show this effect even better)
use your favorite archivers to compress these 2 files

QLZ
RAR -m2 .. -m5 
7z 
arc a -mlzma:1gb:ht4:16:mc8
arc m2 m3  and max
NZ -cd .. cO with m50m ,m200m, m300m
CCM 3 and 5

How did RAR perform compared to the other ?
What is the ratio form NZ -m50m compared to m300m ?

This test set has just 25MB, so there's no need for huge dictionary.
Generally, I know what large dictionaries are for, no need to test it. At least with LZ*.
I don't know CM, I briefly looked at LPAQ sources once and it seemed to handle memory in a totally different way, right?

[pat357]

the issue is not inability to allocate non continuous block of physical memory but lack of free continuous block in process' address space, right?

Indeed, that was my point.
Have you tried to run "memo2g.exe" ?

I don't know CM, I briefly looked at LPAQ sources once and it seemed to handle memory in a totally different way, right?

Larger memory model for CM will also improve compression for redundant data, just like it's for LZ* based compression.

[m^2]

Indeed, that was my point.
Have you tried to run "memo2g.exe" ?

Larger memory model for CM will also improve compression for redundant data, just like it's for LZ* based compression.

I wouldn't believe if you said otherwise, I know how much do the biggest PAQ modes use:P
But does huge amount of memory help on small redundant data?
memo2g? 1.7 GB/

[pat357]

I wouldn't believe if you said otherwise, I know how much do the biggest PAQ modes use:P
But does huge amount of memory help on small redundant data?

I don't think so. Once the whole file fits in the "memory model", increasing memory settings doesn't help anymore for gaining ratio's. Of course the overall-ratio might still increase due better compression of all the data, but this is not specific for "duplicate files".

memo2g?

http://www.haskell.org/bz/memo.7z (I think Bulat has something to do with this little tool)

[m^2]

I don't think so. Once the whole file fits in the "memory model", increasing memory settings doesn't help anymore for gaining ratio's. Of course the overall-ratio might still increase due better compression of all the data, but this is not specific for "duplicate files".

Thanks.

http://www.haskell.org/bz/memo.7z (I think Bulat has something to do with this little tool)

It was meant to be:
You ask for memo2g result? You got memo2g result.
1.7 GB.

[m^2]

New archivers tested.

BALZ 1.15:

BALZ e  	9869984	14.640
BALZ ex 	9798204	24.359

BCM 0.02:

BCM	11554980	562.312	Tested only once, some variation expected

BIT 0.6:

BIT fastest	11036333	88.797
BIT fast	10630360	90.031
BIT normal	10208519	92.531

BZP 0.1:

BZP	12541629	3.640

CMM4 0.2b:

CMM4 11 	10244446	53.281
CMM4 55  	 9141866	61.422

Dark 0.51:

Dark	12581652	13.828

FastLZ 0.1:

FastLZ -2	15479416	0.359
FastLZ opt -2	15693750	0.359
FastLZ opt -1	16078104	0.375
FastLZ -1	16041638	0.390

LPAQ2:

LPAQ2 1 	10575032	90.313	
LPAQ2 3 	 9401627	91.719	
LPAQ2 2 	10298349	92.953	
LPAQ2 4 	 9306369	95.938	
LPAQ2 5 	 9200271	102.204	
LPAQ2 6 	 9171147	107.219	
LPAQ2 7 	 9155945	112.578

LZSS 0.01:

LZSS e  	14779580	4.984
LZSS ex 	14135294	161.796

LZTurbo 0.92:

LZTurbo -11	14524917	0.328	
LZTurbo -21	14170363	0.375	
LZTurbo -31	14640126	0.484	
LZTurbo -41	12817012	0.828	
LZTurbo -22	10930618	1.328	
LZTurbo -12	12066470	1.328	
LZTurbo -32	11393232	1.406	
LZTurbo -13	11715839	1.641	
LZTurbo -23	10588677	1.687	
LZTurbo -33	11065427	1.765	
LZTurbo -42	10986725	2.016	
LZTurbo -43	10656955	2.313	
LZTurbo -14	11633415	2.390	
LZTurbo -24	10515376	2.421	
LZTurbo -34	10995408	2.453	
LZTurbo -15	11584213	2.985	
LZTurbo -25	10473965	3.015	
LZTurbo -44	10583022	3.032	
LZTurbo -35	10954957	3.125	
LZTurbo -45	10540015	3.672	
LZTurbo -16	11554016	4.094	
LZTurbo -26	10448628	4.250	
LZTurbo -36	10930302	4.422	
LZTurbo -46	10512364	4.938	
LZTurbo -51	13157820	5.250	
LZTurbo -52	11781629	5.875	
LZTurbo -17	11534561	6.047	
LZTurbo -27	10432261	6.313	
LZTurbo -53	11496804	6.344	
LZTurbo -37	10914230	6.438	
LZTurbo -47	10494484	7.000	
LZTurbo -54	11430794	7.172	
LZTurbo -55	11393041	7.984	
LZTurbo -56	11369111	9.094	
LZTurbo -57	11353115	11.344	
LZTurbo -18	11518667	14.688	
LZTurbo -28	10418632	15.532	
LZTurbo -38	10900408	15.547	
LZTurbo -48	10477226	16.032	
LZTurbo -58	11338074	20.344	
LZTurbo -19	11295390	50.719	
LZTurbo -29	10318945	53.234	
LZTurbo -39	10785274	53.781	
LZTurbo -49	10253181	54.516	
LZTurbo -59	11056091	58.812

qpress 0.14 BETA:

qpress -L1	16004979	1.235	
qpress -L2	15029740	1.547	
qpress -L3	14743422	2.734	
qpress -BL1	16004979	7.343	
qpress -BL2	15029740	7.625	
qpress -BL3	14743422	8.734

QUAD 1.12:

QUAD	10398797	9.781

Updated "The best" list:

LZTurbo -11           14524917   0.328
LZTurbo -21           14170363   0.375
4x4 2                 14160760   0.500 
4x4 3                 13164904   0.547
LZTurbo -41           12817012   0.828
thor e3               12622556   0.844 
NanoZip -cf           12587260   0.922
slug                  12092726   1.110 
LZTurbo -22           10930618   1.328
NanoZip -cF           10845954   1.438  
LZTurbo -23           10588677   1.687
NanoZip -cd -m700m    10184506   1.860  
FreeArc -m2 -ma-2     10121896   2.969 
FreeArc -m2           10030764   3.016  
NanoZip -cD -m700m     9816465   3.500  
FreeArc -m3            9453453   7.266 
FreeArc -m3 -ms        9445514   8.453  
FreeArc -m4 -ms        9210841  10.188 
FreeArc -m4            9195635  10.219 
FreeArc -m5 -ma-2      9172643  12.140  
FreeArc -m7 -ma- -ms   9171488  12.515
FreeArc -m8 -ma- -ms   9171332  12.547 
FreeArc -m5            9145910  12.641
FreeArc -m7            9143300  12.735 
FreeArc -m9            9143123  12.812
FreeArc -m5 -ms        9135677  13.141  
FreeArc -m6 -ms        9133877  13.188
FreeArc -m9 -ms        9132900  13.203 
FreeArc -max -ma-2 -ms 9089205  19.703 
FreeArc -max -ma- -ms  9088168  19.985
FreeArc -max -ms       9049012  20.593  
NanoZip -cc            8789846 128.969