DARK - a new BWT-based command-line archiver

NO PROBLEMO

Thank You!


But you can exclude the archive name from file list!

Thanks Ilia!


But there is one question.
Is it acceptable that dark will try to compress it's output file (and crash) when you try "dark c q.dark *.*"?

Can you do what Ilia has suggested?

Wow! I've made some super feature - the fragmentator!
Wildcards are supported too
Please, read news on http://darchiver.narod.ru
Last version dark-0.39

dark 0.39 failed on enwik9 from http://cs.fit.edu/~mmahoney/compression/text.html

Command was: dark c -b128m enwik9.dark enwik9

Tested on a 2.2 GHz Athlon 64 with 1 GB memory under WinXP home.
After several minutes I had 15 GB of zero bytes in enwik9.dark before I killed it. Works OK on enwik8 (verified decompression).

Wow! I've made some super feature - the fragmentator!
Wildcards are supported too
Please, read news on http://darchiver.narod.ru
Last version dark-0.39

Thank You!

I missed the last couple of updates:

ver '0.38c': entropy coder now works in solid mode
ver '0.38d': added wildcards support

Could you please make the above two versions available for download for a couple of days as I would like to keep my collection complete.

In the future would you please keep a copy of the previous version on your site along with the latest.

Thanks!

[spark]

I missed the last couple of updates:

I was thinking the same thing.. 'We want historic versions!', kvark

I'm missing the following versions:
0.30b
0.31b
0.33b
0.34a,b
0.35a
0.38a,c,d

yes, I came to the party a bit late.. :-p

Party?
I'm sorry, but I haven't all this versions myself
When I add/change the latter (e.g. 0.33b), I modify the source a little and don't need any backup. So you can download only 'final' versions. But I've found some of those you requested:
dark-0.30-beta
dark-0.34a-beta
dark-0.38d.zip
They are availible for downloading. To be honest, I didn't think you can be SO interested in it... I'll keep all the versions in futere.

Thanx,Matt! It's a pity dark-0.39 falls on enwik9, because I haven't any possibility to download it and test on my own I'll try to fix 'something' as soon as possible.

> In the future would you please keep a copy of the previous
> version on your site along with the latest.
OK, good advice!

I'm testing the fragmentator now. There are interesting results when compressing 'tarred' folders...

And a little special:
dark-0.32a-beta
dark-0.32b-beta
dark-0.33-beta
dark-0.34a-beta (!!)
dark-0.37b (~0.38)

When I add/change the latter (e.g. 0.33b), I modify the source a little and don't need any backup. So you can download only 'final' versions. But I've found some of those you requested:
dark-0.30-beta
dark-0.34a-beta
dark-0.38d.zip
They are availible for downloading

Where? I can't find them anyhere on the Dark home page!


I didn't think you can be SO interested in it...

Of course we're interested! We love this archiver.

So you can download only 'final' versions. But I've found some of those you requested:
dark-0.30-beta
dark-0.34a-beta
dark-0.38d.zip
They are availible for downloading.

And a little special:
dark-0.32a-beta
dark-0.32b-beta
dark-0.33-beta
dark-0.34a-beta (!!)
dark-0.37b (~0.38)


If you don't want to put them on the Dark home page, could you please put them all in one zip file and upload them here.

http://www.mytempdir.com/

[encode]

Another bug report:

The DARK from 'dark-0.39a.zip'

D:>dark c sys1.dark c:windowssystem32*
Dark v0.39 (C)kvark, 2006
4m block (24mb allocated)
c:windowssystem32$winnt$.inf |W 20ms
c:windowssystem321025 |[dir]
c:windowssystem321028 |[dir]
...
c:windowssystem32CatRoot2dberr.txt |W 20ms
c:windowssystem32CatRoot2edb.chk |fW 20ms
c:windowssystem32CatRoot2edb.log |

After that, DARK crashes and the error message appears:

dark.exe has encountered a problem and needs to close.
We are sorry for the inconvenience.
...
<Send Error Report> <Don&#039;t send>

And this is stable version? IMHO, keep BETA, until the baseline engine will be stable, and only then add fragmenters, filters, etc.

And how about the archive listing feature?

There is some files that are exist but cannot be opened. It&#039;s not an engine bug. This defect is fixed in v0.40. Thanx for testing!
Also I&#039;ve found and fixed the cause of crushing on enwik9. The problem was with 128Mb block. Now fixed

The files were here: http://darchiver.narod.ru/dark/
But now I made all links on Dark news page working. So you will have no troubles downloading older versions.

This version: 0.40 is some kind of final. I have some other work to do, that is more important now...
All known bugs was fixed, all uncompleted ideas was realised in this Dark-0.40, so enjoy
The main feature is Solid mode. It&#039;s compression ratio on my tests (with my soft contents) is better than 7z-PPMD (with same memory usage), but speed is 2-3 times faster.
I hope you&#039;ll like it

There is some files that are exist but cannot be opened. It&#039;s not an engine bug. This defect is fixed in v0.40. Thanx for testing!
Also I&#039;ve found and fixed the cause of crushing on enwik9. The problem was with 128Mb block. Now fixed

AWESOME!

The files were here: http://darchiver.narod.ru/dark/
But now I made all links on Dark news page working. So you will have no troubles downloading older versions.

Thank You!

All known bugs was fixed, all uncompleted ideas was realised in this Dark-0.40, so enjoy
The main feature is Solid mode. It&#039;s compression ratio on my tests (with my soft contents) is better than 7z-PPMD (with same memory usage), but speed is 2-3 times faster.
I hope you&#039;ll like it

Its fantastic!!!

This version: 0.40 is some kind of final. I have some other work to do, that is more important now...

This is VERY upsetting news!

Don&#039;t panic!
What do you want me to do when I&#039;m off with my urgent work (~2 weeks in best case :book ?

Afaik:
1. Lising option
2. GUI, GUI, GUI, and 1000 times GUI
3. CRC32?

What else?

Don&#039;t panic!

I was worried that you would never come back to the Dark project.

[kvark]

I assure you, it&#039;s impossible

Matt updated his results. Solid mode hadn&#039;t a chance to be used because of single file to compress, afaik. So only one model improvement: reverse input influence on results compartible to dark-0.32b. Anyway dark climbed up a little... and 7z-ppmd is now behind!!!

Maybe Dark will eventually be worthy of 7-Zip&#039;s old title?

"Archiver with highest compression ratio"

[kvark]

Originally it was planned to have "highest speed".
But now I think ratio is more important.
New version 0.41 improved compression up to 0.1-1%
But it&#039;s stlill unsatisfactory because ybs-0.03f has an advantage of 0.5-1% on texts with same DC...
If only I could know how it works... But Vadim Yoockin is not verbose so much

Originally it was planned to have "highest speed".
But now I think ratio is more important.

Maybe you could have both!

Why not have a useful FAST mode and a record breaking super-HIGH compression mode?


New version 0.41 improved compression up to 0.1-1%
But it&#039;s stlill unsatisfactory because ybs-0.03f has an advantage of 0.5-1% on texts with same DC...
If only I could know how it works... But Vadim Yoockin is not verbose so much

I have faith that you will create a BWT based archiver with highest compression ratio sometime in the near future.

[encode]

Maybe Dark will eventually be worthy of 7-Zip&#039;s old title?
"Archiver with highest compression ratio"

I think 7-Zip haven&#039;t such title... First of all, it completely lacks on multimedia files - on this files PIM and WinRAR simply beats it in ALL cases. Secondly, it have a poor interface a la Windows 95. But guys, today is the year 2006! Anyway, since 7-Zip is free - it&#039;s a super archiver!

Archiver with highest ratio is WinRK with its PWCM codec. In addition, WinRK have the perfect audio and image filters. But it have very poor development/market strategy - i.e. no freeware extracting tools, lots of codecs inside makes the most of users cry, and its non-windows like GUI...

[encode]

[kvark]

Most of people (users I know) talks about 7zip as it has really highest compression ratio. May be: who will compress multimedia files if everybody plays already compressed mp3,ogg,avi, and so on... And it has an acceptable speed, anyway.

>Why not have a useful FAST mode and a record breaking super-
> HIGH compression mode?

Yes,thanx! I really have to do this mode switch sometime

[encode]

Most of people (users I know) talks about 7zip as it has really highest compression ratio. May be: who will compress multimedia files if everybody plays already compressed mp3,ogg,avi, and so on... And it has an acceptable speed, anyway.

It completely depends on user&#039;s needs. Some users prefer listen WAV files and look at BMP files - without any data loss, since MP3 and JPG are lossy formats... Furthermore, modern games, like Half Life 2 - have uncompressed WAV files in their distributions...

We in the compression world know that 7-Zip does not really have the highest compression ratio; but its easy to see why some people may think it has.

Search for "highest compression ratio" on Google you will find many sites like these.

http://www.canadiancontent.net/tech/download/Seven -Zip.html

http://www.easymarketplace.de/winzip-7zip.php

[kvark]

Matt had updated results... And I&#039;m satisfacted with them
Congratilations to PIMPLE with 14-th place

It&#039;s a pity the contest is almost only for compression ratio, but not for speed...
But one thing I like very much:
Dark-0.41 is 49-th on the Overall rating,.. and ybs-0.03f is 50-th
It always was my competitor-1. It uses E8 preprocessing while Dark uses fragmentation. Almost fairly

Dark-0.41 is 49-th on the Overall rating,.. and ybs-0.03f is 50-th
It always was my competitor-1. It uses E8 preprocessing while Dark uses fragmentation. Almost fairly

Congratulations!

And I think that Dark will soon have *MUCH* higher compression.

ooopio
hhk

[LovePimple]

>Why not have a useful FAST mode and a record breaking super-
> HIGH compression mode?

Yes,thanx! I really have to do this mode switch sometime

I&#039;m really looking forward to this feature!

[kvark]

It seems that my HDD had crashed...
Dark project is freezed

[encode]

It&#039;s BAD news...



To override such thing with PIM (PIMPLE/TC/...), For EACH release I&#039;m writing a single CD with project sources. Without that, if my HDD will crash, the PIM project will be closed. Yes, I can restore the engine from the top of my head, but done again such titanic work...



But here I have a lots of Backup CDs...