REP and Delta fails with big files

[SvenBent]

i just tried to use REP and Delta filters on a 6.44GB file.
and both ended up giving me a file slightly below 30kbytes

is there a fixed version that supports big files ?

[Bulat Ziganshin]

10x, will look into this

[pat357]

i just tried to use REP and Delta filters on a 6.44GB file.
and both ended up giving me a file slightly below 30kbytes

is there a fixed version that supports big files ?

I tested FA -delta+rep on a 23 GB file, but could not reproduce the error you got.
Did you use FA or the stand-alone delta.exe/rep.exe ?
What FA version did you use ?

arc a -mdelta+rep:200mb test *.tib
FreeArc 0.50 alpha (August 3 2008) started: 0.00 secs
Compressing 1 file of 23.779.817.984 bytes: 0.01 secs
  Using delta+rep:200mb
  Memory for compression 272mb, decompression 99.9%
  Solid block compression results (592.726 seconds)
    delta: 23.783.048.828 bytes in 244.656 seconds
    rep:200mb: 22.125.748.340 bytes in 348.069 seconds

  Writing directory: 1237.54 secs
  Found 1 directory names: 1237.62 secs
  Directory written: 1237.6
Compressed 1 file, 23.779.817.984 => 22.125.748.340 bytes. Ratio 93.0%
Compression time 592.73 secs, speed 40.119 kB/s. Total 1237.63 secs

[Bulat Ziganshin]

these are different things. probably my frontend used in STANDALOBE rep.delta just use 32-bit ints for filesizes. it should be easy to fix

[pat357]

these are different things. probably my frontend used in STANDALOBE rep.delta just use 32-bit ints for filesizes. it should be easy to fix

I knew there are some issues with the stand-alone versions, that's the reason I don't use them only very seldom.
Dict.exe has a problem with mixed text/binaries (tar balls) and outputs something like a 4 byte file. While you are at these, could you please also have a look at this ?
Thanks !

[SvenBent]

Just to clarify.
I use the rep /delta stand alone programs


however i just tried with nanozip 0.0.4a and it gave me a 333kbytes file

7-zip and winrar works perfectly

[pat357]

Just to clarify.
I use the rep /delta stand alone programs.

Why don't you use ARC for that ? Less typing and probably faster because delta/rep can run parallel (MTT)

however i just tried with nanozip 0.0.4a and it gave me a 333kbytes file

That's weird : I used NZ -cD to compress my 27 GB file and it checked out ok.
What options did you use ?

[SvenBent]

Why don't you use ARC for that ? Less typing and probably faster because delta/rep can run parallel (MTT)

That's weird : I used NZ -cD to compress my 27 GB file and it checked out ok.
What options did you use ?

i use rep/delta stand alone because i brute force combinations with rzm 7-zip and rar.


i just used nz a -co -m1g archieve.nz file1 file2

but could it be because that i use a network mapped drive ?

[Bulat Ziganshin]

good news: i almost finished improving tornado cmdline driver and hopefully next week will release rep/delta with it. this means linux compatibility, progress indicator, large files support and lot of other facilities. also it will be BSD-licensed so i hope that other programmers will employ it too

[Vacon]

Hello everyone,

good news: i almost finished improving tornado cmdline driver and hopefully next week will release rep/delta with it. this means linux compatibility, progress indicator, large files support and lot of other facilities. also it will be BSD-licensed so i hope that other programmers will employ it too

Well, that's really good news!

Best regards!

[Bulat Ziganshin]

i've added new cmdline driver to REP, please test it:

[SvenBent]

Yay new toys to test

are there any other improvements than big file size support?
iIremember something about the memory usage was unnecessary big(2x the size)?

[SvenBent]

seems, compression is a little but worse in default mode
however -a99 had improved compression

but i have only teste one file so far. in -b1024m and -b1024m -a99 mode
md5 checksum after manual decompression was ok

however i notices som chances

-b now needs an m at the end to work EGE -b1024m instead of -b1024

when ypu decomrpess and specify outputfile name it crashes.

i will see i can reproduce this an nail it down to something specific

[Bulat Ziganshin]

> compression is a little but worse

yes, it's because output block is now limited to 8mb (it was up to dictsize before)

>-b now needs an m at the end to work EGE -b1024m instead of -b1024

yes, i think that all options that deals with sizes should use this format (with k/m/g at the end). previously i used megabytes to simplify implementation

>when ypu decomrpess and specify outputfile name it crashes.

probably you don't mastered new cmdline?

rep file.rep -ooutfile

[Bulat Ziganshin]

Yay new toys to test

are there any other improvements than big file size support?
I remember something about the memory usage was unnecessary big(2x the size)?

yes, it's the only real improvement (and sideback that it decreased compression a tiny little)