gzip-Performance

performance compressing a 17.1 GB (18365747200 bytes, to be exact) file at levels 1 to 9. The red and green lines show time in seconds (on a 3 GHz P4), the blue line the size of the compressed file in percent of the original. The irregularities in the real time are believed to be due to other activity on the machine.

The compression doesn't increase much beyond 6, and the time rises sharply after 7. So the common wisdom that a compression level of 6 is generelly the best compromise seems to be justified.

That should also show in the ratio between space gained and time elapsed. However, the graph shows a monotonous decrease for the CPU time, indicating that a level of 1 is best. The line for the elapsed time shows an optimum for level 2 and a suboptimum for level 5 - this may be due to a measurement error, though.

tiger:/ora/app/oracle/oradata/WFN 16:38 :-) 112# gzip -1v < comext01.dbf > /dev/null
 66.6%
gzip -1v < comext01.dbf > /dev/null  328.50s user 16.59s system 98% cpu 5:51.58 total
tiger:/ora/app/oracle/oradata/WFN 16:44 :-) 113# gzip -5v < comext01.dbf > /dev/null
 71.2%
gzip -5v < comext01.dbf > /dev/null  690.07s user 17.64s system 99% cpu 11:53.74 total

% dd if=comext01.dbf bs=8k of=/dev/null
2097153+0 records in
2097153+0 records out
17179877376 bytes (17 GB) copied, 178.214 s, 96.4 MB/s
dd if=comext01.dbf bs=8k of=/dev/null  0.36s user 10.18s system 5% cpu
2:58.96 total

for q in 1 2 3 4 5 6 7 8 9 
do
    for p in gzip lzop
    do
        echo $p $q
        time $p -$q < comext01.dbf > compressed
        ls -l compressed
        echo
    done
done 2>&1 | tee compress.bench.log

Comparison between gzip and lzop compressing a 16 GB (17179877376 bytes) oracle data file. The first 6 levels of lzop are very close together (137-140 CPU seconds, 40.3 to 40.5 % compressed size). Levels 7 to 9 are very slow and don't compress much better and aren't shown in this graph. Even level 1 of gzip compresses much better (33.4% compressed size) but is also much slower (359 CPU seconds). Note that for this kind of input, lzop compresses roughly 1 Gbit/s, while gzip -1 is already a bit slower than even slow hard disks and compression is likely to be always CPU bound. gzip level 6 (the default) compresses a bit better (28.5%) but is much slower (almost exactly 1000 CPU seconds).