[zfs-discuss] Re: LZ4 compression not living up to hype

Durval Menezes durval.menezes at gmail.com
Sat Aug 3 16:01:55 EDT 2013


Hi Chris,

On Sat, Aug 3, 2013 at 4:35 PM, Chris Peden <chris at cptechconsulting.com>wrote:

> Well I am happy with the 80MB/sec not so much with the 50....and I'd like
> to enable compression but if im forced to leave it off then thats fine too.
>  I did a local test using dd
>
> with compression:
> time sh -c "dd if=/dev/zero of=testfile_a bs=100k count=10k && sync"
> 10240+0 records in
> 10240+0 records out
> 1048576000 bytes (1.0 GB) copied, 2.03775 s, 515 MB/s
>
> real    0m2.787s
> user    0m0.004s
> sys     0m2.476s
>
>
> without compression
> time sh -c "dd if=/dev/zero of=testfile_b bs=100k count=10k && sync"
> 10240+0 records in
> 10240+0 records out
> 1048576000 bytes (1.0 GB) copied, 2.1952 s, 478 MB/s
>
> real    0m19.722s
> user    0m0.004s
> sys     0m2.672s
>
>
> shows compression is indeed faster.  Im gonna test over NFS next.
>

Please be aware that reading your data from /dev/zero is going to present
unrealistically high compressible data to LZ4... you should not expect the
kind of performance you're going to get to occur in production. To get
realistic data with a local test, just "time cp -r" a dir tree with
representative data from *another* local disk into ZFS as I did in my
testing above.

Also, from the data you sent, I think you are CPU-pegged: it's not that my
core i7 quad is so much faster than your Athlon X2, but that I have twice
as many *cores*  (and four times as many *threads* due to HT)... and you
reported that your machine's load during compression was hitting about
2.86, so you are about a core short of what you'd need. OTOH, you also
reported seeing a load of 4.35 with compression off, so perhaps something
else in your machine is sucking up CPU cycles... I would investigate that
(use top or similar to see who's hogging the processor).

Cheers,
-- 
   Durval.


>
>
> Chris Peden
> --
> Owner/Consultant
> CP Tech Consulting
> chris at cptechconsulting.com
> http://cptechconsulting.com
> --
>
> On Aug 3, 2013, at 4:16 AM, Ryan How wrote:
>
> > Is the actual file copy speed any slower?
> >
> > At 80MB/sec you'll be coming close to saturating a gigabit network.
> > So I'd expect the disk write speed to be lower because it is compressed
> and there is less being written to disk.
> >
> > You are getting pretty good write speeds. I'd be happy with it :)
> >
> >
> > On 3/08/2013 3:15 PM, chris at cptechconsulting.com wrote:
> >> Ok oddly I did another test with compression off and watched the load
> and it got up to 4.35.  Yet its faster?
> >>
> >> I have no idea now...
> >>
> >> On Thursday, August 1, 2013 5:42:55 AM UTC-4,
> ch... at cptechconsulting.com wrote:
> >> Hello,
> >>
> >> So im building a new home nas.  Ubuntu 12.04 w/ZFS.  Setup a new pool.
>  Copying files to pool over smb.
> >>
> >> LZ4 compression off averages 80MB/sec
> >>
> >> LZ4 compression on averages 50MB/sec
> >>
> >> I read all this praise about how great LZ4 was but I didn't expect this
> big of performance hit.  Anyone else not seeing LZ4 perform well?  Also, im
> running this on desktop level commodity hardware, maybe that has something
> to do with it?
> >>
> >> Feedback appreciated!
> >>
> >>
> >> Thanks,
> >> Chris
> >>
> >>
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.zfsonlinux.org/pipermail/zfs-discuss/attachments/20130803/f3e574db/attachment.html>


More information about the zfs-discuss mailing list