[zfs-discuss] High Resource Usage During Copy

Steve Costaras stevecs at chaven.com
Sun Jun 12 11:16:24 EDT 2011

First thing I usally look for in top would be interrupts (hardware (%hi)/software(%si)) then i/o wait (wa), then cpu% as the first items would inflate cpu% (taking cycles to handle the other items). it's also good to use the '1' (single/seperate line for each core) so you can see the loading a bit easier.

On top of that looking at 'zpool iostat X' (where X is seconds) will show you the 'data' portion that is being handed to the subsystem, but the os's iostat -mx X (X also seconds) will show you all data (including CRC overhead) going to a disk, as well as any service time request size items. Generally, first thing I look for is a disk that is handling more or less (substancially) than others in the same pool. This could indicate a problem with the drive/cabeling/hba/chassis et al. and would merit further investigation. 

Generally for SATA drives, I try and keep the utilization below 40% (below 20% for 'lower quality' drives), SAS/FC you can normally run these up to 60-80% before you start seeing real user level issues. 

-----Original Message-----
From: Okky Hendriansyah [mailto:mawcikurl at gmail.com]
Sent: Sunday, June 12, 2011 09:22 AM
To: zfs-discuss at zfsonlinux.org
Subject: Re: [zfs-discuss] High Resource Usage During Copy

Hi Steve,

These processes were the top tens among other usual process:
a. cp -R -> the copy process itself (or kio_file when using Dolphin to copy the files)
b. [z_wr_iss/$number]
c. [z_wr_int/$number]
d. [arc_reclaim]
e. [kswapd0]

The file transfer speed usually fluctuates between 40 MB/s to 100 MB/s, looks like when I still have plenty of RAM the transfer speed is high. Probably just like you said that it cached the transaction before flushing it all to the disk. Well, as long as its normal. :) Thanks for your explanation!

Best regards,
Okky Hendriansyah

On Sun, Jun 12, 2011 at 8:29 PM, Steve Costaras <stevecs at chaven.com> wrote:

For the cpu usage do a "top -H" and find out what thread(s) are using the cycles and if they are interrupt or calculation. As for memory usage, yes that's normal for ZFS as to avoid fragmentation it will buffer all writes and flush to disk when memory pressure is great enough or when a file is closed. This can also skew performance benchmarks when you have a lot of memory. 

-----Original Message-----
From: Okky Hendriansyah [mailto:mawcikurl at gmail.com]
Sent: Sunday, June 12, 2011 06:51 AM
To:zfs-discuss at zfsonlinux.org
Subject: [zfs-discuss] High Resource Usage During Copy

Hi All,

My box is a quad core CPU with 16 GB of RAM. While running idle the system only use about 1.4 GB of RAM. Today I want to start migrating my VirtualBox images from Ext4 filesystem to a single disk ZFS Pool. I copy the folder while opening System Monitor to track ZFS resource usage. When I start copying, I noticed that the CPU usage i quite high (most of the time it was 90%) although only happened on one of the cores. But looking at my RAM usage history, it seems like it rapidly increase about 100 MB/s, from 1.4 GB to 13 GB and then went down to 3.5 GB for a while and raise again to 13 GB. On and on like that until the copy process is finished.

I'm not coming from a Solaris background, so I do not know whether that kind of behavior could be considered as normal ZFS behavior or not. Are there some configurations I need to change before?


Best regards,
Okky Hendriansyah

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.zfsonlinux.org/pipermail/zfs-discuss/attachments/20110612/6e16f028/attachment.html>

More information about the zfs-discuss mailing list