[zfs-discuss] Re: Bottlenecking on CPU with SSDs, Tuning, Scheduling

Umberto Nicoletti umberto.nicoletti at gmail.com
Tue Sep 10 02:52:48 EDT 2013

tgx_sync points to the the disk being almost full, as in this case:



Il giorno mercoledì 21 agosto 2013 16:02:07 UTC+2, Gordan Bobic ha scritto:
> Setup:
> 2x SSD (Intel 520 + Kingston HyperX)
> mirrored (using ZFS)
> 2 deduped zvols, dd-ing 64GB from one to the other (bs=1G count=64).
> 12x1GB zram L2ARCs.
> dedup=on
> compression=zle
> sync=disabled
> iostat -x consistently shows %util=100 during the process on the zvols, 
> but the %util for the raw underlying disks is considerably lower (varies 
> 10%-95%).
> That implies there is a major CPU bottleneck, and sure enough I see not 
> far from 50% of CPU overall being used on the machine (12-core (2x6) 3.6GHz 
> Xeon!).
> The biggest CPU hogs are:
> 20-30% txg_sync
> 10-20% spl_kmem_cache
> 10-20% arc_adapt
> 5-20% z_wr_iss (lots of these and the next 3!)
> 5-20% z_wr_int
> 5-20% z_rd_iss
> 5-20% z_rd_int
> I cannot see any one process bumping up against the 100% CPU core limit, 
> so it begs the question of what the problem is. My current theory is that 
> there are many short-lived operations being done by processes, which causes 
> a lot of unnecessary context switching which stalls everything all the 
> time, causes unnecessry CPU cache churn, etc. Are there any tweaks that 
> could be applied here? For example, pinning each of the z_* processes to an 
> individual CPU core?
> This may not be an issue on otherwise painfully slow spinning rust, but 
> but bumping up against the CPU related limits with even just one pair of 
> decent SSDs is a bit poor.
> Are there any tuning parameters that can be applied here to alleviate the 
> problem?
> Gordan

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+unsubscribe at zfsonlinux.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.zfsonlinux.org/pipermail/zfs-discuss/attachments/20130909/30685993/attachment.html>

More information about the zfs-discuss mailing list