[zfs-discuss] Re: Copying a file from one ZFS to another starves all other processes

Ryan How ryan at zbit.net.au
Wed May 22 03:46:20 EDT 2013


Ok, So I thought it was a problem with write buffers (eg. large file 
copy buffering, hitting the threshold, then freezing everything as the 
buffer flushes)

So I looked into: grep -A 1 dirty /proc/vmstat

nr_dirty 0
nr_writeback 0

And it is always 0 !!

On a non-zfs filesystem they seem to work as expected and editing the 
thresholds changes the buffering effect.
On a zvol it seems to work (Seem I might have solved my other issue there).

So I guess ZFS bypasses this somehow?

Still stuck as what to do :(

At least I learnt something about how to tweak the linux write buffer.

dd to a zfs gives me 160MB/s
dd to a zvol with linux write buffer disabled (threshold 0) gives me 2.5MB/s
dd to a zvol with linux write buffer enabled gives me 50MB/s

So ZFS must be doing some buffering? Is there any way to tweak that so 
it doesn't freeze my other processes when the buffer gets full?

I'm really grabbing at straws :(

Thanks, Ryan


On 22/05/2013 2:50 PM, Ryan How wrote:
> I've compiled against Ubuntu Raring Kernel (GIT Head).
>
> Just has default kernel config + a few filesystems + network drivers + 
> devtmpfs
>
> Seems to be a bit more stable than it was before. But still get the 
> same issue. Maybe I should set up a straight ubuntu box and see if it 
> works.
>
> On a side note, if there any way to throttle a cp ?. Maybe if I 
> throttle it (I shouldn't have to!) at least it won't kill the system. 
> ionice doesn't seem to make any difference.
>
> Thanks, Ryan
>
>
>
>
>
>
> On 20/05/2013 10:50 AM, Ryan How wrote:
>> BTW, I'm not overlaying ZFS with aufs or anything, they don't 
>> interact at all on the system.
>>
>> On 20/05/2013 10:47 AM, Ryan How wrote:
>>> Thanks,
>>>
>>> I'm running slackware 14.0, and am compiling ZOL with the "Generic 
>>> RPM" KMOD method. I've just downloaded the ubuntu raring kernel 
>>> sources and will try that. Most people seem to be using that and it 
>>> is working. I'm using Aufs too, so I need to compile a custom kernel 
>>> (But not doing any crazy configuration or anything). Seems that 
>>> ubuntu raring already has it patched in :)
>>>
>>> Thanks, Ryan
>>>
>>>
>>>
>>>
>>> On 19/05/2013 11:36 PM, RangerRick wrote:
>>>> Hi Ryan,
>>>>
>>>> What version of Linux are you running?  We have many hundreds of 
>>>> customers using ZOL with SoftNAS and have had no such issues with 
>>>> ZOL on CentOS 6.x.
>>>>
>>>> However, I have seen something similar to what you describe occur 
>>>> when files are copied to and from the LOCAL filesystem (non-ZFS) 
>>>> only on a ZOL system.  Local filesystem caching appears to put 
>>>> pressure on available memory, which in turn causes what appears to 
>>>> be a lockup until that memory pressure is relieved.  We have not 
>>>> observed this when the files being copied are ZFS to ZFS - only if 
>>>> done on the root filesystem, which is non-ZFS.  As we do not use 
>>>> the root filesystem for file sharing, we do not encounter this in 
>>>> practice.
>>>>
>>>> Rick
>>>>
>>>>
>>>> On Friday, May 17, 2013 11:29:32 PM UTC-5, Ryan How wrote:
>>>>
>>>>     With all the ZFS problems I'm having, it seems that it must be
>>>>     a problem
>>>>     with my kernel or something, coz otherwise I'm sure there would
>>>>     be a lot
>>>>     more people posting issues on the mailing list and a lot less
>>>>     people
>>>>     using ZOL!
>>>>
>>>>     They all almost seem related to I/O scheduling and buffering
>>>>     (well the
>>>>     symptoms of it, not sure what the actual problem is)
>>>>
>>>>     One of the problems I'm having is when copying a file (cp fileA
>>>>     fileB),
>>>>     it basically starves everything else. Then my VMs (running as file
>>>>     backed storage directly on ZFS) get corrupt filesystems
>>>>     reporting that
>>>>     the disk controller was not responding (I'm guessing because
>>>>     they are
>>>>     trying to write, get to response from the system, so the guest
>>>>     things
>>>>     the virtual disk is faulty)
>>>>
>>>>     Does anyone else get anything like this?
>>>>
>>>>     When copying as well, all the zfs commands are slow to respond
>>>>     (eg.
>>>>     zpool iostat takes a few seconds to spit out any info)
>>>>
>>>>     Not using dedup or compression, machine has 32GB ram of which
>>>>     6GB is
>>>>     currently free (after running for a few hours)
>>>>
>>>>     I'm not really fussed if performance is slow (but performance is
>>>>     actually quite good), but if I am doing more than 1 disk
>>>>     related thing
>>>>     at a time it is kinda useless.
>>>>
>>>>     Thanks, Ryan
>>>>
>>>>
>>>>
>>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.zfsonlinux.org/pipermail/zfs-discuss/attachments/20130522/c44d0580/attachment.html>


More information about the zfs-discuss mailing list