[zfs-discuss] Re: Getting a handle on the current weak areas of ZFS-on-Linux

Anthony D'Atri aad at dreamsnake.net
Fri Aug 16 23:53:53 EDT 2013


> Just an info: ata-over-ethernet seems a very good, easy and cheap solution

Not the way we did it :-/

> . I am not a vendor, no interest about it, but only a user.
> 
> I don't think, it is exotic, I saw much more issues with iscsi on lists (on several mailing lists) than with AoE.

Well, compared to just plugging in some SAS HBA's and rocking with normal ZFS …

> If the protocol has not been revised since then (which it doesn't seem
> to have been) it's my opinion that AoE is fundamentally inferior to
> anything competent that's based on TCP[*]. Whether its issues matter to
> you will depend on your specific setup and how you feel about things
> like jumbo frames.


Well, what we've got is a proprietary implementation that was purchased without me in the loop  Disk shelves have 10GE links and talk to what was supposed to be an appliance (via two 10GE "HBA" cards that look to be NICs with custom firmware) through an Arista switch.  That appliance turned out to be a cheap box running Solaris 11, with a big ZFS volume [17x (6D+2P)] built from 2TB disks.  The single client (NBU server) accesses the volume via NFS.  Performance doesn't seem to be great, and the complexity is unfortunate.  

I've thought about factoring the Sol11 server (8 cores, 48GB RAM) and NFS out of the equation by moving the "HBA" cards into the client system (40 cores, 128GB RAM), installing the vendor's AoE driver for RHEL, and setting up ZoL, maybe mirrored instead.  The client runs NBU just dumping to the ~288TB filesystem - we may go to NBU dedup, in which case I'm told we can only use 64TB anyway, as Symantec only supports one dedup pool, max 64TB.

> 
> Regarding the how-to-deal-with-errors-in-layer2 related question (it will detect the error and handle it as expected) I asked experts:
> 
> "This is only true if your Ethernet switch does not rewrite the frames as it forwards them. "
> 
> So, I think it can be as safe as iscsi. Of course, one must analyze requirements and make storage strategy before choose the solution.
> 
> Cheers,
> 
> István
> 
>  
> ----------------eredeti üzenet----------------- 
> Feladó: "anthony11" <aad at dreamsnake.net> 
> Címzett: zfs-discuss at zfsonlinux.org 
> Dátum: Fri, 16 Aug 2013 00:34:40 -0700 (PDT) 
> ----------------------------------------------------------
> 
>> 
>> 
>> I thought I'd read that this could lease to significant loss of capacity, and I honestly have no idea about how to consider the data size / planned lifetime constraints.  Especially if the disks are across a 10GE link through an exotic ATA-over-Ethernet driver.
>> 
>> That's a easy one. Just always specify ashift=12 unless you know for certain that your data size and planned lifetime demand something smaller.
>> 
>> On Aug 15, 2013 2:29 PM, "anthony11" <a... at dreamsnake.net> wrote:
>> For me a blaring weak area is the ashift stuff - Joe Sysadmin (and even I) have little idea of what model disk has/claims what sector size.  I'd reeeeally like for ZoL to just figure it out and do the right thing.
>>  
>> To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+unsubscribe at zfsonlinux.org.
>  
>  

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+unsubscribe at zfsonlinux.org.



More information about the zfs-discuss mailing list