NFS export

[Fredrik Widlund]

Hi,

I'm currently building a HA high-performance storage back-end using Arch Linux (2.6.31.4), GlusterFS 2.0.7, Fuse 2.8, XFS, PERC6 HW, and (currently only) 4 SATA drives in a RAID-5 setup.

Semi-random I/O (40 concurrent large dd reads of 100MB files) gives the following numbers:
XFS:  150 MB/s
XFS exported over NFS: 120 MB/s (1Gbps fully saturated)
GlusterFS/XFS locally: 120 MB/s

So, life is grand. I am really impressed by these numbers in general. However, here comes the downer:
GlusterFS/XFS exported over NFS: 30 MB/s

I know this isn't a new problem. Sadly this kind of strikes out the high-performance goals, so I need to try to address this somehow.

I noticed a thread earlier this year about trying to bump the syscall limit from 128kB to 4MB for example. Any pointers regarding if this the way to try to move forward?

Kind regards,
Fredrik Widlund


[cid:imagebd762b.png@...]         Fredrik Widlund, CSO / Chief Architect
Direct: [phone] | Mobile: [phone]

S?dra Hamnv?gen 22 | 115 41 STOCKHOLM
Web and mobile: www.qbrick.com<http://www.qbrick.com/>

Qnews nr. 21 - Klicka h?r<http://www.anp.se/newsletter/382446/424B5A43754043504B7242405143> f?r att l?sa v?rt senaste nyhetsbrev.



------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

[Miklos Szeredi]

On Thu, 29 Oct 2009, Fredrik Widlund wrote:

> Hi,
>
> I'm currently building a HA high-performance storage back-end using
> Arch Linux (2.6.31.4), GlusterFS 2.0.7, Fuse 2.8, XFS, PERC6 HW, and
> (currently only) 4 SATA drives in a RAID-5 setup.
>
> Semi-random I/O (40 concurrent large dd reads of 100MB files) gives
> the following numbers:
> XFS:  150 MB/s
> XFS exported over NFS: 120 MB/s (1Gbps fully saturated)
> GlusterFS/XFS locally: 120 MB/s
>
> So, life is grand. I am really impressed by these numbers in
> general. However, here comes the downer:
> GlusterFS/XFS exported over NFS: 30 MB/s
>
> I know this isn't a new problem. Sadly this kind of strikes out the
> high-performance goals, so I need to try to address this somehow.
>
> I noticed a thread earlier this year about trying to bump the
> syscall limit from 128kB to 4MB for example. Any pointers regarding
> if this the way to try to move forward?
>

I don't think the 128kB limit is the culprit here.  Can you try
running gluster with debugging option (not sure if that's possible) to
see what size requests go to the filesystem?

Equivalently a "strace" on the glusterfs would also be useful.

Thanks,
Miklos

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel