NFS performance and other problems with 5.3 beta

Hi!

For quite a while, I've been curious about the benefits of EABI, so I decided to install SlugOS 5.3 beta yesterday. I strictly followed the wiki (http://www.nslu2-linux.org/wiki/SlugOS/InstallandTurnupABasicSlugOSSystem etc.). To cut a long story short - I'm back at SlugOS 4.8. Here's why:

*NFS performance is considerably worse in 5.3 than in 4.8. In 4.8, my DVB-T STB wrote 2.56 MB/s to a NFS share on the slug, under 5.3 the write rate decreased to a mere 0.86 MB/s which is useless for my purposes.
*Mount behaviour was stranger than ever. The system itself resided on a memstick which was consistently recognized as /dev/sda1, however, my two HDDs changed back and forth between /dev/sdb1 and /dev/sdc1. That's old news, of course, and so I tried the mount-by-label method described in the wiki. Unfortunately, this lead to each drive being mounted twice - first where the slug wanted it and second were I needed it.
*Samba 3.2.8 is a pain. Although this isn't strictly a slug issue, I assume other people will run into problems as well. For some reason beyond mere authentication (lanman vs. ntlm1 vs. ntlm2), I didn't even see the slug in my network neighbourhood (neither in Vista nor XP nor Win98SE). I've set up Samba including DFS on various platforms before and after reverting to SlugOS 4.8, basic configuration took less than 15 minutes, but in 5.3 I spent about 3 hours until I gave up.

Please don't get me wrong - I truly appreciate the developers' ongoing work and the updated information on http://www.nslu2-linux.org/wiki/SlugOS, but in the first place, I need a working fileserver with straightforward configuration. And yes, I RTFMed - if I didn't, I wouldn't have been able to set up SlugOS 4.8 either.

Regards,

T. Brinkmann

[Mike Westerhof (mwester)]

t.brinkmann@... wrote:
> Hi!
>
> For quite a while, I've been curious about the benefits of EABI, so I decided to install SlugOS 5.3 beta yesterday. I strictly followed the wiki (http://www.nslu2-linux.org/wiki/SlugOS/InstallandTurnupABasicSlugOSSystem etc.). To cut a long story short - I'm back at SlugOS 4.8. Here's why:
>
> *NFS performance is considerably worse in 5.3 than in 4.8. In 4.8, my DVB-T STB wrote 2.56 MB/s to a NFS share on the slug, under 5.3 the write rate decreased to a mere 0.86 MB/s which is useless for my purposes.

Was the hard drive on the slug mounted with the "sync" option?  That's
the default when udev mounts the device.  Please also provide details on
the devices used, and the output from the mount command so we can take a
look at this.

> *Mount behaviour was stranger than ever. The system itself resided on a memstick which was consistently recognized as /dev/sda1, however, my two HDDs changed back and forth between /dev/sdb1 and /dev/sdc1. That's old news, of course, and so I tried the mount-by-label method described in the wiki. Unfortunately, this lead to each drive being mounted twice - first where the slug wanted it and second were I needed it.

The strange behavior was actually SlugOS 4.8 -- it's fixed now!  :-)

If you don't want the devices automatically mounted for you, you need to
blacklist them in /etc/udev/mount.blacklist.  That you may not have had
to do that in 4.8 was a bug, and not a feature... I looked at trying to
turn it into a feature, but it's just not possible to create consistent
behavior due to the various ways it is possible to mount devices anymore.

A side effect of this auto-mounting is that the default options for
things mounted in /media is "sync" -- meaning that it goes really really
slowly.

> *Samba 3.2.8 is a pain. Although this isn't strictly a slug issue, I assume other people will run into problems as well. For some reason beyond mere authentication (lanman vs. ntlm1 vs. ntlm2), I didn't even see the slug in my network neighbourhood (neither in Vista nor XP nor Win98SE). I've set up Samba including DFS on various platforms before and after reverting to SlugOS 4.8, basic configuration took less than 15 minutes, but in 5.3 I spent about 3 hours until I gave up.

Yes.  IMO Samba 3.2 is a fabulous step forward for the international
data-center servers.  It sucks for small devices. To start with, 20+ MB
to install.  Then it doesn't even work -- you need to install locales,
and configure them all.

I have it working - of course.  But it was truly horrible to get it working.

Please complain to the Samba folks.  :(

> Please don't get me wrong - I truly appreciate the developers' ongoing work and the updated information on http://www.nslu2-linux.org/wiki/SlugOS, but in the first place, I need a working fileserver with straightforward configuration. And yes, I RTFMed - if I didn't, I wouldn't have been able to set up SlugOS 4.8 either.

Glad you appreciate it!  I'm betting its relatively trivial to get the
NFS performance fixed (might even be a side-effect of the lack of a
blacklist entry that would do it).  If you can share some information,
we can take a look at it.

> Regards,
>
> T. Brinkmann

Mike (mwester)

[Mike Westerhof (mwester)]

t.brinkmann@... wrote:

> *NFS performance is considerably worse in 5.3 than in 4.8. In 4.8, my DVB-T STB wrote 2.56 MB/s to a NFS share on the slug, under 5.3 the write rate decreased to a mere 0.86 MB/s which is useless for my purposes.

Just as a test, I get 1.7MB/s via NFS to a slow USB stick on the NSLU2:

$ dd if=/dev/zero of=foo bs=1M count=32
32+0 records in
32+0 records out
33554432 bytes (34 MB) copied, 19.3238 s, 1.7 MB/s

I think there's something not right on your setup. :-)

Mike (mwester)

[Mike Westerhof (mwester)]

Mike (mwester) wrote:

> t.brinkmann@... wrote:
>
>> *NFS performance is considerably worse in 5.3 than in 4.8. In 4.8, my DVB-T STB wrote 2.56 MB/s to a NFS share on the slug, under 5.3 the write rate decreased to a mere 0.86 MB/s which is useless for my purposes.
>
> Just as a test, I get 1.7MB/s via NFS to a slow USB stick on the NSLU2:
>
> $ dd if=/dev/zero of=foo bs=1M count=32
> 32+0 records in
> 32+0 records out
> 33554432 bytes (34 MB) copied, 19.3238 s, 1.7 MB/s
>
> I think there's something not right on your setup. :-)

Ok, just so we don't end up spreading concern needlessly throughout the
community, I scrounged up a disk drive, and re-ran the tests.

I guessed that you left everything pretty much default, and nfs exported
/media/sd<whatever>:


$ dd if=/dev/zero of=foo bs=1M count=30
30+0 records in
30+0 records out
31457280 bytes (31 MB) copied, 41.832 s, 752 kB/s

I get about what you quoted -- pretty sad.

So, on the slug, I mounted the drive the way it would be mounted if you
didn't use the auto-mounting udev mechanism (specifically, I removed the
"sync" option from the mount options):

$ dd if=/dev/zero of=foo bs=1M count=30
30+0 records in
30+0 records out
31457280 bytes (31 MB) copied, 8.01544 s, 3.9 MB/s

So, right back up to 3.9 MB/s -- there are few other NFS options that
one can tune or tweak (I got up to 4.0 MB/s), but that's pretty reasonable.

-Mike (mwester)

Before you start hitting me - I can't provide 5.3 output or logs
because I've reinstalled 4.8 on the same memstick, so everything's
lost. I simply need a working system and I won't have time to give
SlugOS 5.3 another try until next weekend (provided I'll find all the
necessary information about Samba in advance). Nevertheless, I'll try
my best to answer.

"Mike (mwester)" <mwester@...> wrote:
>Was the hard drive on the slug mounted with the "sync" option?
That's the default when udev mounts the device.  

The only option I set in fstab was "defaults", so if "sync" is a
default, the answer is yes.

>Please also provide details on the devices used, and the output from
the mount command so we can take a look at this.

The first device is a Medion 250GB HDD. dmesg says: WDC WD25
00JB-55GVA0      08.0 PQ: 0 ANSI: 0. There's a single ext3 partition
on it.
The second device is a Toshiba 1TB HDD. dmesg: TOSHIBA  USB
3.5"-HDD     100  PQ: 0 ANSI: 2. There's a single ext2 partition on
it.

>If you don't want the devices automatically mounted for you, you
need to blacklist them in /etc/udev/mount.blacklist.  

Oops, I didn't know that. Automount is fine with me as long as
filesystems end up in the same place every time. If not, my scripts
might cause severe damage.

>Then it doesn't even work -- you need to install locales, and
configure them all.
I think I got that one right - I followed
http://www.nslu2-linux.org/wiki/HowToUseInternationalCharactersOnOpenSlug.
With German characters like ä, ö, ü and ß, that's a must I was aware of.

> Please complain to the Samba folks.  :(
I might...

I've just read your second reply. I agree that there may have been a
mistake in my setup - most likely in connection with mounting the
drives. NFS itself can hardly be misconfigured - /etc/exports isn't
that hard and it should be the same for both SlugOS 4.8 and 5.3.

Regards,

T. Brinkmann

[brian_zhou]

If I understand it correctly, the benefit of EABI is mostly in floating point performance.

-Brian

--- In nslu2-linux@..., "t.brinkmann@..." <t.brinkmann@...> wrote:

>
> Hi!
>
> For quite a while, I've been curious about the benefits of EABI, so I decided to install SlugOS 5.3 beta yesterday. I strictly followed the wiki (http://www.nslu2-linux.org/wiki/SlugOS/InstallandTurnupABasicSlugOSSystem etc.). To cut a long story short - I'm back at SlugOS 4.8. Here's why:
>
> *NFS performance is considerably worse in 5.3 than in 4.8. In 4.8, my DVB-T STB wrote 2.56 MB/s to a NFS share on the slug, under 5.3 the write rate decreased to a mere 0.86 MB/s which is useless for my purposes.
> *Mount behaviour was stranger than ever. The system itself resided on a memstick which was consistently recognized as /dev/sda1, however, my two HDDs changed back and forth between /dev/sdb1 and /dev/sdc1. That's old news, of course, and so I tried the mount-by-label method described in the wiki. Unfortunately, this lead to each drive being mounted twice - first where the slug wanted it and second were I needed it.
> *Samba 3.2.8 is a pain. Although this isn't strictly a slug issue, I assume other people will run into problems as well. For some reason beyond mere authentication (lanman vs. ntlm1 vs. ntlm2), I didn't even see the slug in my network neighbourhood (neither in Vista nor XP nor Win98SE). I've set up Samba including DFS on various platforms before and after reverting to SlugOS 4.8, basic configuration took less than 15 minutes, but in 5.3 I spent about 3 hours until I gave up.
>
> Please don't get me wrong - I truly appreciate the developers' ongoing work and the updated information on http://www.nslu2-linux.org/wiki/SlugOS, but in the first place, I need a working fileserver with straightforward configuration. And yes, I RTFMed - if I didn't, I wouldn't have been able to set up SlugOS 4.8 either.
>
> Regards,
>
> T. Brinkmann
>

--- In nslu2-linux@..., "Brian Zhou" <b88zhou@...> wrote:
>
> If I understand it correctly, the benefit of EABI is mostly in floating point performance.
>
> -Brian
>
True, but I guess that most of us need the fileservers on the slug in the first place. Improvements in other areas are highly welcome but less necessary. In the last paragraph of http://www.nslu2-linux.org/wiki/HowTo/MP3Encoding there's some information on improved mp3 encoding - I guess that's a task I won't use my slug for...

T. Brinkmann

[lexj1]

--- In nslu2-linux@..., "Mike (mwester)" <mwester@...> wrote:

>
> t.brinkmann@... wrote:
> > Hi!
> >
> > For quite a while, I've been curious about the benefits of EABI, so I decided to install SlugOS 5.3 beta yesterday. I strictly followed the wiki (http://www.nslu2-linux.org/wiki/SlugOS/InstallandTurnupABasicSlugOSSystem etc.). To cut a long story short - I'm back at SlugOS 4.8. Here's why:
> >
> > *NFS performance is considerably worse in 5.3 than in 4.8. In 4.8, my DVB-T STB wrote 2.56 MB/s to a NFS share on the slug, under 5.3 the write rate decreased to a mere 0.86 MB/s which is useless for my purposes.
>
> Was the hard drive on the slug mounted with the "sync" option?  That's
> the default when udev mounts the device.  Please also provide details on
> the devices used, and the output from the mount command so we can take a
> look at this.
>
> > *Mount behaviour was stranger than ever. The system itself resided on a memstick which was consistently recognized as /dev/sda1, however, my two HDDs changed back and forth between /dev/sdb1 and /dev/sdc1. That's old news, of course, and so I tried the mount-by-label method described in the wiki. Unfortunately, this lead to each drive being mounted twice - first where the slug wanted it and second were I needed it.
>
> The strange behavior was actually SlugOS 4.8 -- it's fixed now!  :-)
>
> If you don't want the devices automatically mounted for you, you need to
> blacklist them in /etc/udev/mount.blacklist.  That you may not have had
> to do that in 4.8 was a bug, and not a feature... I looked at trying to
> turn it into a feature, but it's just not possible to create consistent
> behavior due to the various ways it is possible to mount devices anymore.
>
> A side effect of this auto-mounting is that the default options for
> things mounted in /media is "sync" -- meaning that it goes really really
> slowly.
>
> > *Samba 3.2.8 is a pain. Although this isn't strictly a slug issue, I assume other people will run into problems as well. For some reason beyond mere authentication (lanman vs. ntlm1 vs. ntlm2), I didn't even see the slug in my network neighbourhood (neither in Vista nor XP nor Win98SE). I've set up Samba including DFS on various platforms before and after reverting to SlugOS 4.8, basic configuration took less than 15 minutes, but in 5.3 I spent about 3 hours until I gave up.
>
> Yes.  IMO Samba 3.2 is a fabulous step forward for the international
> data-center servers.  It sucks for small devices. To start with, 20+ MB
> to install.  Then it doesn't even work -- you need to install locales,
> and configure them all.
>
> I have it working - of course.  But it was truly horrible to get it working.
>
> Please complain to the Samba folks.  :(
>
> > Please don't get me wrong - I truly appreciate the developers' ongoing work and the updated information on http://www.nslu2-linux.org/wiki/SlugOS, but in the first place, I need a working fileserver with straightforward configuration. And yes, I RTFMed - if I didn't, I wouldn't have been able to set up SlugOS 4.8 either.
>
> Glad you appreciate it!  I'm betting its relatively trivial to get the
> NFS performance fixed (might even be a side-effect of the lack of a
> blacklist entry that would do it).  If you can share some information,
> we can take a look at it.
>
> > Regards,
> >
> > T. Brinkmann
>
> Mike (mwester)
>

Thanks for the post, your a lifesaver!  I have been searching for a fix to my slow NFS speeds for most of a day.

Adding my partitions to the /etc/udev/mount.blacklist fixed the problem.  Now I am getting almost 4MB/s, up from 850 kB/s.  Also, after it was fixed the sync vs async doesn't seem to have any appreciable effect on the speed.

Lex