Corrupted data on write to Windows 2003 Server

View: New views
5 Messages — Rating Filter:   Alert me  

Corrupted data on write to Windows 2003 Server

by Danny Sung-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

When copying a file from linux to windows, I noticed corrupted data
whenever my data size is larger than about 100-200MB.  I ran a few
tests, using 'dd', writing a 1GB file with varying block sizes (1, 10,
50, 100, and 200 megs).   Each one reported a different md5 when reading
back.  Only the 200 MB block size was successful, but it may have been a
fluke... I need to run the test a few more times.

The actual number of bytes in the destination file is correct.  But
running 'cmp' afterwards, I discovered that the destination ends up
being 0 padded after a certain point.  The exact spot seems to wander,
but it appears to be around the 800-900MB mark for these files.

I don't get any error messages or anything from the actual 'dd' command.
  The only message I get is a kernel message saying: "CIFS VFS: No
writable handles for inode", which I suppose explains it all.  But it'd
be nice if the user commands reported some sort of failure.

I don't suppose there are any mount options I can give to cifs to make
it do more checking or retries or anything?  Or how do I actually
prevent the "no writable handles" problem?

Of course, if I use rsync, that seems to get around the problem.  But it
seems kind dumb to have to do that for mounted filesystems. =P



Linux system: Fedora Core 5
Linux aeolus.eng.lantronix.com 2.6.18-1.2200.fc5smp #1 SMP Sat Oct 14
17:15:35 EDT 2006 i686 i686 i386 GNU/Linux


Windows System: Windows Server 2003 R2 Standard w/ SP1

mount options:  cifs (rw,mand)

I don't think this uses samba since cifs is a kernel module, but just in
case, my samba version is: 3.0.23c

Thanks for any assistance,
Danny

---
Please ignore the following disclaimer, not the message. =)
**********************************************************************
This e-mail is the property of Lantronix. It is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential, or otherwise protected from disclosure. Distribution or copying of this e-mail, or the information contained herein, to anyone other than the intended recipient is prohibited.

_______________________________________________
linux-cifs-client mailing list
linux-cifs-client@...
https://lists.samba.org/mailman/listinfo/linux-cifs-client

Re: Corrupted data on write to Windows 2003 Server

by Shirish S Pargaonkar :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

linux-cifs-client-bounces+shirishp=us.ibm.com@... wrote on 12/08/2006 01:24:24 PM:

> When copying a file from linux to windows, I noticed corrupted data
> whenever my data size is larger than about 100-200MB.  I ran a few
> tests, using 'dd', writing a 1GB file with varying block sizes (1, 10,
> 50, 100, and 200 megs).   Each one reported a different md5 when reading
> back.  Only the 200 MB block size was successful, but it may have been a
> fluke... I need to run the test a few more times.
>
> The actual number of bytes in the destination file is correct.  But
> running 'cmp' afterwards, I discovered that the destination ends up
> being 0 padded after a certain point.  The exact spot seems to wander,
> but it appears to be around the 800-900MB mark for these files.
>
> I don't get any error messages or anything from the actual 'dd' command.
>   The only message I get is a kernel message saying: "CIFS VFS: No
> writable handles for inode", which I suppose explains it all.  But it'd
> be nice if the user commands reported some sort of failure.
>
> I don't suppose there are any mount options I can give to cifs to make
> it do more checking or retries or anything?  Or how do I actually
> prevent the "no writable handles" problem?
>
> Of course, if I use rsync, that seems to get around the problem.  But it
> seems kind dumb to have to do that for mounted filesystems. =P
>
>
>
> Linux system: Fedora Core 5
> Linux aeolus.eng.lantronix.com 2.6.18-1.2200.fc5smp #1 SMP Sat Oct 14
> 17:15:35 EDT 2006 i686 i686 i386 GNU/Linux
>
>
> Windows System: Windows Server 2003 R2 Standard w/ SP1
>
> mount options:  cifs (rw,mand)
>
> I don't think this uses samba since cifs is a kernel module, but just in
> case, my samba version is: 3.0.23c
>
> Thanks for any assistance,
> Danny
>
> ---
> Please ignore the following disclaimer, not the message. =)
> **********************************************************************
> This e-mail is the property of Lantronix. It is intended only for
> the person or entity to which it is addressed and may contain
> information that is privileged, confidential, or otherwise protected
> from disclosure. Distribution or copying of this e-mail, or the
> information contained herein, to anyone other than the intended
> recipient is prohibited.
>
> _______________________________________________
> linux-cifs-client mailing list
> linux-cifs-client@...
> https://lists.samba.org/mailman/listinfo/linux-cifs-client

Danny,

We have seen this problem and are working on it.  Sizes of the source and destination files are same but
md5sum would report different values.  I have seen this problem with large files like 500MB+.
I have seen some (page) ranges in destination file filled with NUL bytes.
But we have observed this problem only with copying files to Windows server and not with Samba server.

As a work-around, you may want to use option forcedirectio during mounts.

Regards,

Shirish S. Pargaonkar

_______________________________________________
linux-cifs-client mailing list
linux-cifs-client@...
https://lists.samba.org/mailman/listinfo/linux-cifs-client


Re: Corrupted data on write to Windows 2003 Server

by Danny Sung-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks so much!  That solved the problem.  Curiously, I also seem to get
better performance!  7 MB/sec rather than 5 MB/s.  Perhaps I should run
the test again... it's possible the system was loaded with backups the
first time.


Shirish S Pargaonkar wrote:

> linux-cifs-client-bounces+shirishp=us.ibm.com@... wrote on
> 12/08/2006 01:24:24 PM:
>
>  > When copying a file from linux to windows, I noticed corrupted data
>  > whenever my data size is larger than about 100-200MB.  I ran a few
>  > tests, using 'dd', writing a 1GB file with varying block sizes (1, 10,
>  > 50, 100, and 200 megs).   Each one reported a different md5 when reading
>  > back.  Only the 200 MB block size was successful, but it may have been a
>  > fluke... I need to run the test a few more times.
>  >
>  > The actual number of bytes in the destination file is correct.  But
>  > running 'cmp' afterwards, I discovered that the destination ends up
>  > being 0 padded after a certain point.  The exact spot seems to wander,
>  > but it appears to be around the 800-900MB mark for these files.
>  >
>  > I don't get any error messages or anything from the actual 'dd' command.
>  >   The only message I get is a kernel message saying: "CIFS VFS: No
>  > writable handles for inode", which I suppose explains it all.  But it'd
>  > be nice if the user commands reported some sort of failure.
>  >
>  > I don't suppose there are any mount options I can give to cifs to make
>  > it do more checking or retries or anything?  Or how do I actually
>  > prevent the "no writable handles" problem?
>  >
>  > Of course, if I use rsync, that seems to get around the problem.  But it
>  > seems kind dumb to have to do that for mounted filesystems. =P
>  >
>  >
>  >
>  > Linux system: Fedora Core 5
>  > Linux aeolus.eng.lantronix.com 2.6.18-1.2200.fc5smp #1 SMP Sat Oct 14
>  > 17:15:35 EDT 2006 i686 i686 i386 GNU/Linux
>  >
>  >
>  > Windows System: Windows Server 2003 R2 Standard w/ SP1
>  >
>  > mount options:  cifs (rw,mand)
>  >
>  > I don't think this uses samba since cifs is a kernel module, but just in
>  > case, my samba version is: 3.0.23c
>  >
>  > Thanks for any assistance,
>  > Danny
>  >
>  > ---
>  > Please ignore the following disclaimer, not the message. =)
>  > **********************************************************************
>  > This e-mail is the property of Lantronix. It is intended only for
>  > the person or entity to which it is addressed and may contain
>  > information that is privileged, confidential, or otherwise protected
>  > from disclosure. Distribution or copying of this e-mail, or the
>  > information contained herein, to anyone other than the intended
>  > recipient is prohibited.
>  >
>  > _______________________________________________
>  > linux-cifs-client mailing list
>  > linux-cifs-client@...
>  > https://lists.samba.org/mailman/listinfo/linux-cifs-client
>
> Danny,
>
> We have seen this problem and are working on it.  Sizes of the source
> and destination files are same but
> md5sum would report different values.  I have seen this problem with
> large files like 500MB+.
> I have seen some (page) ranges in destination file filled with NUL bytes.
> But we have observed this problem only with copying files to Windows
> server and not with Samba server.
>
> As a work-around, you may want to use option forcedirectio during mounts.
>
> Regards,
>
> Shirish S. Pargaonkar
>

_______________________________________________
linux-cifs-client mailing list
linux-cifs-client@...
https://lists.samba.org/mailman/listinfo/linux-cifs-client

Re: Corrupted data on write to Windows 2003 Server

by Danny Sung-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Just giving you an update.  forcedirectio seems to be a lot more
reliable than not having the option.  However, for a particularly large
file (eg. 6GB) it aborts with a message saying "Host down".

Shirish S Pargaonkar wrote:

> Danny,
>
> We have seen this problem and are working on it.  Sizes of the source
> and destination files are same but
> md5sum would report different values.  I have seen this problem with
> large files like 500MB+.
> I have seen some (page) ranges in destination file filled with NUL bytes.
> But we have observed this problem only with copying files to Windows
> server and not with Samba server.
>
> As a work-around, you may want to use option forcedirectio during mounts.
>
> Regards,
>
> Shirish S. Pargaonkar
>
**********************************************************************
This e-mail is the property of Lantronix. It is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential, or otherwise protected from disclosure. Distribution or copying of this e-mail, or the information contained herein, to anyone other than the intended recipient is prohibited.

_______________________________________________
linux-cifs-client mailing list
linux-cifs-client@...
https://lists.samba.org/mailman/listinfo/linux-cifs-client

Re: Corrupted data on write to Windows 2003 Server

by Shirish S Pargaonkar :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

linux-cifs-client-bounces+shirishp=us.ibm.com@... wrote on 12/08/2006 01:24:24 PM:

> When copying a file from linux to windows, I noticed corrupted data
> whenever my data size is larger than about 100-200MB.  I ran a few
> tests, using 'dd', writing a 1GB file with varying block sizes (1, 10,
> 50, 100, and 200 megs).   Each one reported a different md5 when reading
> back.  Only the 200 MB block size was successful, but it may have been a
> fluke... I need to run the test a few more times.
>
> The actual number of bytes in the destination file is correct.  But
> running 'cmp' afterwards, I discovered that the destination ends up
> being 0 padded after a certain point.  The exact spot seems to wander,
> but it appears to be around the 800-900MB mark for these files.
>
> I don't get any error messages or anything from the actual 'dd' command.
>   The only message I get is a kernel message saying: "CIFS VFS: No
> writable handles for inode", which I suppose explains it all.  But it'd
> be nice if the user commands reported some sort of failure.
>
> I don't suppose there are any mount options I can give to cifs to make
> it do more checking or retries or anything?  Or how do I actually
> prevent the "no writable handles" problem?
>
> Of course, if I use rsync, that seems to get around the problem.  But it
> seems kind dumb to have to do that for mounted filesystems. =P
>
>
>
> Linux system: Fedora Core 5
> Linux aeolus.eng.lantronix.com 2.6.18-1.2200.fc5smp #1 SMP Sat Oct 14
> 17:15:35 EDT 2006 i686 i686 i386 GNU/Linux
>
>
> Windows System: Windows Server 2003 R2 Standard w/ SP1
>
> mount options:  cifs (rw,mand)
>
> I don't think this uses samba since cifs is a kernel module, but just in
> case, my samba version is: 3.0.23c
>
> Thanks for any assistance,
> Danny
>
> ---
> Please ignore the following disclaimer, not the message. =)
> **********************************************************************
> This e-mail is the property of Lantronix. It is intended only for
> the person or entity to which it is addressed and may contain
> information that is privileged, confidential, or otherwise protected
> from disclosure. Distribution or copying of this e-mail, or the
> information contained herein, to anyone other than the intended
> recipient is prohibited.
>

> _______________________________________________
> linux-cifs-client mailing list
> linux-cifs-client@...
> https://lists.samba.org/mailman/listinfo/linux-cifs-client

This patch to cifs.ko should fix the problem, and you do not have to cifs mount with forcedirectio

http://master.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff_plain;h=cb876f451455b6187a7d69de2c112c45ec4b7f99

Regards,

Shirish

_______________________________________________
linux-cifs-client mailing list
linux-cifs-client@...
https://lists.samba.org/mailman/listinfo/linux-cifs-client