[Fwd: File Locking and Permissions Issue]

[Jack Lauman]

-------- Original Message --------
Subject: File Locking and Permissions Issue
Date: Wed, 14 May 2008 15:10:28 -0700
From: Jack Lauman <jlauman@...>
Organization: nwcascades.com
To: mailto:samba@...

I'm trying to get Lacerte 2007 Tax Accounting software working on a
Samba v3.0.28a based server.  When one user is logged in it works fine.
   When two or more users are accessing the database files it slows to a
crawl.

Lacerte's tech support personnel set up an identical scenereo in their
lab.  Here's what the found:

<quote>
I compared the open files with one computer in Lacerte vs. two
computers in Lacerte and noticed one thing peculiar: when one computer
is using Lacerte, all files are opened with exclusive+batch oplocks
including Data1i07.dbf, however when 2 computers are running Lacerte, a
few files open without oplocks, notably data1i07.dbf.  I have also
attached a copy of my smbstatus output (put into an Excel spreadsheet,
computer 1 highlighted in yellow, computer 2 in blue.)  This seems to be
the same way it was working on your server.

One more thing to note - I was logged in as the same user on both
workstations - if you can try that configuration and it works, that
might clue us in to some kind of permissions issue.
</quote>

I've attached both files to this message.  Any help in resolving this
matter would be greatly appreciated.

Thanks,

Jack Lauman


[global]
        server string = Linux Samba Server V %v
   workgroup = WORKGROUP
        interfaces = eth0 127.0.0.1
        bind interfaces only = yes
   security = user
        log file = /var/log/samba/log.%m
        time server = yes
        socket options = TCP_NODELAY IPTOS_LOWDELAY
        os level = 65
        preferred master = yes
        domain master = yes
        local master = yes
        guest account = smbuser
        printcap name = cups
        printing = cups
        wins support = yes
        use client driver = yes
        encrypt passwords = yes
        smb passwd file = /etc/samba/smbpasswd
   idmap uid = 16777216-33554431
   idmap gid = 16777216-33554431
   template shell = /bin/false
   winbind use default domain = no
[homes]
        browsable = no
        writable = yes
[intuit]
        comment = Lacerte Server
        path = /home/arends/intuit
        read only = no
        guest ok = yes

[public]
        comment = Public file access
        path = /home/public
        read only = no
        create mask = 2775
        directory mask = 0775
        guest ok = yes
[web]
        comment = WWW
        path = /var/www/html
        valid users = arends
        read only = no
[printers]
        printer = raw
        comment = All Printers
        path = /var/spool/samba
        guest ok = yes
        public = yes
        writable = yes
        printable = yes
        browseable = yes
[print$]
        comment = Printer Drivers
        path = /etc/samba/printer
        guest ok = no
        broweseable = yes
        read only = yes
        write list = arends,gijane,root


--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/listinfo/samba

[Volker Lendecke]

On Wed, May 14, 2008 at 03:30:54PM -0700, Jack Lauman wrote:
> I'm trying to get Lacerte 2007 Tax Accounting software working on a
> Samba v3.0.28a based server.  When one user is logged in it works fine.
>   When two or more users are accessing the database files it slows to a
> crawl.

But it still does work?

> I compared the open files with one computer in Lacerte vs. two
> computers in Lacerte and noticed one thing peculiar: when one computer
> is using Lacerte, all files are opened with exclusive+batch oplocks
> including Data1i07.dbf, however when 2 computers are running Lacerte, a
> few files open without oplocks, notably data1i07.dbf.  I have also

That's expected. Oplocks with r/w are only possible for
exclusive access.

I've had a customer experiencing exactly the same issue that
you have. Different application, but it was used against
Samba in production and it was terribly slow. They tested it
against Windows -- it was fast. It turned out that against
Samba they had several concurrent users and the test against
Windows only had one user. Just the simple operation of
copying the file away from the Windows server while the test
on Windows was running made the app show exactly the same
behaviour against Windows as it formerly had against Samba:
It was almost unusably slow because its oplock had been
broken.

I'm not saying that Samba is bug-free here, but what you are
describing really smells like a horrendously badly written
Windows application that has never been really tested with
multi-user access.

Volker


--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/listinfo/samba

[Michael Heydon-2]

Jack Lauman wrote:
> <snip>
>
> I compared the open files with one computer in Lacerte vs. two
> computers in Lacerte and noticed one thing peculiar: when one computer
> is using Lacerte, all files are opened with exclusive+batch oplocks
> including Data1i07.dbf, however when 2 computers are running Lacerte, a
> few files open without oplocks, notably data1i07.dbf.
>
I'm assuming that both users need to write to these files? Maybe I'm
missing something but this seems to be entirely expected behaviour.
Oplocks allow a client to cache data rather than having to constantly
sync to the server, obviously if there is more than one client doing
this things break.

You could use fake oplocks to grant oplocks to all clients, but unless
the application is designed for it (which I doubt it is) you will just
wind up corrupting your data.

If the application is regularly opening and closing files (and therefore
possibly being granted oplocks and then having them broken) you might
find that performance improves by disabling oplocks altogether (well,
performance for multiple users, performance for a single user would suffer).
> <snip> I've attached both files to this message.  Any help in
> resolving this
> matter would be greatly appreciated.
>
I think the list strips non-text attachments, so no excel file. Not that
I think it's terribly important since it sounds like your system is
working exactly as it should.
> Thanks,
>
> Jack Lauman
>


*Michael Heydon - IT Administratorr *
michaelh@... <mailto:michaelh@...>

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/listinfo/samba

[Scott Lovenberg]

Michael Heydon wrote:

> Jack Lauman wrote:
>> <snip>
>>
>> I compared the open files with one computer in Lacerte vs. two
>> computers in Lacerte and noticed one thing peculiar: when one computer
>> is using Lacerte, all files are opened with exclusive+batch oplocks
>> including Data1i07.dbf, however when 2 computers are running Lacerte, a
>> few files open without oplocks, notably data1i07.dbf.
> I'm assuming that both users need to write to these files? Maybe I'm
> missing something but this seems to be entirely expected behaviour.
> Oplocks allow a client to cache data rather than having to constantly
> sync to the server, obviously if there is more than one client doing
> this things break.
>
> You could use fake oplocks to grant oplocks to all clients, but unless
> the application is designed for it (which I doubt it is) you will just
> wind up corrupting your data.
>
> If the application is regularly opening and closing files (and
> therefore possibly being granted oplocks and then having them broken)
> you might find that performance improves by disabling oplocks
> altogether (well, performance for multiple users, performance for a
> single user would suffer).
>> <snip> I've attached both files to this message.  Any help in
>> resolving this
>> matter would be greatly appreciated.
>>
> I think the list strips non-text attachments, so no excel file. Not
> that I think it's terribly important since it sounds like your system
> is working exactly as it should.
>> Thanks,
>>
>> Jack Lauman
>>
>
>
> *Michael Heydon - IT Administratorr *
> michaelh@... <mailto:michaelh@...>
>

Just a thought, but if you're using an enterprise distro, you might be
able to cheat the system by granting fake oplocks and using a
distributed file system, but there still could be coherency and race
conditions under some circumstances.  It would probably depend on your
usage patterns for the application as to whether you could push the
envelope and get away with it.  If your access is mostly write once and
read thereafter, it might be alright.  YMMV.

I've always had issues with Office 2000 and multiple users.  You can
almost feel the whiplash of Access or Excel slowing down the moment a
second connection is established.  Though, I must admit, I've never had
corruption due to concurrent access, so it at least works for the speed
trade-off.  Unless the app slows down to a crawl, it's probably better
safe than sorry.  Especially if you're potentially rolling a corrupted
file in to your backups.
--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/listinfo/samba

[Jack Lauman]

For some reason the excel file was stripped in my last post.  (See
below).  We would appreciate any help we can get on this.  Intuit tried
this on their test server and we don't know whats causing the files to
open without oplocks when a second user tries to access the database.

Thanks,

Jack

Jack Lauman wrote:

>
>
> -------- Original Message --------
> Subject: File Locking and Permissions Issue
> Date: Wed, 14 May 2008 15:10:28 -0700
> From: Jack Lauman <jlauman@...>
> Organization: nwcascades.com
> To: mailto:samba@...
>
> I'm trying to get Lacerte 2007 Tax Accounting software working on a
> Samba v3.0.28a based server.  When one user is logged in it works fine.
>   When two or more users are accessing the database files it slows to a
> crawl.
>
> Lacerte's tech support personnel set up an identical scenereo in their
> lab.  Here's what the found:
>
> <quote>
> I compared the open files with one computer in Lacerte vs. two
> computers in Lacerte and noticed one thing peculiar: when one computer
> is using Lacerte, all files are opened with exclusive+batch oplocks
> including Data1i07.dbf, however when 2 computers are running Lacerte, a
> few files open without oplocks, notably data1i07.dbf.  I have also
> attached a copy of my smbstatus output (put into an Excel spreadsheet,
> computer 1 highlighted in yellow, computer 2 in blue.)  This seems to be
> the same way it was working on your server.
>
> One more thing to note - I was logged in as the same user on both
> workstations - if you can try that configuration and it works, that
> might clue us in to some kind of permissions issue.
> </quote>
>
> I've attached both files to this message.  Any help in resolving this
> matter would be greatly appreciated.
>
> Thanks,
>
> Jack Lauman
>

Locked files:

Pid Uid DenyMode Access R/W Oplock SharePath Name Time
15314 500 DENY_WRITE 0x20089 RDONLY EXCLUSIVE+BATCH
/home/arends/intuit Lacerte/07tax/Setup07/Setup.XML Tue May 13 23:10:24 2008
15001 500 DENY_NONE 0x100001 RDONLY NONE /home/arends/intuit . Tue May
13 23:07:44 2008
15314 500 DENY_NONE 0x2019f RDWR EXCLUSIVE+BATCH /home/arends/intuit
Lacerte/07tax/IDATA/DATA1I07.DBF Tue May 13 23:10:25 2008
15314 500 DENY_NONE 0x2019f RDWR EXCLUSIVE+BATCH /home/arends/intuit
Lacerte/07tax/IDATA/urn/User002.LW7 Tue May 13 23:10:24 2008
15314 500 DENY_WRITE 0x20089 RDONLY EXCLUSIVE+BATCH
/home/arends/intuit Lacerte/06tax/OPTION06/OPINDEX.W6 Tue May 13
23:10:24 2008
15314 500 DENY_NONE 0x20089 RDONLY EXCLUSIVE+BATCH
/home/arends/intuit Lacerte/07tax/IDATA/DDRIDI07.DAT Tue May 13 23:10:24
2008
15314 500 DENY_NONE 0x2019f RDWR EXCLUSIVE+BATCH /home/arends/intuit
Lacerte/07tax/IDATA/DATA1I07.MDX Tue May 13 23:10:17 2008
15314 500 DENY_NONE 0x20089 RDONLY EXCLUSIVE+BATCH
/home/arends/intuit Lacerte/07tax/IDATA/urn/User001.LW7 Tue May 13
23:10:24 2008

Locked files:
Pid Uid DenyMode Access R/W Oplock SharePath Name Time
15001 500 DENY_NONE 0x100001 RDONLY NONE /home/arends/intuit . Tue May
13 23:07:44 2008
15314 500 DENY_NONE 0x2019f RDWR NONE /home/arends/intuit
Lacerte/07tax/IDATA/DATA1I07.DBF   Tue May 13 23:10:25 2008
15001 500 DENY_NONE 0x2019f RDWR NONE /home/arends/intuit
Lacerte/07tax/IDATA/DATA1I07.DBF Tue May 13 23:11:20 2008
15314 500 DENY_NONE 0x2019f RDWR NONE /home/arends/intuit
Lacerte/07tax/IDATA/urn/User002.LW7 Tue May 13 23:10:24 2008
15001 500 DENY_NONE 0x20089 RDONLY EXCLUSIVE+BATCH
/home/arends/intuit Lacerte/07tax/IDATA/DDRIDI07.DAT Tue May 13 23:11:16
2008
15001 500 DENY_NONE 0x2019f RDWR EXCLUSIVE+BATCH /home/arends/intuit
Lacerte/07tax/IDATA/urn/User003.LW7 Tue May 13 23:11:16 2008
15001 500 DENY_NONE 0x20089 RDONLY EXCLUSIVE+BATCH
/home/arends/intuit Lacerte/07tax/IDATA/urn/User001.LW7 Tue May 13
23:11:16 2008

[global]
        server string = Linux Samba Server V %v
   workgroup = WORKGROUP
        interfaces = eth0 127.0.0.1
        bind interfaces only = yes
   security = user
        log file = /var/log/samba/log.%m
        time server = yes
        socket options = TCP_NODELAY IPTOS_LOWDELAY
        os level = 65
        preferred master = yes
        domain master = yes
        local master = yes
        guest account = smbuser
        printcap name = cups
        printing = cups
        wins support = yes
        use client driver = yes
        encrypt passwords = yes
        smb passwd file = /etc/samba/smbpasswd
   idmap uid = 16777216-33554431
   idmap gid = 16777216-33554431
   template shell = /bin/false
   winbind use default domain = no
[homes]
        browsable = no
        writable = yes
[intuit]
        comment = Lacerte Server
        path = /home/arends/intuit
        read only = no
        guest ok = yes

[public]
        comment = Public file access
        path = /home/public
        read only = no
        create mask = 2775
        directory mask = 0775
        guest ok = yes
[web]
        comment = WWW
        path = /var/www/html
        valid users = arends
        read only = no
[printers]
        printer = raw
        comment = All Printers
        path = /var/spool/samba
        guest ok = yes
        public = yes
        writable = yes
        printable = yes
        browseable = yes
[print$]
        comment = Printer Drivers
        path = /etc/samba/printer
        guest ok = no
        broweseable = yes
        read only = yes
        write list = arends,gijane,root

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/listinfo/samba

[Volker Lendecke]

On Thu, May 15, 2008 at 10:45:51AM -0700, Jack Lauman wrote:
> For some reason the excel file was stripped in my last post.  (See
> below).  We would appreciate any help we can get on this.  Intuit tried
> this on their test server and we don't know whats causing the files to
> open without oplocks when a second user tries to access the database.

Did you read our replies to your initial post at all?

If Intuit tells you that they do not know why a second
opener does not get an oplock you should really escalate
that support issue much, much higher inside their
organization. You have have very obviously been put off by
some first-level support person that has no clue about the
products.

Volker


--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/listinfo/samba

[Jason Arends]

Volker Lendecke <Volker.Lendecke <at> SerNet.DE> writes:

>
> On Thu, May 15, 2008 at 10:45:51AM -0700, Jack Lauman wrote:
> > For some reason the excel file was stripped in my last post.  (See
> > below).  We would appreciate any help we can get on this.  Intuit tried
> > this on their test server and we don't know whats causing the files to
> > open without oplocks when a second user tries to access the database.
>
> Did you read our replies to your initial post at all?
>
> If Intuit tells you that they do not know why a second
> opener does not get an oplock you should really escalate
> that support issue much, much higher inside their
> organization. You have have very obviously been put off by
> some first-level support person that has no clue about the
> products.
>
> Volker
>
>

Thanks for the responses so far... I am actually the tech from Lacerte who was
working with Jack - I'd like to clarify some things to try to help find a
resolution:

- The software is designed for multiple users to access the database on a
network and works that way for thousands of customers (using Windows servers,
Linux is actually not supported by us and we don't test on it)
- In Jack's case, each computer can access the program fine when the database is
local, and when the database is on the server it works fine as long as only one
computer accesses it
- I have seen cases where performance degradation in our software occurs when
oplocks are disabled on Windows networks.  Oplocks was not disabled on Jack's
machines, but we did try disabling it on both the server and workstations to see
if it made a difference and it did not help.
- I now realize that when a second user accesses a file, the server will break
the oplock the first user had on it, which is why it showed up that way in the
log
- I tested our software on my home network with Samba on a Linux server and 2
workstations (XP and Vista) and didn't run into the same problem as Jack - I was
able to open the software on both workstations and access the same database
without the performance issue.  My smbstatus logs also appeared identical to
Jack's as far as the oplocks go.  The smb.conf file that Jack attached was the
one from my server that I had sent him to compare with his, so consider that as
the working one.  One difference is that I was authenticating as the same user
from 2 different workstations while Jack had different users.

I'd like to have Jack set up a share on one of the Windows workstations and
point 2 computers at that database and see if the same issue occurs, just to
make sure we are looking in the right direction.

(For the record, the issue already was escalated and I am not first-level
support.  Unfortunately if it went higher the case would probably be dismissed
as "OS not supported")

~jason

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/listinfo/samba

[Jeremy Allison]

On Fri, May 16, 2008 at 05:07:50PM +0000, Jason Arends wrote:

> Thanks for the responses so far... I am actually the tech from Lacerte who was
> working with Jack - I'd like to clarify some things to try to help find a
> resolution:

I appreciate the help and time you've been willing to put into this.

> - The software is designed for multiple users to access the database on a
> network and works that way for thousands of customers (using Windows servers,
> Linux is actually not supported by us and we don't test on it)
> - In Jack's case, each computer can access the program fine when the database is
> local, and when the database is on the server it works fine as long as only one
> computer accesses it
> - I have seen cases where performance degradation in our software occurs when
> oplocks are disabled on Windows networks.  Oplocks was not disabled on Jack's
> machines, but we did try disabling it on both the server and workstations to see
> if it made a difference and it did not help.
> - I now realize that when a second user accesses a file, the server will break
> the oplock the first user had on it, which is why it showed up that way in the
> log
> - I tested our software on my home network with Samba on a Linux server and 2
> workstations (XP and Vista) and didn't run into the same problem as Jack - I was
> able to open the software on both workstations and access the same database
> without the performance issue.  My smbstatus logs also appeared identical to
> Jack's as far as the oplocks go.  The smb.conf file that Jack attached was the
> one from my server that I had sent him to compare with his, so consider that as
> the working one.  One difference is that I was authenticating as the same user
> from 2 different workstations while Jack had different users.
>
> I'd like to have Jack set up a share on one of the Windows workstations and
> point 2 computers at that database and see if the same issue occurs, just to
> make sure we are looking in the right direction.

That seems like a good plan. Please let the list know if you get stuck
and need further help.

> (For the record, the issue already was escalated and I am not first-level
> support.  Unfortunately if it went higher the case would probably be dismissed
> as "OS not supported")

How can we change that ? There are many organizations (Linux distro's,
OEM's, major vendors like IBM, HP, Sun etc.) who ship Samba as a
supported product. Is there something we can do to help change the
policy in your company ?

Thanks,

Jeremy.
--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/listinfo/samba

[Volker Lendecke]

On Fri, May 16, 2008 at 10:24:23AM -0700, Jeremy Allison wrote:
> > I'd like to have Jack set up a share on one of the
> > Windows workstations and point 2 computers at that
> > database and see if the same issue occurs, just to make
> > sure we are looking in the right direction.
>
> That seems like a good plan. Please let the list know if
> you get stuck and need further help.

In case it works differently between files stored on Windows
and files stored on Samba, we need comparative sniffs.

> > (For the record, the issue already was escalated and I
> > am not first-level support.  Unfortunately if it went
> > higher the case would probably be dismissed as "OS not
> > supported")

Apologies, I was quite rude. But I really got the impression
that the first and second answers on this list had been
ignored. And a second opener on the same file does
definitely break an oplock :-)

If you have your software running everywhere even without
oplocks, there must be something else that causes the
slowdown. Sniffs might really help.

> How can we change that ? There are many organizations
> (Linux distro's, OEM's, major vendors like IBM, HP, Sun
> etc.) who ship Samba as a supported product. Is there
> something we can do to help change the policy in your
> company ?

Yes, I'd be really happy to help.

Volker


--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/listinfo/samba