Nabble - Samba - samba-technical

Re: important: Wrong logic (to my mind) in schema_set.c

2009-11-29T15:57:11Z

On Mon, 2009-11-30 at 01:07 +0300, Matthieu Patou wrote:

> On 30/11/2009 00:30, Andrew Bartlett wrote:
> > On Sun, 2009-11-29 at 17:02 +0300, Matthieu Patou wrote:
> >
> >> Dear all, I've been trying to upgrade a very old provision (alpha3) I've
> >> been hit by errors like this:
> >>
> >> _ldb.LdbError: (80, 'schema_load_init: dsdb_set_schema() failed: 32:No
> >> such object')
> >> After a careful investigation I am 99% sure that those two changes
> >> (introduced in changeset 6b05a907) are wrong
> >>
> >> @@ -135,7 +135,7 @@ static int dsdb_schema_set_attributes(struct
> >> ldb_context *ldb, struct dsdb_schem
> >>
> >> mod_msg = ldb_msg_diff(ldb, res->msgs[0], msg);
> >> if (mod_msg->num_elements> 0) {
> >> - ret = ldb_modify(ldb, mod_msg);
> >> + ret = samdb_replace(ldb, mem_ctx, mod_msg);
> >> }
> >> }
> >>
> >> @@ -163,7 +163,7 @@ static int dsdb_schema_set_attributes(struct
> >> ldb_context *ldb, struct dsdb_schem
> >>
> >> mod_msg = ldb_msg_diff(ldb, res_idx->msgs[0], msg_idx);
> >> if (mod_msg->num_elements> 0) {
> >> - ret = ldb_modify(ldb, mod_msg);
> >> + ret = samdb_replace(ldb, mem_ctx, mod_msg);
> >> }
> >>
> >> Because the different elements with different flags (add,replace,delete)
> >> and using samdb_replace will override this flag by using just replace.
> >> Obviously if the ldb_diff was telling you that you should add the
> >> attribute 'foo' trying to make ldb modify the 'foo' attribute will not
> >> be very good.
> >>
> >> Also I'm quite puzzled by the use ldb_msg_diff and the order of
> >> parameters. Because res->msgs[0] is an message element obtained from a
> >> search in the LDB database represented by the ldb object.
> >> So to my mind by doing
> >> mod_msg = ldb_msg_diff(ldb, res->msgs[0], msg);
> >> we are comparing res->msgs[0] and msg and using res->msgs[0] as a
> >> reference, so there is no point trying to apply changes in the ldb
> >> object as the attributes have already the value contained in the
> >> different messageElement of the mod_msg object.
> >>
> >> So I think parameters should be swapped it makes just sens from the
> >> point of view of the comparison and the modification of the ldb but for
> >> the overall logic I'm much more reserved.
> >>
> > I've looked over the code, and while I agree the samdb_replace seems
> > wrong, but before we added it, it just didn't work. The goal of this
> > routine is to build new @ATTRIBUTE records and @INDEX records, and to
> > then, if they differ from the DB, replace them in a DB.
> >
> >
> It's not very clear from what @ATTRIBUTE and @INDEX are built can you
> explain me ?

What we do is walk over the schema, and if the schema elements tell us
that an object is case sensitive, or if it is indexed, then we add it to
those attributes.

The idea is then that every time we load the schema, we check that this
on-disk record (used by ldb before the schema is loaded, and used to
determine indexing) is in sync with the schema.

> > The purpose of res->msgs[0] is to give the current DB as a base to the
> > diff function.
> >
> >
> Yeah I checked that ldb_msg_diff use the last ldbMessage as the ref, I
> tried to revert to ldb_modify but it didn't make the trick, as it was
> proposing to add attribute to @ATTRIBUTE that were already in the database.
>
> I'll try to remake trace so that we can further discuss it.

Thanks,

Andrew Bartlett

--
Andrew Bartlett http://samba.org/~abartlet/
Authentication Developer, Samba Team http://samba.org
Samba Developer, Cisco Inc.

signature.asc (196 bytes) Download Attachment

Re: important: Wrong logic (to my mind) in schema_set.c

2009-11-29T15:41:23Z

Hello,

I finally rested the whole thing reverting to ldb_modify is to my mind
the good solution.

I attached a patch for this and 2 patch for updateprovision.

The whole serie allow update of quite old provision (ie. alpha3).

Matthieu

On 30/11/2009 01:07, Matthieu Patou wrote:

> On 30/11/2009 00:30, Andrew Bartlett wrote:
>> On Sun, 2009-11-29 at 17:02 +0300, Matthieu Patou wrote:
>>> Dear all, I've been trying to upgrade a very old provision (alpha3) I've
>>> been hit by errors like this:
>>>
>>> _ldb.LdbError: (80, 'schema_load_init: dsdb_set_schema() failed: 32:No
>>> such object')
>>> After a careful investigation I am 99% sure that those two changes
>>> (introduced in changeset 6b05a907) are wrong
>>>
>>> @@ -135,7 +135,7 @@ static int dsdb_schema_set_attributes(struct
>>> ldb_context *ldb, struct dsdb_schem
>>>
>>> mod_msg = ldb_msg_diff(ldb, res->msgs[0], msg);
>>> if (mod_msg->num_elements> 0) {
>>> - ret = ldb_modify(ldb, mod_msg);
>>> + ret = samdb_replace(ldb, mem_ctx, mod_msg);
>>> }
>>> }
>>>
>>> @@ -163,7 +163,7 @@ static int dsdb_schema_set_attributes(struct
>>> ldb_context *ldb, struct dsdb_schem
>>>
>>> mod_msg = ldb_msg_diff(ldb, res_idx->msgs[0], msg_idx);
>>> if (mod_msg->num_elements> 0) {
>>> - ret = ldb_modify(ldb, mod_msg);
>>> + ret = samdb_replace(ldb, mem_ctx, mod_msg);
>>> }
>>>
>>> Because the different elements with different flags (add,replace,delete)
>>> and using samdb_replace will override this flag by using just replace.
>>> Obviously if the ldb_diff was telling you that you should add the
>>> attribute 'foo' trying to make ldb modify the 'foo' attribute will not
>>> be very good.
>>>
>>> Also I'm quite puzzled by the use ldb_msg_diff and the order of
>>> parameters. Because res->msgs[0] is an message element obtained from a
>>> search in the LDB database represented by the ldb object.
>>> So to my mind by doing
>>> mod_msg = ldb_msg_diff(ldb, res->msgs[0], msg);
>>> we are comparing res->msgs[0] and msg and using res->msgs[0] as a
>>> reference, so there is no point trying to apply changes in the ldb
>>> object as the attributes have already the value contained in the
>>> different messageElement of the mod_msg object.
>>>
>>> So I think parameters should be swapped it makes just sens from the
>>> point of view of the comparison and the modification of the ldb but for
>>> the overall logic I'm much more reserved.
>> I've looked over the code, and while I agree the samdb_replace seems
>> wrong, but before we added it, it just didn't work. The goal of this
>> routine is to build new @ATTRIBUTE records and @INDEX records, and to
>> then, if they differ from the DB, replace them in a DB.
>>
> It's not very clear from what @ATTRIBUTE and @INDEX are built can you
> explain me ?
>> The purpose of res->msgs[0] is to give the current DB as a base to the
>> diff function.
>>
> Yeah I checked that ldb_msg_diff use the last ldbMessage as the ref, I
> tried to revert to ldb_modify but it didn't make the trick, as it was
> proposing to add attribute to @ATTRIBUTE that were already in the database.
>
> I'll try to remake trace so that we can further discuss it.
>
> Matthieu.

[0001-s4-remove-a-leftover-debug-message.patch]

From 11509ff6cfe4ec9b03f6f1a6112033076b8c23d6 Mon Sep 17 00:00:00 2001 From: Matthieu Patou <mat@...> Date: Mon, 30 Nov 2009 02:36:52 +0300 Subject: [PATCH 1/3] s4: remove a leftover debug message :-) --- source4/scripting/bin/upgradeprovision | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/source4/scripting/bin/upgradeprovision b/source4/scripting/bin/upgradeprovision index 4b0cfec..f8735fc 100755 --- a/source4/scripting/bin/upgradeprovision +++ b/source4/scripting/bin/upgradeprovision @@ -341,7 +341,6 @@ def dn_sort(x,y): # it also populate hash structure for later use in the upgrade process def handle_security_desc(ischema,att,msgElt,hashallSD,old,new): if ischema == 1 and att == "defaultSecurityDescriptor" and msgElt.flags() == ldb.FLAG_MOD_REPLACE: - print "coin coin" hashSD = {} hashSD["oldSD"] = old[0][att] hashSD["newSD"] = new[0][att] -- 1.6.3.3

[0002-s4-reorder-action-to-permit-old-provision-to-be-corr.patch]

From c49bc545ebe68ec486da5521282b8981cc2dea8a Mon Sep 17 00:00:00 2001 From: Matthieu Patou <mat@...> Date: Mon, 30 Nov 2009 02:37:35 +0300 Subject: [PATCH 2/3] s4: reorder action to permit old provision to be correctly upgraded --- source4/scripting/bin/upgradeprovision | 13 ++++++++----- 1 files changed, 8 insertions(+), 5 deletions(-) diff --git a/source4/scripting/bin/upgradeprovision b/source4/scripting/bin/upgradeprovision index f8735fc..bcf4d10 100755 --- a/source4/scripting/bin/upgradeprovision +++ b/source4/scripting/bin/upgradeprovision @@ -495,6 +495,7 @@ def check_diff_name(newpaths,paths,creds,session,basedn,names,ischema): # Connect to the reference provision and get all the attribute in the partition referred by name newsam_ldb = Ldb(newpaths.samdb, session_info=session, credentials=creds,lp=lp) sam_ldb = Ldb(paths.samdb, session_info=session, credentials=creds,lp=lp, options=["modules:samba_dsdb"]) + sam_ldb.transaction_start() if ischema: res = newsam_ldb.search(expression="objectClass=*",base=basedn, scope=SCOPE_SUBTREE,attrs=["dn"]) res2 = sam_ldb.search(expression="objectClass=*",base=basedn, scope=SCOPE_SUBTREE,attrs=["dn"]) @@ -502,6 +503,7 @@ def check_diff_name(newpaths,paths,creds,session,basedn,names,ischema): res = newsam_ldb.search(expression="objectClass=*",base=basedn, scope=SCOPE_SUBTREE,attrs=["dn"],controls=["search_options:1:2"]) res2 = sam_ldb.search(expression="objectClass=*",base=basedn, scope=SCOPE_SUBTREE,attrs=["dn"],controls=["search_options:1:2"]) + sam_ldb.transaction_commit() # Create a hash for speeding the search of new object for i in range(0,len(res)): hash_new[str(res[i]["dn"]).lower()] = res[i]["dn"] @@ -533,9 +535,10 @@ def check_diff_name(newpaths,paths,creds,session,basedn,names,ischema): sam_ldb.set_schema_from_ldb(schema.ldb) # And now we can connect to the DB - the schema won't be loaded from the DB sam_ldb.connect(paths.samdb) - sam_ldb.transaction_start() else: - sam_ldb.transaction_start() + sam_ldb = Ldb(paths.samdb, session_info=session, credentials=creds,lp=lp, options=["modules:samba_dsdb"]) + + sam_ldb.transaction_start() empty = ldb.Message() message(SIMPLE,"There are %d missing objects"%(len(listMissing))) @@ -762,12 +765,12 @@ newpaths = get_paths(targetdir=provisiondir) populate_backlink(newpaths,creds,session,names.schemadn) # Check the difference update_basesamdb(newpaths,paths,names) -update_secrets(newpaths,paths,creds,session) -update_privilege(newpaths,paths) -update_machine_account_password(paths,creds,session,names) if opts.full: update_samdb(newpaths,paths,creds,session,names) +update_secrets(newpaths,paths,creds,session) +update_privilege(newpaths,paths) +update_machine_account_password(paths,creds,session,names) # SD should be created with admin but as some previous acl were so wrong that admin can't modify them we have first # to recreate them with the good form but with system account and then give the ownership to admin ... admin_session_info = admin_session(lp, str(names.domainsid)) -- 1.6.3.3

[0003-s4-replace-wrong-sambdb_replace-by-ldb_modify.patch]

From 87be99a95103585515f5c0b79ef3dfc1b731d240 Mon Sep 17 00:00:00 2001 From: Matthieu Patou <mat@...> Date: Mon, 30 Nov 2009 02:38:46 +0300 Subject: [PATCH 3/3] s4: replace wrong sambdb_replace by ldb_modify --- source4/dsdb/schema/schema_set.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/source4/dsdb/schema/schema_set.c b/source4/dsdb/schema/schema_set.c index c321bc9..a984c03 100644 --- a/source4/dsdb/schema/schema_set.c +++ b/source4/dsdb/schema/schema_set.c @@ -137,7 +137,7 @@ static int dsdb_schema_set_attributes(struct ldb_context *ldb, struct dsdb_schem mod_msg = ldb_msg_diff(ldb, res->msgs[0], msg); if (mod_msg->num_elements > 0) { - ret = samdb_replace(ldb, mem_ctx, mod_msg); + ldb_modify(ldb,mod_msg); } } @@ -165,7 +165,7 @@ static int dsdb_schema_set_attributes(struct ldb_context *ldb, struct dsdb_schem mod_msg = ldb_msg_diff(ldb, res_idx->msgs[0], msg_idx); if (mod_msg->num_elements > 0) { - ret = samdb_replace(ldb, mem_ctx, mod_msg); + ldb_modify(ldb,mod_msg); } } if (ret == LDB_ERR_OPERATIONS_ERROR || ret == LDB_ERR_INSUFFICIENT_ACCESS_RIGHTS || ret == LDB_ERR_INVALID_DN_SYNTAX) { -- 1.6.3.3

Creating vanilla object for helping the upgrade of non provisionned object

2009-11-29T14:28:29Z

Hello andrew,

I was thinking and making tests with creation of vanilla objects in a
provision.
The goal of this creation is to create same objects in the current
provision and in the reference and to see how they differ and then to
reapply on non provisionned objects (ie. users create after provision)
the change.

But it turns out to be a quite complicated task: you have to create all
the hierarchy for object and soon created object will have to comply
with the mustContain attribute so it's gonna be quite hard soon to
create with just a program.
And at the other hand I'm not very sure of what it can brings.
It is supposed to help with the upgrade of non provisionned object but I
curious to know what are the attribute that are forced to a certain
value like it was done more or less with template.ldb before.

Matthieu.

Re: important: Wrong logic (to my mind) in schema_set.c

2009-11-29T14:07:52Z

On 30/11/2009 00:30, Andrew Bartlett wrote:

> On Sun, 2009-11-29 at 17:02 +0300, Matthieu Patou wrote:
>
>> Dear all, I've been trying to upgrade a very old provision (alpha3) I've
>> been hit by errors like this:
>>
>> _ldb.LdbError: (80, 'schema_load_init: dsdb_set_schema() failed: 32:No
>> such object')
>> After a careful investigation I am 99% sure that those two changes
>> (introduced in changeset 6b05a907) are wrong
>>
>> @@ -135,7 +135,7 @@ static int dsdb_schema_set_attributes(struct
>> ldb_context *ldb, struct dsdb_schem
>>
>> mod_msg = ldb_msg_diff(ldb, res->msgs[0], msg);
>> if (mod_msg->num_elements> 0) {
>> - ret = ldb_modify(ldb, mod_msg);
>> + ret = samdb_replace(ldb, mem_ctx, mod_msg);
>> }
>> }
>>
>> @@ -163,7 +163,7 @@ static int dsdb_schema_set_attributes(struct
>> ldb_context *ldb, struct dsdb_schem
>>
>> mod_msg = ldb_msg_diff(ldb, res_idx->msgs[0], msg_idx);
>> if (mod_msg->num_elements> 0) {
>> - ret = ldb_modify(ldb, mod_msg);
>> + ret = samdb_replace(ldb, mem_ctx, mod_msg);
>> }
>>
>> Because the different elements with different flags (add,replace,delete)
>> and using samdb_replace will override this flag by using just replace.
>> Obviously if the ldb_diff was telling you that you should add the
>> attribute 'foo' trying to make ldb modify the 'foo' attribute will not
>> be very good.
>>
>> Also I'm quite puzzled by the use ldb_msg_diff and the order of
>> parameters. Because res->msgs[0] is an message element obtained from a
>> search in the LDB database represented by the ldb object.
>> So to my mind by doing
>> mod_msg = ldb_msg_diff(ldb, res->msgs[0], msg);
>> we are comparing res->msgs[0] and msg and using res->msgs[0] as a
>> reference, so there is no point trying to apply changes in the ldb
>> object as the attributes have already the value contained in the
>> different messageElement of the mod_msg object.
>>
>> So I think parameters should be swapped it makes just sens from the
>> point of view of the comparison and the modification of the ldb but for
>> the overall logic I'm much more reserved.
>>
> I've looked over the code, and while I agree the samdb_replace seems
> wrong, but before we added it, it just didn't work. The goal of this
> routine is to build new @ATTRIBUTE records and @INDEX records, and to
> then, if they differ from the DB, replace them in a DB.
>
>

It's not very clear from what @ATTRIBUTE and @INDEX are built can you
explain me ?
> The purpose of res->msgs[0] is to give the current DB as a base to the
> diff function.
>
>
Yeah I checked that ldb_msg_diff use the last ldbMessage as the ref, I
tried to revert to ldb_modify but it didn't make the trick, as it was
proposing to add attribute to @ATTRIBUTE that were already in the database.

I'll try to remake trace so that we can further discuss it.

Matthieu.

Re: important: Wrong logic (to my mind) in schema_set.c

2009-11-29T13:30:29Z

On Sun, 2009-11-29 at 17:02 +0300, Matthieu Patou wrote:

> Dear all, I've been trying to upgrade a very old provision (alpha3) I've
> been hit by errors like this:
>
> _ldb.LdbError: (80, 'schema_load_init: dsdb_set_schema() failed: 32:No
> such object')
> After a careful investigation I am 99% sure that those two changes
> (introduced in changeset 6b05a907) are wrong
>
> @@ -135,7 +135,7 @@ static int dsdb_schema_set_attributes(struct
> ldb_context *ldb, struct dsdb_schem
>
> mod_msg = ldb_msg_diff(ldb, res->msgs[0], msg);
> if (mod_msg->num_elements > 0) {
> - ret = ldb_modify(ldb, mod_msg);
> + ret = samdb_replace(ldb, mem_ctx, mod_msg);
> }
> }
>
> @@ -163,7 +163,7 @@ static int dsdb_schema_set_attributes(struct
> ldb_context *ldb, struct dsdb_schem
>
> mod_msg = ldb_msg_diff(ldb, res_idx->msgs[0], msg_idx);
> if (mod_msg->num_elements > 0) {
> - ret = ldb_modify(ldb, mod_msg);
> + ret = samdb_replace(ldb, mem_ctx, mod_msg);
> }
>
> Because the different elements with different flags (add,replace,delete)
> and using samdb_replace will override this flag by using just replace.
> Obviously if the ldb_diff was telling you that you should add the
> attribute 'foo' trying to make ldb modify the 'foo' attribute will not
> be very good.
>
> Also I'm quite puzzled by the use ldb_msg_diff and the order of
> parameters. Because res->msgs[0] is an message element obtained from a
> search in the LDB database represented by the ldb object.
> So to my mind by doing
> mod_msg = ldb_msg_diff(ldb, res->msgs[0], msg);
> we are comparing res->msgs[0] and msg and using res->msgs[0] as a
> reference, so there is no point trying to apply changes in the ldb
> object as the attributes have already the value contained in the
> different messageElement of the mod_msg object.
>
> So I think parameters should be swapped it makes just sens from the
> point of view of the comparison and the modification of the ldb but for
> the overall logic I'm much more reserved.

I've looked over the code, and while I agree the samdb_replace seems
wrong, but before we added it, it just didn't work. The goal of this
routine is to build new @ATTRIBUTE records and @INDEX records, and to
then, if they differ from the DB, replace them in a DB.

The purpose of res->msgs[0] is to give the current DB as a base to the
diff function.

Andrew Bartlett

--
Andrew Bartlett http://samba.org/~abartlet/
Authentication Developer, Samba Team http://samba.org
Samba Developer, Cisco Inc.

signature.asc (196 bytes) Download Attachment

Patches for upgradeprovision

2009-11-29T08:57:34Z

Hello,

Please find attach patches for upgradeprovision, it would be nice to
have them in alpha9.

Patch 1: upgradeprovision will now look at the current domain level in
the to be upgraded provision so that the reference provision is also
created with the same domain level.
Patch 2: make upgradeprovision upgrade defaultSecurityDescriptor
Patch 3: handle upgrade of provision where the name attribute was not
present initialy (ie. my provision !)
Patch 4: it turns out that targetdir can cause problem so remove this
option as it can be replaced by -s path_to_smb.conf

They are also available at
http://repo.or.cz/w/Samba/ekacnet.git branch upgradeschema

let me know !
Matthieu

[0003-s4-Handle-the-case-in-secrets.ldb-without-name-attri.patch]

From add4ea60eb7e71dad9f41822c9ea56e924cdc273 Mon Sep 17 00:00:00 2001 From: Matthieu Patou <mat@...> Date: Sun, 29 Nov 2009 16:26:31 +0300 Subject: [PATCH 3/4] s4: Handle the case in secrets.ldb without name attribute --- source4/scripting/bin/upgradeprovision | 30 +++++++++++++++++++++++++----- 1 files changed, 25 insertions(+), 5 deletions(-) diff --git a/source4/scripting/bin/upgradeprovision b/source4/scripting/bin/upgradeprovision index 38bc092..cb9e1d2 100755 --- a/source4/scripting/bin/upgradeprovision +++ b/source4/scripting/bin/upgradeprovision @@ -73,7 +73,7 @@ hashAttrNotCopied = { "dn": 1,"whenCreated": 1,"whenChanged": 1,"objectGUID": 1 "showInAdvancedViewOnly": 1,"instanceType": 1, "cn": 1, "msDS-Behavior-Version":1, "nextRid":1,\ "nTMixedDomain": 1,"versionNumber":1, "lmPwdHistory":1, "pwdLastSet": 1, "ntPwdHistory":1, "unicodePwd":1,\ "dBCSPwd":1,"supplementalCredentials":1,"gPCUserExtensionNames":1, "gPCMachineExtensionNames":1,\ - "maxPwdAge":1, "mail":1, "secret":1,"possibleInferiors":1} + "maxPwdAge":1, "mail":1, "secret":1,"possibleInferiors":1, "sAMAccountType":1} # Usually for an object that already exists we do not overwrite attributes as they might have been changed for good # reasons. Anyway for a few of thems it's mandatory to replace them otherwise the provision will be broken somehow. @@ -458,12 +458,31 @@ def update_secrets(newpaths,paths,creds,session): delta.remove(att) for att in delta: i = i + 1 + + if att == "name": + message(CHANGE,"Found attribute name on %s, must rename the DN "%(res2[0].dn)) + secrets_ldb.rename(res2[0].dn,ldb.Dn(secrets_ldb,"%sfoo"%str(res2[0].dn))) + secrets_ldb.rename(ldb.Dn(secrets_ldb,"%sfoo"%str(res2[0].dn)),res2[0].dn) + else: + delta.remove(att) + + + for entry in listPresent: + res = newsecrets_ldb.search(expression="dn=%s"%entry,base="", scope=SCOPE_SUBTREE) + res2 = secrets_ldb.search(expression="dn=%s"%entry,base="", scope=SCOPE_SUBTREE) + delta = secrets_ldb.msg_diff(res2[0],res[0]) + i=0 + for att in hashAttrNotCopied.keys(): + delta.remove(att) + for att in delta: + i = i + 1 if att != "dn": message(CHANGE," Adding/Changing attribute %s to %s"%(att,res2[0].dn)) delta.dn = res2[0].dn secrets_ldb.modify(delta) + # Check difference between the current provision and the reference provision. # It looks for all object which base DN is name if ischema is false then scan is done in # cross partition mode. @@ -595,8 +614,8 @@ def check_updated_sd(newpaths,paths,creds,session,names): # Simple update method for updating the SD that rely on the fact that nobody should have modified the SD # This assumption is safe right now (alpha9) but should be removed asap -def update_sd(newpaths,paths,creds,session,names): - sam_ldb = Ldb(paths.samdb, session_info=session, credentials=creds,lp=lp) +def update_sd(paths,creds,session,names): + sam_ldb = Ldb(paths.samdb, session_info=session, credentials=creds,lp=lp,options=["modules:samba_dsdb"]) sam_ldb.transaction_start() # First update the SD for the rootdn sam_ldb.set_session_info(session) @@ -755,8 +774,9 @@ if opts.full: # SD should be created with admin but as some previous acl were so wrong that admin can't modify them we have first # to recreate them with the good form but with system account and then give the ownership to admin ... admin_session_info = admin_session(lp, str(names.domainsid)) -update_sd(newpaths,paths,creds,session,names) -update_sd(newpaths,paths,creds,admin_session_info,names) +message(SIMPLE,"Updating SD") +update_sd(paths,creds,session,names) +update_sd(paths,creds,admin_session_info,names) check_updated_sd(newpaths,paths,creds,session,names) message(SIMPLE,"Upgrade finished !") # remove reference provision now that everything is done ! -- 1.6.3.3

[0004-s4-Remove-targetdir-as-it-can-cause-some-trouble-and.patch]

From 87390442208c9f9fa110773f83700ca0b63f0d73 Mon Sep 17 00:00:00 2001 From: Matthieu Patou <mat@...> Date: Sun, 29 Nov 2009 19:44:08 +0300 Subject: [PATCH 4/4] s4: Remove targetdir as it can cause some trouble and can be replaced by an adapted -s smb.conf --- source4/scripting/bin/upgradeprovision | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diff --git a/source4/scripting/bin/upgradeprovision b/source4/scripting/bin/upgradeprovision index cb9e1d2..4b0cfec 100755 --- a/source4/scripting/bin/upgradeprovision +++ b/source4/scripting/bin/upgradeprovision @@ -112,8 +112,6 @@ parser.add_option("--debugchange", help="Print information on what is different parser.add_option("--debugchangesd", help="Print information security descriptors differences", action="store_true") parser.add_option("--debugall", help="Print all available information (very verbose)", action="store_true") parser.add_option("--full", help="Perform full upgrade of the samdb (schema, configuration, new objects, ...", action="store_true") -parser.add_option("--targetdir", type="string", metavar="DIR", - help="Set target directory") opts = parser.parse_args()[0] @@ -749,7 +747,7 @@ def update_machine_account_password(paths,creds,session,names): # From here start the big steps of the program # First get files paths -paths=get_paths(targetdir=opts.targetdir,smbconf=smbconf) +paths=get_paths(smbconf=smbconf) paths.setup = setup_dir def setup_path(file): return os.path.join(setup_dir, file) -- 1.6.3.3

[0001-s4-load-the-domain-level-of-the-current-provision-an.patch]

From 0c1a0b230f34ddc17ede455036e58aa7f60afd58 Mon Sep 17 00:00:00 2001 From: Matthieu Patou <mat@...> Date: Fri, 27 Nov 2009 20:50:04 +0300 Subject: [PATCH 1/4] s4: load the domain level of the current provision and create a provision with the same domain level --- source4/scripting/bin/upgradeprovision | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-) diff --git a/source4/scripting/bin/upgradeprovision b/source4/scripting/bin/upgradeprovision index 2f0ce84..486bad1 100755 --- a/source4/scripting/bin/upgradeprovision +++ b/source4/scripting/bin/upgradeprovision @@ -39,7 +39,7 @@ from base64 import b64encode import samba from samba.credentials import DONT_USE_KERBEROS from samba.auth import system_session, admin_session -from samba import Ldb +from samba import Ldb, DS_DOMAIN_FUNCTION_2003, DS_DOMAIN_FUNCTION_2008, DS_DC_FUNCTION_2008_R2 from ldb import SCOPE_SUBTREE, SCOPE_ONELEVEL, SCOPE_BASE, LdbError import ldb import samba.getopt as options @@ -222,10 +222,14 @@ def guess_names_from_current_provision(credentials,session_info,paths): names.ntdsguid = str(ndr_unpack( misc.GUID,res5[0]["objectGUID"][0])) # domain guid/sid - attrs6 = ["objectGUID", "objectSid", ] + attrs6 = ["objectGUID", "objectSid","msDS-Behavior-Version" ] res6 = samdb.search(expression="(objectClass=*)",base=basedn, scope=SCOPE_BASE, attrs=attrs6) names.domainguid = str(ndr_unpack( misc.GUID,res6[0]["objectGUID"][0])) names.domainsid = ndr_unpack( security.dom_sid,res6[0]["objectSid"][0]) + if int(res6[0]["msDS-Behavior-Version"][0]) < DS_DOMAIN_FUNCTION_2003: + names.domainlevel = DS_DOMAIN_FUNCTION_2003 + else: + names.domainlevel = int(res6[0]["msDS-Behavior-Version"][0]) # policy guid attrs7 = ["cn","displayName"] @@ -262,6 +266,7 @@ def print_names(names): message(GUESS, "domainsid :"+str(names.domainsid)) message(GUESS, "domainguid :"+names.domainguid) message(GUESS, "ntdsguid :"+names.ntdsguid) + message(GUESS, "domainlevel :"+str(names.domainlevel)) # Create a fresh new reference provision # This provision will be the reference for knowing what has changed in the @@ -297,6 +302,7 @@ def newprovision(names,setup_dir,creds,session,smbconf): slapd_path=None, setup_ds_path=None, nosync=None, + dom_for_fun_level=names.domainlevel, ldap_dryrun_mode=None) return provdir -- 1.6.3.3

[0002-s4-don-t-forget-to-update-defaultSecurityDescriptor.patch]

From 10aeddec0c80900d1741a5462d91326447a858d3 Mon Sep 17 00:00:00 2001 From: Matthieu Patou <mat@...> Date: Sun, 29 Nov 2009 02:28:26 +0300 Subject: [PATCH 2/4] s4: don't forget to update defaultSecurityDescriptor --- source4/scripting/bin/upgradeprovision | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/source4/scripting/bin/upgradeprovision b/source4/scripting/bin/upgradeprovision index 486bad1..38bc092 100755 --- a/source4/scripting/bin/upgradeprovision +++ b/source4/scripting/bin/upgradeprovision @@ -79,7 +79,8 @@ hashAttrNotCopied = { "dn": 1,"whenCreated": 1,"whenChanged": 1,"objectGUID": 1 # reasons. Anyway for a few of thems it's mandatory to replace them otherwise the provision will be broken somehow. hashOverwrittenAtt = { "prefixMap": replace, "systemMayContain": replace,"systemOnly":replace, "searchFlags":replace,\ "mayContain":replace, "systemFlags":replace, - "oEMInformation":replace, "operatingSystemVersion":replace, "adminPropertyPages":1,"possibleInferiors":replace+delete} + "oEMInformation":replace, "operatingSystemVersion":replace, "adminPropertyPages":replace, + "defaultSecurityDescriptor": replace} backlinked = [] def define_what_to_log(opts): @@ -342,11 +343,12 @@ def dn_sort(x,y): # it also populate hash structure for later use in the upgrade process def handle_security_desc(ischema,att,msgElt,hashallSD,old,new): if ischema == 1 and att == "defaultSecurityDescriptor" and msgElt.flags() == ldb.FLAG_MOD_REPLACE: + print "coin coin" hashSD = {} hashSD["oldSD"] = old[0][att] hashSD["newSD"] = new[0][att] hashallSD[str(old[0].dn)] = hashSD - return 1 + return 0 if att == "nTSecurityDescriptor" and msgElt.flags() == ldb.FLAG_MOD_REPLACE: if ischema == 0: hashSD = {} -- 1.6.3.3

important: Wrong logic (to my mind) in schema_set.c

2009-11-29T06:02:32Z

Dear all, I've been trying to upgrade a very old provision (alpha3) I've
been hit by errors like this:

_ldb.LdbError: (80, 'schema_load_init: dsdb_set_schema() failed: 32:No
such object')
After a careful investigation I am 99% sure that those two changes
(introduced in changeset 6b05a907) are wrong

@@ -135,7 +135,7 @@ static int dsdb_schema_set_attributes(struct
ldb_context *ldb, struct dsdb_schem

mod_msg = ldb_msg_diff(ldb, res->msgs[0], msg);
if (mod_msg->num_elements > 0) {
- ret = ldb_modify(ldb, mod_msg);
+ ret = samdb_replace(ldb, mem_ctx, mod_msg);
}
}

@@ -163,7 +163,7 @@ static int dsdb_schema_set_attributes(struct
ldb_context *ldb, struct dsdb_schem

mod_msg = ldb_msg_diff(ldb, res_idx->msgs[0], msg_idx);
if (mod_msg->num_elements > 0) {
- ret = ldb_modify(ldb, mod_msg);
+ ret = samdb_replace(ldb, mem_ctx, mod_msg);
}

Because the different elements with different flags (add,replace,delete)
and using samdb_replace will override this flag by using just replace.
Obviously if the ldb_diff was telling you that you should add the
attribute 'foo' trying to make ldb modify the 'foo' attribute will not
be very good.

Also I'm quite puzzled by the use ldb_msg_diff and the order of
parameters. Because res->msgs[0] is an message element obtained from a
search in the LDB database represented by the ldb object.
So to my mind by doing
mod_msg = ldb_msg_diff(ldb, res->msgs[0], msg);
we are comparing res->msgs[0] and msg and using res->msgs[0] as a
reference, so there is no point trying to apply changes in the ldb
object as the attributes have already the value contained in the
different messageElement of the mod_msg object.

So I think parameters should be swapped it makes just sens from the
point of view of the comparison and the modification of the ldb but for
the overall logic I'm much more reserved.

Can someone have a closer look on this ?

Matthieu.

Re: NetShareEnum: disable sanity check for better compatibility with Windows

2009-11-29T05:46:12Z

On Sun, Nov 29, 2009 at 02:15:18PM +0100, Giovanni Bajo wrote:

> the attached patches increase Samba compatibility with Windows by
> disabling a sanity check which prevents Samba from issuing an answer to
> a NetShareEnum request in case of a corrupted (invalid) packet.
>
> Long version. I have recently bought a Samsung BD-P3600 bluray player.
> This unit has wired/wifi connection and a primitive/buggy support for
> streaming files off Windows shares. It works well if the shares are on a
> Windows PC, but it fails to detect shares on Linux and Mac OSX. I
> investigated the problem and it turns out that the bluray player sends
> an invalid packet:
>
> 0000 00 1b fc 91 80 08 00 12 fb 2a d9 7c 08 00 45 00 ........ .*.|..E.
> 0010 00 9e b8 2e 40 00 40 06 fe c2 c0 a8 01 17 c0 a8 ....@.@. ........
> 0020 01 01 cc 54 00 8b 46 cc 34 d6 de 00 9f ae 80 18 ...T..F. 4.......
> 0030 0b 68 01 e3 00 00 01 01 08 0a 00 00 99 c7 00 64 .h...... .......d
> 0040 41 2d 00 00 00 66 ff 53 4d 42 25 00 00 00 00 08 A-...f.S MB%.....
> 0050 01 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 ........ ........
> 0060 3b 00 00 00 9f 00 0e 1a 00 00 00 63 00 ff ff 00 ;....... ...c....
> 0070 00 00 00 00 00 00 00 00 00 1a 00 4c 00 00 00 66 ........ ...L...f
> 0080 00 00 00 27 00 5c 50 49 50 45 5c 4c 41 4e 4d 41 ...'.\PI PE\LANMA
> 0090 4e 00 00 00 57 72 4c 65 68 00 42 31 33 42 57 7a N...WrLe h.B13BWz
> 00a0 57 57 57 7a 42 39 42 00 01 00 ff ff WWWzB9B. ....
>
> Microsoft Windows Lanman Remote API Protocol
> Function Code: NetShareEnum (0)
> Parameter Descriptor: WrLeh
> Return Descriptor: B13BWzWWWzB9B
> Detail Level: 1
> Receive Buffer Length: 65535
>
> This packet is malformed because, for level 1 query, the correct return
> descriptor would be "B13BWz". I've attached the full wireshark dump, in
> case it matters.
>
> What happens is that Samba refuses to answer to this query because it
> fails an internal sanity check; instead, Windows happily answers and the
> conversation goes on.
>
> I verified my fix against a samba2 server (this is what I'm running on
> my DDWRT-based router), and forward-ported the same patch on samba3 and
> samba4 trees by visual inspection.
>
> This is my first contribution to Samba, so I'm looking for some guidance
> for this patch to be accepted.

It seems that the descriptor they send is actually a level2
descriptor. Can you get us a trace against a Windows box? I
would be interested to see what they return in this case. Do
they decide based upon the level or the descriptor?

And, would it be possible not to completely drop the check
but to also allow the level2 descriptor for level1?

Volker

signature.asc (204 bytes) Download Attachment

Wrong logic (to my mind) in schema_set.c

2009-11-29T05:44:52Z

Dear all, I've been trying to upgrade a very old provision (alpha3) I've
been hit by errors like this:
index 8b1188a..5ded04e 100644
--- a/source4/dsdb/schema/schema_set.c
+++ b/source4/dsdb/schema/schema_set.c
@@ -135,7 +135,7 @@ static int dsdb_schema_set_attributes(struct
ldb_context *ldb, struct dsdb_schem

mod_msg = ldb_msg_diff(ldb, res->msgs[0], msg);
if (mod_msg->num_elements > 0) {
- ret = ldb_modify(ldb, mod_msg);
+ ret = samdb_replace(ldb, mem_ctx, mod_msg);
}
}

@@ -154,7 +154,7 @@ static int dsdb_schema_set_attributes(struct
ldb_context *ldb, struct dsdb_schem
if (ret == LDB_ERR_NO_SUCH_OBJECT) {
ret = ldb_add(ldb, msg_idx);
} else if (ret != LDB_SUCCESS) {
- } else if (res->count != 1) {
+ } else if (res_idx->count != 1) {
ret = ldb_add(ldb, msg_idx);
} else {
ret = LDB_SUCCESS;
@@ -163,7 +163,7 @@ static int dsdb_schema_set_attributes(struct
ldb_context *ldb, struct dsdb_schem

mod_msg = ldb_msg_diff(ldb, res_idx->msgs[0], msg_idx);
if (mod_msg->num_elements > 0) {
- ret = ldb_modify(ldb, mod_msg);
+ ret = samdb_replace(ldb, mem_ctx, mod_msg);
}

NetShareEnum: disable sanity check for better compatibility with Windows

2009-11-29T05:15:18Z

Hello,

the attached patches increase Samba compatibility with Windows by
disabling a sanity check which prevents Samba from issuing an answer to
a NetShareEnum request in case of a corrupted (invalid) packet.

Long version. I have recently bought a Samsung BD-P3600 bluray player.
This unit has wired/wifi connection and a primitive/buggy support for
streaming files off Windows shares. It works well if the shares are on a
Windows PC, but it fails to detect shares on Linux and Mac OSX. I
investigated the problem and it turns out that the bluray player sends
an invalid packet:

0000 00 1b fc 91 80 08 00 12 fb 2a d9 7c 08 00 45 00 ........ .*.|..E.
0010 00 9e b8 2e 40 00 40 06 fe c2 c0 a8 01 17 c0 a8 ....@.@. ........
0020 01 01 cc 54 00 8b 46 cc 34 d6 de 00 9f ae 80 18 ...T..F. 4.......
0030 0b 68 01 e3 00 00 01 01 08 0a 00 00 99 c7 00 64 .h...... .......d
0040 41 2d 00 00 00 66 ff 53 4d 42 25 00 00 00 00 08 A-...f.S MB%.....
0050 01 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 ........ ........
0060 3b 00 00 00 9f 00 0e 1a 00 00 00 63 00 ff ff 00 ;....... ...c....
0070 00 00 00 00 00 00 00 00 00 1a 00 4c 00 00 00 66 ........ ...L...f
0080 00 00 00 27 00 5c 50 49 50 45 5c 4c 41 4e 4d 41 ...'.\PI PE\LANMA
0090 4e 00 00 00 57 72 4c 65 68 00 42 31 33 42 57 7a N...WrLe h.B13BWz
00a0 57 57 57 7a 42 39 42 00 01 00 ff ff WWWzB9B. ....

Microsoft Windows Lanman Remote API Protocol
Function Code: NetShareEnum (0)
Parameter Descriptor: WrLeh
Return Descriptor: B13BWzWWWzB9B
Detail Level: 1
Receive Buffer Length: 65535

This packet is malformed because, for level 1 query, the correct return
descriptor would be "B13BWz". I've attached the full wireshark dump, in
case it matters.

What happens is that Samba refuses to answer to this query because it
fails an internal sanity check; instead, Windows happily answers and the
conversation goes on.

I verified my fix against a samba2 server (this is what I'm running on
my DDWRT-based router), and forward-ported the same patch on samba3 and
samba4 trees by visual inspection.

This is my first contribution to Samba, so I'm looking for some guidance
for this patch to be accepted.

Thanks!
--
Giovanni Bajo
Develer S.r.l.
http://www.develer.com

[netshareenum.patch]

diff --git a/source3/smbd/lanman.c b/source3/smbd/lanman.c index 6fc72c2..b3b3ec7 100644 --- a/source3/smbd/lanman.c +++ b/source3/smbd/lanman.c @@ -1799,9 +1799,6 @@ static bool api_RNetShareEnum( connection_struct *conn, uint16 vuid, if (!prefix_ok(str1,"WrLeh")) { return False; } - if (!check_share_info(uLevel,str2)) { - return False; - } /* Ensure all the usershares are loaded. */ become_root(); diff --git a/source4/ntvfs/ipc/ipc_rap.c b/source4/ntvfs/ipc/ipc_rap.c index 04ea700..66b93e8 100644 --- a/source4/ntvfs/ipc/ipc_rap.c +++ b/source4/ntvfs/ipc/ipc_rap.c @@ -260,20 +260,6 @@ static NTSTATUS _rap_netshareenum(struct rap_call *call) RAP_GOTO(rap_srv_pull_bufsize(call, &r.in.bufsize)); RAP_GOTO(rap_srv_pull_expect_multiple(call)); - switch(r.in.level) { - case 0: - if (strcmp(call->datadesc, "B13") != 0) - return NT_STATUS_INVALID_PARAMETER; - break; - case 1: - if (strcmp(call->datadesc, "B13BWz") != 0) - return NT_STATUS_INVALID_PARAMETER; - break; - default: - return NT_STATUS_INVALID_PARAMETER; - break; - } - result = rap_netshareenum(call, call->event_ctx, call->lp_ctx, &r); if (!NT_STATUS_IS_OK(result))

[netshareenum.samba2.patch]

Index: source/smbd/lanman.c =================================================================== --- source/smbd/lanman.c (revisione 26701) +++ source/smbd/lanman.c (copia locale) @@ -1625,7 +1625,6 @@ int f_len = 0, s_len = 0; if (!prefix_ok(str1,"WrLeh")) return False; - if (!check_share_info(uLevel,str2)) return False; data_len = fixed_len = string_len = 0; for (i=0;i<count;i++)

dump2.pcap.bz2 (6K) Download Attachment

Re: Updatings SD

2009-11-29T03:59:35Z

Hi Mattieu,
Good work! Time was not lost, I found some possible instability in my code while playing with the script. Also I still thing its possible to accomplish what you are doing without the need of a second control, just using sd_flags control. However, I suppose we can use your control for the time being, and decide later whether to keep it or use sd_flags. I will talk to Andrew when I see him on IRC to include your script in the alpha.

Regards,
Nadya
----- Original Message -----
> From: samba-technical-bounces@... <samba-technical-bounces@...>
> To: samba-technical@... <samba-technical@...>, Matthieu Patou <mat+Informatique.Samba@...>
> Sent: Sunday, November 29, 2009 1:11:43 AM GMT+0200 Europe;Athens
> Subject: Re: Updatings SD

> > Hello Nadya,
>
> I put more trace and I find the problem: defaultSecurityDescriptor was
>
> not upgraded because previously I had a plan to make something rather
> complicated with SD update that implied to have the previous SD.
>
> So the story is solved no need to investigate more !
> Sorry for the lost time on research ...
>
> Matthieu.
> On 26/11/2009 20:06, Matthieu Patou wrote:
> > Hello Nadya,
> >
> > I'll need your help to help me to understand why I'm failling with
> the
> > upgradesprovision to correctly update some SD:
> >
> > cn=administrator,cn=users,dc=smb4,dc=tst
> > cn=dns,cn=users,dc=smb4,dc=tst
> > cn=guest,cn=users,dc=smb4,dc=tst
> > cn=krbtgt,cn=users,dc=smb4,dc=tst
> >
> > As all the other SD are OK I'm rather encline to think that the
> > update_sd does its job pretty well ... If you had a look at the
> > updateprovision.log you will see the DN then the updated SD and
> finally
> > the reference SDDL.
> >
> > The SDDL share a lot but diverge on some points can you try to light
> my
> > way ?
> >
> >
> > Of course I also have problems with this but it's related to policy
> > object and we know that we still have a slight problem on one flag
> for
> > the SACL (so I'm not very nervous with this)
> >
> > cn={f6f519f1-7ec2-4f98-9b87-3d05e2dc697b},cn=policies,cn=system,
> dc=smb4,dc=tst
> >
> > cn=machine,cn{f6f519f1-7ec2-4f98-9b87-3d05e2dc697b},cn=policies,
> cn=system,dc=smb4,dc=tst
> >
> > cn=user,cn{f6f519f1-7ec2-4f98-9b87-3d05e2dc697b},cn=policies,
> cn=system,dc=smb4,dc=tst
> >
> >
> >
> > Of course let me know !
> >
> > Matthieu

Re: update_machine_account_password

2009-11-29T01:25:31Z

On Sun, 2009-11-29 at 12:07 +0300, Matthieu Patou wrote:

> On 29/11/2009 10:02, Andrew Bartlett wrote:
> > On Fri, 2009-11-27 at 20:23 +0300, Matthieu Patou wrote:
> >
> >> Andrew B,
> >>
> >> For a reason that I can't explain update_machine_account_password do not
> >> work properly to update the supplementalCredentials and that's why using
> >> smbclient for s4 against the updated tridge provision is failing
> >> (because it's provision use w2K8 domain level and everything is done so
> >> that aes is activated).
> >>
> >> I tried several trick and failed so a guru of this stuff is required.
> >> Note that setting the password with sbin/setpassword but things back in
> >> order (well once the kvno has been modified to put secrets.ldb and
> >> sam.ldb in sync)
> >>
> >> Could you have a look ?
> >>
> > The easy answer it to simply use the same code as setpassword, whatever
> > that is. (That way, we keep this script using well known and otherwise
> > tested code).
> >
> >
> I tried also with the code of setpassword without success as I wrote in
> my other email on this thread what maid the password change succeed for
> the AES stuff is the fact that I used -s path_to_smb.conf when the
> database is not located in the default path.

Ahh, that would be a problem. We would be using the wrong realm and
domain. We should store the domain in the @SAMBA_DSDB record, and build
the realm from the default basedn. We should also reconsider whenever
we use lp_ctx in ldb.

But using the right smb.conf is also required, and is the short and
long-term fix.

Andrew Bartlett

--
Andrew Bartlett http://samba.org/~abartlet/
Authentication Developer, Samba Team http://samba.org
Samba Developer, Cisco Inc.

signature.asc (196 bytes) Download Attachment

Re: update_machine_account_password

2009-11-29T01:07:44Z

On 29/11/2009 10:02, Andrew Bartlett wrote:

> On Fri, 2009-11-27 at 20:23 +0300, Matthieu Patou wrote:
>
>> Andrew B,
>>
>> For a reason that I can't explain update_machine_account_password do not
>> work properly to update the supplementalCredentials and that's why using
>> smbclient for s4 against the updated tridge provision is failing
>> (because it's provision use w2K8 domain level and everything is done so
>> that aes is activated).
>>
>> I tried several trick and failed so a guru of this stuff is required.
>> Note that setting the password with sbin/setpassword but things back in
>> order (well once the kvno has been modified to put secrets.ldb and
>> sam.ldb in sync)
>>
>> Could you have a look ?
>>
> The easy answer it to simply use the same code as setpassword, whatever
> that is. (That way, we keep this script using well known and otherwise
> tested code).
>
>

I tried also with the code of setpassword without success as I wrote in
my other email on this thread what maid the password change succeed for
the AES stuff is the fact that I used -s path_to_smb.conf when the
database is not located in the default path.

Matthieu.

Re: update_machine_account_password

2009-11-28T23:02:55Z

On Fri, 2009-11-27 at 20:23 +0300, Matthieu Patou wrote:

> Andrew B,
>
> For a reason that I can't explain update_machine_account_password do not
> work properly to update the supplementalCredentials and that's why using
> smbclient for s4 against the updated tridge provision is failing
> (because it's provision use w2K8 domain level and everything is done so
> that aes is activated).
>
> I tried several trick and failed so a guru of this stuff is required.
> Note that setting the password with sbin/setpassword but things back in
> order (well once the kvno has been modified to put secrets.ldb and
> sam.ldb in sync)
>
> Could you have a look ?

The easy answer it to simply use the same code as setpassword, whatever
that is. (That way, we keep this script using well known and otherwise
tested code).

Andrew Bartlett

--
Andrew Bartlett http://samba.org/~abartlet/
Authentication Developer, Samba Team http://samba.org
Samba Developer, Cisco Inc.

signature.asc (196 bytes) Download Attachment

Re: Updatings SD

2009-11-28T15:15:47Z

Hello Nadya,

I put more trace and I find the problem: defaultSecurityDescriptor was
not upgraded because previously I had a plan to make something rather
complicated with SD update that implied to have the previous SD.

So the story is solved no need to investigate more !
Sorry for the lost time on research ...

Matthieu.
On 26/11/2009 20:06, Matthieu Patou wrote:

> Hello Nadya,
>
> I'll need your help to help me to understand why I'm failling with the
> upgradesprovision to correctly update some SD:
>
> cn=administrator,cn=users,dc=smb4,dc=tst
> cn=dns,cn=users,dc=smb4,dc=tst
> cn=guest,cn=users,dc=smb4,dc=tst
> cn=krbtgt,cn=users,dc=smb4,dc=tst
>
> As all the other SD are OK I'm rather encline to think that the
> update_sd does its job pretty well ... If you had a look at the
> updateprovision.log you will see the DN then the updated SD and finally
> the reference SDDL.
>
> The SDDL share a lot but diverge on some points can you try to light my
> way ?
>
>
> Of course I also have problems with this but it's related to policy
> object and we know that we still have a slight problem on one flag for
> the SACL (so I'm not very nervous with this)
>
> cn={f6f519f1-7ec2-4f98-9b87-3d05e2dc697b},cn=policies,cn=system,dc=smb4,dc=tst
>
> cn=machine,cn{f6f519f1-7ec2-4f98-9b87-3d05e2dc697b},cn=policies,cn=system,dc=smb4,dc=tst
>
> cn=user,cn{f6f519f1-7ec2-4f98-9b87-3d05e2dc697b},cn=policies,cn=system,dc=smb4,dc=tst
>
>
>
> Of course let me know !
>
> Matthieu

s4-drs: cope with bogus empty attributes from w2k8-r2 (Re: [SCM] Samba Shared Repository - branch master updated)

2009-11-28T15:02:31Z

Hi Metze,

> > + /* delete any empty elements */
> > + for (i=0; i < msg->num_elements; i++) {
> > + if (msg->elements[i].name == NULL) {
> > + ldb_msg_remove_element(msg, &msg->elements[i]);
> > + i--;
> > + }
> > + }
> > +
>
> We need to be careful about this, an empty element means
> we should remove existing values.
> If remove the empty ones here, we need to let the repl_meta_data module
> remove them based on the received meta_data array.

my comment in the above is perhaps misleading. I'm removing "empty"
ones that have a NULL name. It only gets a NULL name when the attid is
invalid (not in the prefixmap) and it also has no elements.

So we are not removing all the empty entries, so we don't need any
change in repl_meta_data.c at the moment. Before the above change we
were just exiting the DRS replication with an error.

This change was just to cope with what appears to be a bug in w2k8-r2,
where it sometimes sends completely bogus empty entries with bad attid
values. Kamen is looking into this with Bill Wesse, but I wanted a way
to avoid this problem now as otherwise I can't do any DRS testing. I
haven't found any way to remove these broken objects/attributes from
the w2k8 server, as they are already deleted, and they don't seem to
be being removed as part of the normal tombstoning. Re-installing
w2k8-r2 each time this happens is too painful for regular development.

I actually suspect the bug in w2k8-r2 is triggered by our ldap.py
testsuite somehow, as the bad objects are ones we create in that
testsuite, with one of the test objectclasses we create.

Cheers, Tridge

Re: update_machine_account_password

2009-11-28T13:44:02Z

Andrew,
Don't know it comes to my mind by I decided to make a test with -s
parameter instead of --targetdir
that is to say:

./scripting/bin/upgradeprovision -s /home/mat/tridge/etc/smb.conf
instead of
./scripting/bin/upgradeprovision --targetdir /home/mat/tridge

And now it's working. I am wondering what the -s trigger or set so that
just using --target dir do not the same effet.

Worth investigating, I guess I should remove the --targetdir option.

Matthieu.
On 27/11/2009 20:23, Matthieu Patou wrote:

Re: Patches for allowing upgradeprovision to upgrade SD

2009-11-28T13:15:54Z

On 28/11/2009 02:31, tridge@... wrote:

> Hi Matthieu,
>
> Thanks for that! I've confirmed that it fixes the upgrade for my
> box. I'm now doing more tests and expect to push the patch soon.
>
> I'm very impressed at how this upgradeprovision script has worked
> out. Thanks for all your work on it!
>
> Cheers, Tridge
>

Tridge,
I've one more patch in my repo that I will (soooooon) push because
right now we do a provision with a w2k3 domain and for instance your
provision is 2008 so the script must detect this and make a reference
provision with the same domain level as the updated provision.

Stay tuned !

Matthieu.

Re: the function of SMB_STRUCT_STAT *sbuf in vfs readdir() ?

2009-11-28T10:14:16Z

On Sat, Nov 28, 2009 at 02:51:37PM +0100, Olivier Sessink wrote:
> what is the function of SMB_STRUCT_STAT *sbuf in the VFS readdir API ?
>
> is it supposed to be filled by the SMB_VFS_NEXT_READDIR call? (e.g. can
> you use it after that call?)

Some operating systems provide something similar to the NFS
readdirplus operation, also giving stat info in a single
syscall. The READDIR sbuf argument is giving VFS
implementations a chance to fill that in. If you don't do
that in a transparent module, just pass it down.

You can't rely on it being set though, the check macros is
VALID_STAT().

Volker

signature.asc (204 bytes) Download Attachment

Re: [SCM] Samba Shared Repository - branch master updated

2009-11-28T08:30:37Z

Hi Matthias,

On Sat, Nov 28, 2009 at 03:57:21PM +0000, Matthias Dieter Wallnöfer wrote:
> sorry for the remove of the first code block "if not opts.interactive and (opts.realm is None or opts.domain is None):". I was too fast and thought maybe this was the problem since the interactive mode didn't work on my box. Later I discovered that this is due to the split function of the call socket.getfqdn().split(".", 1)[1].upper()). Therefore I would let in the error handling like this since it's really a bugfix.

> Okay, the first part I will restore
Please see my comments about the other code parts as well - I don't
think they are correct either. Is there any particular reason for
prompting the user twice when we are not able to determine a sane
default?

Cheers,

Jelmer

> --- Jelmer Vernooij <jelmer@...> schrieb am Sa, 28.11.2009:

> Von: Jelmer Vernooij <jelmer@...>
> Betreff: Re: [SCM] Samba Shared Repository - branch master updated
> An: samba-technical@...
> CC: samba-cvs@...
> Datum: Samstag, 28. November 2009, 15:10

> On Fri, 2009-11-27 at 06:04 -0600, Matthias Dieter Wallnöfer wrote:
> > The branch, master has been updated
> > via a7fa3a9... s4:provision.py - cosmetic output correction
> > via 6b835b0... s4:setup/provision - make the interactive mode work again
> >    from 7504b03... s4:WHATSNEW4.txt - Add also here a comment about the "upgrade_from_s3" script

> > http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master

> > - Log -----------------------------------------------------------------
> > commit a7fa3a9703bf9f0c72031c84998f4fb21ba95429
> > Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> > Date:   Fri Nov 27 13:07:52 2009 +0100

> >    s4:provision.py - cosmetic output correction

> > commit 6b835b0691faab904246e587ffa6ff74b9fb53e2
> > Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> > Date:   Fri Nov 27 13:02:31 2009 +0100

> >    s4:setup/provision - make the interactive mode work again

> > -----------------------------------------------------------------------

> > Summary of changes:
> > source4/scripting/python/samba/provision.py | 2 +-
> > source4/setup/provision    |   16 +++-------------
> > 2 files changed, 4 insertions(+), 14 deletions(-)

> > index ed350dd..f1aa07c 100755
> > --- a/source4/setup/provision
> > +++ b/source4/setup/provision
> > @@ -122,14 +122,6 @@ def message(text):
> > if len(sys.argv) == 1:
> >     opts.interactive = True
> >
> > -if not opts.interactive and (opts.realm is None or opts.domain is None):
> > -    if opts.realm is None:
> > -        print >>sys.stderr, "No realm set"
> > -    if opts.domain is None:
> > -        print >>sys.stderr, "No domain set"
> > -    parser.print_usage()
> > -    sys.exit(1)
> > -
> ^^^ This change isn't necessary to make the interactive mode work (it
> explicitly checks for interactive mode not being in use), and it breaks
> usage when not in interactive mode because we end up with no
> realm/domain being set.

> > if opts.interactive:
> >     from getpass import getpass
> >     import socket
> > @@ -137,19 +129,17 @@ if opts.interactive:
> >         if default is not None:
> >             print "%s [%s]: " % (prompt,default),
> >         else:
> > -            print "%s: " % (prompt,),
> > +            print "%s: " % (prompt),
> >         return sys.stdin.readline().rstrip("\n") or default
> >     try:
> >         opts.realm = ask("Realm", socket.getfqdn().split(".", 1)[1].upper())
> >     except IndexError:
> > -        print >>sys.stderr, "Cannot guess realm from %s" % ( socket.getfqdn())
> > -        sys.exit(1)
> > +        opts.realm = ask("Realm", None)
> ^^^ We've already asked the user for a realm, why ask twice? Rather, we
> should not set a default value if the users FQDN is broken because it
> contains no dots.

> >     try:
> >         opts.domain = ask("Domain", opts.realm.split(".")[0])
> >     except IndexError:
> > -        print >>sys.stderr, "Cannot guess domain from %s" % ( opts.realm())
> > -        sys.exit(1)
> > +        opts.domain = ask("Domain", None)
> ^^^ Same here, let's just get the default value right in the first
> place.

> Cheers,

> Jelmer

> __________________________________________________
> Do You Yahoo!?
> Sie sind Spam leid? Yahoo! Mail verfügt über einen herausragenden Schutz gegen Massenmails.
> http://mail.yahoo.com

--
Jelmer Vernooij <jelmer@...> - http://jelmer.vernstok.nl/
17:29:41 up 14 min, 3 users, load average: 0.53, 0.58, 0.44

Re: Commit: 0920e0b63b806c8ed4839271048dd4924ed02b2b

2009-11-28T08:00:15Z

On Sat, Nov 28, 2009 at 07:32, <tridge@...> wrote:
> it is far too dangerous. The environment variable LDB_URL is the
> standard environment variable used by the ldb tools to avoid using the
> -H option. It is not uncommon for someone to set this to point at a
> real LDB. In my case I had it pointed at my sam.ldb, which meant that
> running "make test" not only failed, it also corrupted by sam database.
>
This is good to know.
I haven't realized it is used that way - I thought it is temporary DB to be
used during testing - I don't know why I was thinking it is cleaned every
test suite is started :)

> > Could we just use another name for the environment variable used?
>
> no, please don't use an environment variable for this sort of thing.
>
> If you want to have a way to override the location of this test ldb,
> then use lp_parm_string() and then pass the location using the
> --option command line option. To see some examples of this, try:
>
> git grep lp_parm_string source4/torture/
>
> for example, you could have:
>
> ldb_url = lp_parm_string(tctx->lp_ctx, NULL, "drstest", "ldb_url");
>
> and then set it with:
>
> --option drstest:ldb_url=foo.ldb

This is much better solution. I will do it that way.

Thanks for spotting that out and... sorry for inconveniences you might
have had with this approach.

--
CU,
Kamen Mazdrashki
kamen.mazdrashki@...
http://repo.or.cz/w/Samba/kamenim.git
-------------------------------------
CISCO SYSTEMS BULGARIA EOOD
http://www.cisco.com/global/BG/

Re: [SCM] Samba Shared Repository - branch master updated

2009-11-28T07:57:21Z

Jelmer,

sorry for the remove of the first code block "if not opts.interactive and (opts.realm is None or opts.domain is None):". I was too fast and thought maybe this was the problem since the interactive mode didn't work on my box. Later I discovered that this is due to the split function of the call socket.getfqdn().split(".", 1)[1].upper()). Therefore I would let in the error handling like this since it's really a bugfix.

Okay, the first part I will restore

Matthias

--- Jelmer Vernooij <jelmer@...> schrieb am Sa, 28.11.2009:

Von: Jelmer Vernooij <jelmer@...>
Betreff: Re: [SCM] Samba Shared Repository - branch master updated
An: samba-technical@...
CC: samba-cvs@...
Datum: Samstag, 28. November 2009, 15:10

On Fri, 2009-11-27 at 06:04 -0600, Matthias Dieter Wallnöfer wrote:

> The branch, master has been updated
> via a7fa3a9... s4:provision.py - cosmetic output correction
> via 6b835b0... s4:setup/provision - make the interactive mode work again
>    from 7504b03... s4:WHATSNEW4.txt - Add also here a comment about the "upgrade_from_s3" script
>
> http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master
>
>
> - Log -----------------------------------------------------------------
> commit a7fa3a9703bf9f0c72031c84998f4fb21ba95429
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date:   Fri Nov 27 13:07:52 2009 +0100
>
>    s4:provision.py - cosmetic output correction
>
> commit 6b835b0691faab904246e587ffa6ff74b9fb53e2
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date:   Fri Nov 27 13:02:31 2009 +0100
>
>    s4:setup/provision - make the interactive mode work again
>
> -----------------------------------------------------------------------
>
> Summary of changes:
> source4/scripting/python/samba/provision.py | 2 +-
> source4/setup/provision    |   16 +++-------------
> 2 files changed, 4 insertions(+), 14 deletions(-)

> index ed350dd..f1aa07c 100755
> --- a/source4/setup/provision
> +++ b/source4/setup/provision
> @@ -122,14 +122,6 @@ def message(text):
> if len(sys.argv) == 1:
>     opts.interactive = True
>
> -if not opts.interactive and (opts.realm is None or opts.domain is None):
> -    if opts.realm is None:
> -        print >>sys.stderr, "No realm set"
> -    if opts.domain is None:
> -        print >>sys.stderr, "No domain set"
> -    parser.print_usage()
> -    sys.exit(1)
> -

^^^ This change isn't necessary to make the interactive mode work (it
explicitly checks for interactive mode not being in use), and it breaks
usage when not in interactive mode because we end up with no
realm/domain being set.

> if opts.interactive:
>     from getpass import getpass
>     import socket
> @@ -137,19 +129,17 @@ if opts.interactive:
>         if default is not None:
>             print "%s [%s]: " % (prompt,default),
>         else:
> -            print "%s: " % (prompt,),
> +            print "%s: " % (prompt),
>         return sys.stdin.readline().rstrip("\n") or default
>     try:
>         opts.realm = ask("Realm", socket.getfqdn().split(".", 1)[1].upper())
>     except IndexError:
> -        print >>sys.stderr, "Cannot guess realm from %s" % ( socket.getfqdn())
> -        sys.exit(1)
> +        opts.realm = ask("Realm", None)

^^^ We've already asked the user for a realm, why ask twice? Rather, we
should not set a default value if the users FQDN is broken because it
contains no dots.

>     try:
>         opts.domain = ask("Domain", opts.realm.split(".")[0])
>     except IndexError:
> -        print >>sys.stderr, "Cannot guess domain from %s" % ( opts.realm())
> -        sys.exit(1)
> +        opts.domain = ask("Domain", None)
^^^ Same here, let's just get the default value right in the first
place.

Cheers,

Jelmer

__________________________________________________
Do You Yahoo!?
Sie sind Spam leid? Yahoo! Mail verfügt über einen herausragenden Schutz gegen Massenmails.
http://mail.yahoo.com

Re: [SCM] Samba Shared Repository - branch master updated

2009-11-28T07:53:42Z

I moved the method down since I find it easier to locate there. That was the primary reason. Thanks for pointing out the two linespace convention for python. Didn't know that. I will reintroduce them.

Matthias

--- Jelmer Vernooij <jelmer@...> schrieb am Sa, 28.11.2009:

Von: Jelmer Vernooij <jelmer@...>
Betreff: Re: [SCM] Samba Shared Repository - branch master updated
An: samba-technical@...
CC: samba-cvs@...
Datum: Samstag, 28. November 2009, 15:16

Hi Matthias,

On Fri, 2009-11-27 at 08:59 -0600, Matthias Dieter Wallnöfer wrote:

> The branch, master has been updated
> via 0cc45b4... s4:upgrade.py - rework to make the upgrade s3 -> s4 possible again
> via bd6c133... s4:upgrade_from_s3 - Fix message outputs
> via dbb8989... s4:upgrade.py - the import of WINS databases don't seem to work always
> via 70b3161... s4:samba3.py - ignore comments in "smb.conf" files
> via 82adfa3... s4:samba3.py - don't read those informations out from the TDB
> via f299efa... s4:samba3.py - support the TDB version 3
>    from a5d854a... s4:provision - Fix up the provision of "standalone" and "member" mode
>
> http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master
>
>
> - Log -----------------------------------------------------------------
> commit 0cc45b47dc787abb2c3c31e4fc824798d9f3efe9
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date:   Fri Nov 27 15:50:26 2009 +0100
>
>    s4:upgrade.py - rework to make the upgrade s3 -> s4 possible again
>
>    Able to read basic settings "workgroup", "realm", "netbios name" and the
>    function mode from the s3 "smb.conf" and use them for the provision of s4.
>
> commit bd6c133e506fdb5dee13e0a144ef99c6d452be42
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date:   Fri Nov 27 15:49:18 2009 +0100
>
>    s4:upgrade_from_s3 - Fix message outputs
>
>    The quiet parameter was interpreted in the reverse manner.
>
> commit dbb8989e05ac3189a5eca11fa40d572388ea02fc
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date:   Fri Nov 27 15:02:18 2009 +0100
>
>    s4:upgrade.py - the import of WINS databases don't seem to work always
>
>    Disable it for now until the cause has been found
>
> commit 70b31610909544c58fd87c5e0aa00e02eb5f6d4b
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date:   Fri Nov 27 15:00:41 2009 +0100
>
>    s4:samba3.py - ignore comments in "smb.conf" files
>
> commit 82adfa39b75aa628c88f828278c6ac09335d1a49
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date:   Fri Nov 27 14:59:12 2009 +0100
>
>    s4:samba3.py - don't read those informations out from the TDB
>
>    At the moment those three calls are broken
>
> commit f299efa8f05c6a5b739222bdf75690a4591d3650
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date:   Fri Nov 27 14:58:37 2009 +0100
>
>    s4:samba3.py - support the TDB version 3
>
> -----------------------------------------------------------------------
>
> Summary of changes:
> source4/scripting/bin/upgrade_from_s3    | 6 +-
> source4/scripting/python/samba/samba3.py |   11 +-
> source4/scripting/python/samba/upgrade.py | 144 ++++++++++++++--------------
> 3 files changed, 81 insertions(+), 80 deletions(-)
>
>
> Changeset truncated at 500 lines:
>
> diff --git a/source4/scripting/bin/upgrade_from_s3 b/source4/scripting/bin/upgrade_from_s3
> index 03f4415..7e1e1fd 100755
> --- a/source4/scripting/bin/upgrade_from_s3
> +++ b/source4/scripting/bin/upgrade_from_s3
> @@ -50,14 +50,14 @@ opts, args = parser.parse_args()
>
> def message(text):
> """Print a message if quiet is not set."""
> - if opts.quiet:
> + if not opts.quiet:
> print text
>
> if len(args) < 1:
> parser.print_usage()
> sys.exit(1)
>
> -message("Reading Samba3 databases and smb.conf\n")
> +message("Reading Samba3 databases and smb.conf")
>
> libdir = args[0]
> if not os.path.isdir(libdir):
> @@ -71,7 +71,7 @@ else:
>
> samba3 = Samba3(libdir, smbconf)
>
> -message("Provisioning\n")
> +message("Provisioning")
>
> setup_dir = opts.setupdir
> if setup_dir is None:
> diff --git a/source4/scripting/python/samba/samba3.py b/source4/scripting/python/samba/samba3.py
> index 179efa2..c21b457 100644
> --- a/source4/scripting/python/samba/samba3.py
> +++ b/source4/scripting/python/samba/samba3.py
> @@ -509,7 +509,7 @@ class TdbSam(TdbDatabase):
> """Samba 3 TDB passdb backend reader."""
> def _check_version(self):
> self.version = fetch_uint32(self.tdb, "INFO/version\0") or 0
> - assert self.version in (0, 1, 2)
> + assert self.version in (0, 1, 2, 3)

^^ Please revert these changes. Clearly we don't support version 3 yet,
so we shouldn't pretend like we do. I'd rather see us fail with a clear
assertion error than breaking the v2 support. Furthermore, supporting
version 3 properly shouldn't be too hard.

> def usernames(self):
> """Iterate over the usernames in this Tdb database."""
> @@ -592,9 +592,10 @@ class TdbSam(TdbDatabase):
> for entry in hours:
> for i in range(8):
> user.hours.append(ord(entry) & (2 ** i) == (2 ** i))
> - (user.bad_password_count, data) = unpack_uint16(data)
> - (user.logon_count, data) = unpack_uint16(data)
> - (user.unknown_6, data) = unpack_uint32(data)
> + # FIXME
> + #(user.bad_password_count, data) = unpack_uint16(data)
> + #(user.logon_count, data) = unpack_uint16(data)
> + #(user.unknown_6, data) = unpack_uint32(data)
> assert len(data) == 0
> return user
>
> @@ -683,7 +684,7 @@ class ParamFile(object):
> section = None
> for i, l in enumerate(open(filename, 'r').xreadlines()):
> l = l.strip()
> - if not l:
> + if not l or l[0] == '#' or l[0] == ';':
> continue
> if l[0] == "[" and l[-1] == "]":
> section = self._sanitize_name(l[1:-1])

> diff --git a/source4/scripting/python/samba/upgrade.py b/source4/scripting/python/samba/upgrade.py
> index 8194552..44b43a1 100644
> --- a/source4/scripting/python/samba/upgrade.py
> +++ b/source4/scripting/python/samba/upgrade.py
> @@ -92,7 +93,6 @@ def import_sam_account(samldb,acc,domaindn,domainsid):
> "ntPwdHash:": acc.nt_password,
> })
>
> -
> def import_sam_group(samldb, sid, gid, sid_name_use, nt_name, comment, domaindn):
> """Upgrade a SAM group.
>
> @@ -132,7 +132,6 @@ def import_sam_group(samldb, sid, gid, sid_name_use, nt_name, comment, domaindn)
> "samba3SidNameUse": str(sid_name_use)
> })
>
> -

^^ The two empty lines are intentional here - PEP8 (standard coding
style for Python) specifies two empty lines between top-level objects.
Please don't remove them.

> @@ -157,7 +156,6 @@ def import_idmap(samdb,samba3_idmap,domaindn):
> "type": "group",
> "unixID": str(gid)})
>
> -
> def import_wins(samba4_winsdb, samba3_winsdb):
> """Import settings from a Samba3 WINS database.
>
> @@ -208,73 +206,6 @@ def import_wins(samba4_winsdb, samba3_winsdb):
> "objectClass": "winsMaxVersion",
> "maxVersion": str(version_id)})
>
> -def upgrade_provision(samba3, setup_dir, message, credentials, session_info, smbconf, targetdir):

^^^ Why did you move this function? It makes it very hard to spot what
actual changes you made.

Cheers,

Jelmer

__________________________________________________
Do You Yahoo!?
Sie sind Spam leid? Yahoo! Mail verfügt über einen herausragenden Schutz gegen Massenmails.
http://mail.yahoo.com

Re: [SCM] Samba Shared Repository - branch master updated

2009-11-28T07:51:44Z

Will revert soon.

Matthias

--- Jelmer Vernooij <jelmer@...> schrieb am Sa, 28.11.2009:

Von: Jelmer Vernooij <jelmer@...>
Betreff: Re: [SCM] Samba Shared Repository - branch master updated
An: samba-technical@...
CC: samba-cvs@...
Datum: Samstag, 28. November 2009, 15:22

Hi Matthias,

On Fri, 2009-11-27 at 09:42 -0600, Matthias Dieter Wallnöfer wrote:

> - Log -----------------------------------------------------------------
> commit 6c3e2417a0639cd7c367de93615c422cf5217456
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date:   Fri Nov 27 16:39:27 2009 +0100
>
>    s4:samba3.py (and test) - deactivate the tests until those parameters are fixed
>
> commit 08b3c396d7d534c9bfa29a7cd015a97d504d45a9
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date:   Fri Nov 27 16:35:28 2009 +0100
>
>    s4:upgrade.py - Umlaut problem
>
> commit 1af31aed0bd430d0af3a52962fa21f5c08309f01
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date:   Fri Nov 27 16:34:44 2009 +0100
>
>    s4:tests.sh - Make also here the change from "upgrade" to "upgrade_from_s3"
>
> -----------------------------------------------------------------------
>
> Summary of changes:
> source4/scripting/python/samba/samba3.py    | 2 +-
> source4/scripting/python/samba/tests/samba3.py | 6 +++---
> source4/scripting/python/samba/upgrade.py | 2 +-
> source4/selftest/tests.sh | 2 +-
> 4 files changed, 6 insertions(+), 6 deletions(-)

Please revert these as well - these tests were working fine previous to
your changes, they really shouldn't be disabled without good reason!

Cheers,

Jelmer

__________________________________________________
Do You Yahoo!?
Sie sind Spam leid? Yahoo! Mail verfügt über einen herausragenden Schutz gegen Massenmails.
http://mail.yahoo.com

Re: [SCM] Samba Shared Repository - branch master updated

2009-11-28T06:31:51Z

Hi Matthias,

> - Log -----------------------------------------------------------------
> commit 7d400715e9af2056690c03a1a2f45c7f343fa313
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 21:14:44 2009 +0100
>
> s4:registry/util - Don't include the trailing '\0' in the internal data format but add it on the back-conversion to a string
>
> As far as I know the registry library saves all data (including) strings without
> the null termination. So do it also here in a similar way.
Is there anything in particular that prompted this change? It's
unnecessary as talloc_convert_string_convenience should already
null-terminate.

Cheers,

Jelmer

signature.asc (852 bytes) Download Attachment

Re: [SCM] Samba Shared Repository - branch master updated

2009-11-28T06:22:14Z

Hi Matthias,

On Fri, 2009-11-27 at 09:42 -0600, Matthias Dieter Wallnöfer wrote:

> - Log -----------------------------------------------------------------
> commit 6c3e2417a0639cd7c367de93615c422cf5217456
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 16:39:27 2009 +0100
>
> s4:samba3.py (and test) - deactivate the tests until those parameters are fixed
>
> commit 08b3c396d7d534c9bfa29a7cd015a97d504d45a9
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 16:35:28 2009 +0100
>
> s4:upgrade.py - Umlaut problem
>
> commit 1af31aed0bd430d0af3a52962fa21f5c08309f01
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 16:34:44 2009 +0100
>
> s4:tests.sh - Make also here the change from "upgrade" to "upgrade_from_s3"
>
> -----------------------------------------------------------------------
>
> Summary of changes:
> source4/scripting/python/samba/samba3.py | 2 +-
> source4/scripting/python/samba/tests/samba3.py | 6 +++---
> source4/scripting/python/samba/upgrade.py | 2 +-
> source4/selftest/tests.sh | 2 +-
> 4 files changed, 6 insertions(+), 6 deletions(-)

Please revert these as well - these tests were working fine previous to

your changes, they really shouldn't be disabled without good reason!

Cheers,

Jelmer

signature.asc (852 bytes) Download Attachment

Re: [SCM] Samba Shared Repository - branch master updated

2009-11-28T06:16:30Z

Hi Matthias,

On Fri, 2009-11-27 at 08:59 -0600, Matthias Dieter Wallnöfer wrote:

> The branch, master has been updated
> via 0cc45b4... s4:upgrade.py - rework to make the upgrade s3 -> s4 possible again
> via bd6c133... s4:upgrade_from_s3 - Fix message outputs
> via dbb8989... s4:upgrade.py - the import of WINS databases don't seem to work always
> via 70b3161... s4:samba3.py - ignore comments in "smb.conf" files
> via 82adfa3... s4:samba3.py - don't read those informations out from the TDB
> via f299efa... s4:samba3.py - support the TDB version 3
> from a5d854a... s4:provision - Fix up the provision of "standalone" and "member" mode
>
> http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master
>
>
> - Log -----------------------------------------------------------------
> commit 0cc45b47dc787abb2c3c31e4fc824798d9f3efe9
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 15:50:26 2009 +0100
>
> s4:upgrade.py - rework to make the upgrade s3 -> s4 possible again
>
> Able to read basic settings "workgroup", "realm", "netbios name" and the
> function mode from the s3 "smb.conf" and use them for the provision of s4.
>
> commit bd6c133e506fdb5dee13e0a144ef99c6d452be42
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 15:49:18 2009 +0100
>
> s4:upgrade_from_s3 - Fix message outputs
>
> The quiet parameter was interpreted in the reverse manner.
>
> commit dbb8989e05ac3189a5eca11fa40d572388ea02fc
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 15:02:18 2009 +0100
>
> s4:upgrade.py - the import of WINS databases don't seem to work always
>
> Disable it for now until the cause has been found
>
> commit 70b31610909544c58fd87c5e0aa00e02eb5f6d4b
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 15:00:41 2009 +0100
>
> s4:samba3.py - ignore comments in "smb.conf" files
>
> commit 82adfa39b75aa628c88f828278c6ac09335d1a49
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 14:59:12 2009 +0100
>
> s4:samba3.py - don't read those informations out from the TDB
>
> At the moment those three calls are broken
>
> commit f299efa8f05c6a5b739222bdf75690a4591d3650
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 14:58:37 2009 +0100
>
> s4:samba3.py - support the TDB version 3
>
> -----------------------------------------------------------------------
>
> Summary of changes:
> source4/scripting/bin/upgrade_from_s3 | 6 +-
> source4/scripting/python/samba/samba3.py | 11 +-
> source4/scripting/python/samba/upgrade.py | 144 ++++++++++++++--------------
> 3 files changed, 81 insertions(+), 80 deletions(-)
>
>
> Changeset truncated at 500 lines:
>
> diff --git a/source4/scripting/bin/upgrade_from_s3 b/source4/scripting/bin/upgrade_from_s3
> index 03f4415..7e1e1fd 100755
> --- a/source4/scripting/bin/upgrade_from_s3
> +++ b/source4/scripting/bin/upgrade_from_s3
> @@ -50,14 +50,14 @@ opts, args = parser.parse_args()
>
> def message(text):
> """Print a message if quiet is not set."""
> - if opts.quiet:
> + if not opts.quiet:
> print text
>
> if len(args) < 1:
> parser.print_usage()
> sys.exit(1)
>
> -message("Reading Samba3 databases and smb.conf\n")
> +message("Reading Samba3 databases and smb.conf")
>
> libdir = args[0]
> if not os.path.isdir(libdir):
> @@ -71,7 +71,7 @@ else:
>
> samba3 = Samba3(libdir, smbconf)
>
> -message("Provisioning\n")
> +message("Provisioning")
>
> setup_dir = opts.setupdir
> if setup_dir is None:
> diff --git a/source4/scripting/python/samba/samba3.py b/source4/scripting/python/samba/samba3.py
> index 179efa2..c21b457 100644
> --- a/source4/scripting/python/samba/samba3.py
> +++ b/source4/scripting/python/samba/samba3.py
> @@ -509,7 +509,7 @@ class TdbSam(TdbDatabase):
> """Samba 3 TDB passdb backend reader."""
> def _check_version(self):
> self.version = fetch_uint32(self.tdb, "INFO/version\0") or 0
> - assert self.version in (0, 1, 2)
> + assert self.version in (0, 1, 2, 3)

^^ Please revert these changes. Clearly we don't support version 3 yet,

so we shouldn't pretend like we do. I'd rather see us fail with a clear
assertion error than breaking the v2 support. Furthermore, supporting
version 3 properly shouldn't be too hard.

^^ The two empty lines are intentional here - PEP8 (standard coding

style for Python) specifies two empty lines between top-level objects.
Please don't remove them.

^^^ Why did you move this function? It makes it very hard to spot what

actual changes you made.

Cheers,

Jelmer

signature.asc (852 bytes) Download Attachment

Re: [SCM] Samba Shared Repository - branch master updated

2009-11-28T06:10:26Z

On Fri, 2009-11-27 at 06:04 -0600, Matthias Dieter Wallnöfer wrote:

> The branch, master has been updated
> via a7fa3a9... s4:provision.py - cosmetic output correction
> via 6b835b0... s4:setup/provision - make the interactive mode work again
> from 7504b03... s4:WHATSNEW4.txt - Add also here a comment about the "upgrade_from_s3" script
>
> http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master
>
>
> - Log -----------------------------------------------------------------
> commit a7fa3a9703bf9f0c72031c84998f4fb21ba95429
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 13:07:52 2009 +0100
>
> s4:provision.py - cosmetic output correction
>
> commit 6b835b0691faab904246e587ffa6ff74b9fb53e2
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 13:02:31 2009 +0100
>
> s4:setup/provision - make the interactive mode work again
>
> -----------------------------------------------------------------------
>
> Summary of changes:
> source4/scripting/python/samba/provision.py | 2 +-
> source4/setup/provision | 16 +++-------------
> 2 files changed, 4 insertions(+), 14 deletions(-)

> index ed350dd..f1aa07c 100755
> --- a/source4/setup/provision
> +++ b/source4/setup/provision
> @@ -122,14 +122,6 @@ def message(text):
> if len(sys.argv) == 1:
> opts.interactive = True
>
> -if not opts.interactive and (opts.realm is None or opts.domain is None):
> - if opts.realm is None:
> - print >>sys.stderr, "No realm set"
> - if opts.domain is None:
> - print >>sys.stderr, "No domain set"
> - parser.print_usage()
> - sys.exit(1)
> -

^^^ This change isn't necessary to make the interactive mode work (it

explicitly checks for interactive mode not being in use), and it breaks
usage when not in interactive mode because we end up with no
realm/domain being set.

> if opts.interactive:
> from getpass import getpass
> import socket
> @@ -137,19 +129,17 @@ if opts.interactive:
> if default is not None:
> print "%s [%s]: " % (prompt,default),
> else:
> - print "%s: " % (prompt,),
> + print "%s: " % (prompt),
> return sys.stdin.readline().rstrip("\n") or default
> try:
> opts.realm = ask("Realm", socket.getfqdn().split(".", 1)[1].upper())
> except IndexError:
> - print >>sys.stderr, "Cannot guess realm from %s" % ( socket.getfqdn())
> - sys.exit(1)
> + opts.realm = ask("Realm", None)

^^^ We've already asked the user for a realm, why ask twice? Rather, we

should not set a default value if the users FQDN is broken because it
contains no dots.

> try:
> opts.domain = ask("Domain", opts.realm.split(".")[0])
> except IndexError:
> - print >>sys.stderr, "Cannot guess domain from %s" % ( opts.realm())
> - sys.exit(1)
> + opts.domain = ask("Domain", None)
^^^ Same here, let's just get the default value right in the first
place.

Cheers,

Jelmer

signature.asc (852 bytes) Download Attachment

Re: [SCM] Samba Shared Repository - branch master updated

2009-11-28T06:06:48Z

Hi Matthias,

On Fri, 2009-11-27 at 05:26 -0600, Matthias Dieter Wallnöfer wrote:

> The branch, master has been updated
> via 7504b03... s4:WHATSNEW4.txt - Add also here a comment about the "upgrade_from_s3" script
> via 3d57da8... s4:howto.txt - add a notice about the upgrade possibilities and fix line spaces
> via fd31328... s4:upgrade script - rename it to "upgrade_from_s3" and do some rework
> from d6c60f8... s3-nsstest: drastically shrink size and dependencies of nsstest binary.
>
> http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master
>
>
> - Log -----------------------------------------------------------------
> commit 7504b03b541026c84a4f454b7572a3280296a8d9
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 11:07:11 2009 +0100
>
> s4:WHATSNEW4.txt - Add also here a comment about the "upgrade_from_s3" script
>
> commit 3d57da80e574f5f47effb41fd45361eec22f119e
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 11:02:20 2009 +0100
>
> s4:howto.txt - add a notice about the upgrade possibilities and fix line spaces
>
> commit fd313282a24e58570c2fa5b3fdfd0b84d0053363
> Author: Matthias Dieter Wallnöfer <mwallnoefer@...>
> Date: Fri Nov 27 10:50:03 2009 +0100
>
> s4:upgrade script - rename it to "upgrade_from_s3" and do some rework
>
> - Give a better name to the script
> - Move it to the location where also "upgradeprovision" resides
> - Fix up trailing whitespaces and tabs
>
> -----------------------------------------------------------------------
>
> Summary of changes:
> WHATSNEW4.txt | 11 ++++--
> howto4.txt | 18 +++++++---
> .../upgrade => scripting/bin/upgrade_from_s3} | 37 +++++++++++---------
> 3 files changed, 41 insertions(+), 25 deletions(-)
> rename source4/{setup/upgrade => scripting/bin

Please keep the upgrade script under setup/. Its name is too generic to

be installed in /bin. Alternatively, perhaps we should rename it to
something specific to Samba.

This would have broken the build (there are tests for the upgrade script
that you haven't changed), please update the blackbox tests as well when
renaming binaries.

Cheers,

Jelmerb

signature.asc (852 bytes) Download Attachment

Re: how to build samba 3.5

2009-11-28T05:52:15Z

Stefan (metze) Metzmacher wrote:
> Hi Olivier,
>
>>> On Fri, Nov 27, 2009 at 03:37:12PM +0100, Olivier Sessink wrote:
>>>> I would like to port a VFS module to samba 3.5 (because of the API changes).
>>>>
>>>> I've tried to build a samba git checkout, but autoconf fails:
>>>>
>>>> /usr/bin/m4:configure.in:23: cannot open `pkg.m4': No such file or directory
[..]

> you should always use './autogen.sh' instead of autoconf directly.

thanks, works like a charm.

Olivier

the function of SMB_STRUCT_STAT *sbuf in vfs readdir() ?

2009-11-28T05:51:37Z

Hi all,

what is the function of SMB_STRUCT_STAT *sbuf in the VFS readdir API ?

is it supposed to be filled by the SMB_VFS_NEXT_READDIR call? (e.g. can
you use it after that call?)

thanks,

Olivier

s4-drs: cope with bogus empty attributes from w2k8-r2 (Re: [SCM] Samba Shared Repository - branch master updated)

2009-11-28T04:10:25Z

Hi Tridge,

> The branch, master has been updated
> via 1287c1d... s4-drs: cope with bogus empty attributes from w2k8-r2
> from db41a0a... s4: fix SD update and password change in upgrade script
>
> http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master
>
>
> - Log -----------------------------------------------------------------
> commit 1287c1d115fb7e8f3954bc05ff65007968403a9c
> Author: Andrew Tridgell <tridge@...>
> Date: Sat Nov 28 13:27:06 2009 +1100
>
> s4-drs: cope with bogus empty attributes from w2k8-r2
>
> w2k8-r2 sometimes sends empty attributes with completely bogus attrid
> values in a DRS replication response. This allows us to continue with
> the vampire operation despite these broken elements.
>
> -----------------------------------------------------------------------
>
> Summary of changes:
> source4/dsdb/repl/replicated_objects.c | 17 +++++++++++++++++
> 1 files changed, 17 insertions(+), 0 deletions(-)
>
>
> Changeset truncated at 500 lines:
>
> diff --git a/source4/dsdb/repl/replicated_objects.c b/source4/dsdb/repl/replicated_objects.c
> index 020d5f1..a8a93e4 100644
> --- a/source4/dsdb/repl/replicated_objects.c
> +++ b/source4/dsdb/repl/replicated_objects.c
> @@ -129,6 +129,15 @@ static WERROR dsdb_convert_object_ex(struct ldb_context *ldb,
> }
>
> status = dsdb_attribute_drsuapi_to_ldb(ldb, schema, a, msg->elements, e);
> + if (!NT_STATUS_IS_OK(status) && a->value_ctr.num_values == 0) {
> + /* w2k8-r2 occasionally sends bogus empty
> + attributes with rubbish attribute IDs. The
> + only think we can do is discard these */
> + DEBUG(0,(__location__ ": Discarding bogus empty DsReplicaAttribute with attid 0x%x\n",
> + a->attid));
> + ZERO_STRUCTP(e);
> + continue;
> + }
> W_ERROR_NOT_OK_RETURN(status);
>
> m->attid = a->attid;
> @@ -149,6 +158,14 @@ static WERROR dsdb_convert_object_ex(struct ldb_context *ldb,
> }
> }
>
> + /* delete any empty elements */
> + for (i=0; i < msg->num_elements; i++) {
> + if (msg->elements[i].name == NULL) {
> + ldb_msg_remove_element(msg, &msg->elements[i]);
> + i--;
> + }
> + }
> +

We need to be careful about this, an empty element means
we should remove existing values.
If remove the empty ones here, we need to let the repl_meta_data module
remove them based on the received meta_data array.

metze

signature.asc (268 bytes) Download Attachment

Re: how to build samba 3.5

2009-11-27T22:23:10Z

Hi Olivier,

>> On Fri, Nov 27, 2009 at 03:37:12PM +0100, Olivier Sessink wrote:
>>> I would like to port a VFS module to samba 3.5 (because of the API changes).
>>>
>>> I've tried to build a samba git checkout, but autoconf fails:
>>>
>>> /usr/bin/m4:configure.in:23: cannot open `pkg.m4': No such file or directory
>>>
>>> Is there a quick way to resolv this? Or another way to port my vfs
>>> module: is there a samba-3.5 snapshot available somewhere?
>> I'm using autoconf 2.63, with that it works. What exact
>> command did you issue?
>
> autoconf 2.61
>
> $ cd samba/source3
> $ autoconf

you should always use './autogen.sh' instead of autoconf directly.

metze

signature.asc (268 bytes) Download Attachment

Re: Commit: 0920e0b63b806c8ed4839271048dd4924ed02b2b

2009-11-27T21:32:23Z

Hi Kamen,

> Actually the bug you've fixed with this commit is not a bug - it was
> intended to work that way on purpose :).

it was a bug I think :-)

> My aim was to make it work as follows:
> 1. If LDB_URL is set in environment, then test uses this url and
> temporary LDB used for testing is not deleted after test is done.
> This way we can exam data in LDB if anything goes wrong and we
> need to review data used for testing.

it is far too dangerous. The environment variable LDB_URL is the
standard environment variable used by the ldb tools to avoid using the
-H option. It is not uncommon for someone to set this to point at a
real LDB. In my case I had it pointed at my sam.ldb, which meant that
running "make test" not only failed, it also corrupted by sam database.

> Could we just use another name for the environment variable used?

no, please don't use an environment variable for this sort of thing.

If you want to have a way to override the location of this test ldb,
then use lp_parm_string() and then pass the location using the
--option command line option. To see some examples of this, try:

git grep lp_parm_string source4/torture/

for example, you could have:

ldb_url = lp_parm_string(tctx->lp_ctx, NULL, "drstest", "ldb_url");

and then set it with:

--option drstest:ldb_url=foo.ldb

Cheers, Tridge

Re: [PATCH] s4-drs: replmd_delete implementation

2009-11-27T21:24:17Z

Hi Eduardo,

> My git tree is updated to these latest changes: git://
> repo.or.cz/Samba/eduardoll.git

Thanks, I've looked through your changes and I have some comments
below.

> It will fail whenever sAMAccountType is specified in the message to be
> modified. Just for testing purpose, I modified the code above to not fail
> when el->flag == LDB_FLAG_MOD_DELETE:
>
> el = ldb_msg_find_element(req->op.mod.message, "sAMAccountType");
> if ((el != NULL) && (el->flags != LDB_FLAG_MOD_DELETE)) {
> ldb_asprintf_errstring(ldb,
> "sAMAccountType must not be specified!");
> return LDB_ERR_UNWILLING_TO_PERFORM;
> }

I think a better approach would be to set the "relax" control on the
modify operation. The "relax" control can be set for internal
operations to say "don't do strict checking, I know what I'm doing".

If you run "git grep RELAX_OID" you'll find some existing places that
use this control for similar purposes. You'll need to check for the
relax control in samldb.c and if it is set then allow for modifies on
sAMAccountType. Then in repl_meta_data.c you'll need to make sure the
modify is done with the relax control set.

> Regarding the delete test on Windows 2008, I could not find out an easy way
> to insert a new object into the CN=Schema,CN=Configuration,DC=x partition.

If you look in lib/ldb/tests/python/ldap.py you'll see some examples
of adding a new schema entry.

I looked into this a bit further, and it looks to me like deleting
schema objects is not something we should be allowing anyway.

I think the simplest thing to do is to not do the rename/modify and
instead do a normal delete if the NC doesn't have a Deleted Objects
container.

> Another thing I noticed is that all the deleted objects in
> *
> CN=Deleted Objects,CN=Configuration,DC=x*
>
> have lastKnownParent:
>
> CN=NTDS
> Settings,CN=W2K8,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=adsamba,DC=ltc,DC=inovasoft,DC=unicamp,DC=br.

that's just because those are the objects that have been deleted on
your system.

> When browsing my DC using LDP I also noticed that shows a wellKnownObjects
> list that has de deleted container:
>
> wellKnownObjects:
> B:32:18E2EA80684F11D2B9AA00C04F79F805:CN=Deleted
> Objects,DC=adsamba,DC=ltc,DC=inovasoft,DC=unicamp,DC=br;

right

>
> Does samba has this wellKnownObjects? If so, should I create a entry in the
> samba's wellKnownObjects list?

Samba already creates this wellKnownObjects entry (try "git grep
wellKnownObject" and you'll see it in setup/*.ldif).

I think the correct way of finding the Deleted Objects container for a
NC is to fetch the wellKnownObjects attribute for the root of the NC,
and then use the GUID embedded in that to find the Deleted Objects
container. You could modify dsdb_get_deleted_objects_dn() to do this
(thought you will have to pass the DN of the object being deleted).

Now for some more specific comments on the code in your tree.

In dsdb_get_deleted_objects_dn() you do a search for the
defaultNamingContext. It would have been much simpler to just call
samdb_base_dn() which already returns this.

I think though that dsdb_get_deleted_objects_dn() needs to be redone
to take a DN argument, as the correct Deleted Objects container is
different depending on what DN is being deleted. So I think you should
probably first write a function which finds the root of a NC given a
DN, and then construct the Deleted Objects DN from that. The function
to find the root of a NC might look like this:

struct ldb_dn *dsdb_find_nc_root(struct ldb_context *samdb, const struct ldb_dn *dn);

it could operate in a couple of different ways. One possibility is to
get the list of naming contexts from the rootDSE and then find the NC
that matches the DN that is passed in. To make this easy you would
sort the DNs from the list of naming contexts using a function similar
to partition_sort_compare(). You'd then find the matching one by
calling ldb_dn_compare_base().

Another alternative is to write a function that walks up the tree,
looking for a DN with instanceType containing
INSTANCE_TYPE_IS_NC_HEAD.

In your replmd_delete() function you have the same bug that we
discussed in the tutorial last week where you use
ldb_dn_add_child_fmt() on a DN that came from a cache, thus modifying
that DN. The current dsdb_get_deleted_objects_dn() function stores and
returns values from cache.deleted_dn. When you call
ldb_dn_add_child_fmt() you are modifying that DN directly, which means
the 2nd time you call dsdb_get_deleted_objects_dn() you will not get
the Deleted Objects DN, but instead you will get the DN you last
modified.

Further down your replmd_delete() code you directly call
ldb_rename(). That is OK for now, but later we will need to add a
dsdb_module_rename() call in dsdb/samdb/ldb_modules/util.c to allow
you to do a rename on the correct part of the module stack.

For the modify part of the replmd_delete() code, you need to read
section 3.1.1.5.5.1.1 of [MS-ADTS].pdf to see how you should be
handling which attributes need to be added and removed from the object
when it is deleted. You should probably also read the other parts of
[MS-ADTS] that deal with deleted objects.

If you read that doc, you'll see that you will need to do something
like this:

1) fetch the existing object from the database

2) for each of the attributes of the object, get the schema attribute
structure using dsdb_attribute_by_lDAPDisplayName(). That will
return a struct dsdb_attribute, which contains a searchFlags element,
which says whether the attribute should be preserved on deletion.

3) you'll also need to check the attribute against the list of
attributes specified in 3.1.1.5.5.1.1 as being always preserved.

Then for each attribute in the existing object that should be removed,
you'll need to build up the msg ready for the modify. You need to make
sure that you only specify an attribute for deletion if it is in the
object currently (other the ldb_modify will give an error).

Finally, we will need a dsdb_module_modify() function, similar to what
I mentioned about dsdb_module_rename() above.

Would you like me to help you by creating some of the helper functions
you will need? For example, I could add the dsdb_module_modify(),
dsdb_module_rename() and dsdb_find_nc_root() functions. I could also
do the "relax" change needed in samldb.c. That would just leave you
with the core logic of replmd_delete(). Or if you prefer to do all the
functions yourself let me know.

Cheers, Tridge