winbindd_cache.tdb size issue in 3.2.15

[valyekhin]

Hello.

After update Samba from v. 3.0.28a to v.  3.2.15 we've noticed that  
winbibdd_cache.tdb file has been increased up to 10 times. After some  
testing we got the following file sizes (in bytes)

6 users in AD:  3.0.28a - 53248, 3.2.15 - 389120
1005 users in AD: 0.28a - 1802240, 3.2.15 - 13963264
10005 users in AD: 0.28a - 18464768, 3.2.15 - 144846848

We've found out that 3.2.15 uses another algorythm for making room for  
new records in the tdb file. If 3.0.28a allocates space for at least  
10 more records, 3.2.15 allocates space for at least 100 more records,  
and at least 25% more space.
We've created a patch which transfers algorythm for allocation space from
3.0.28a.
After using this patch we've got the next results:
6 users in AD:  3.0.28a - 53248, 3.2.15 patched - 61440
1005 users in AD: 0.28a - 1802240, 3.2.15 patched - 1922916
10005 users in AD: 0.28a - 18464768, 3.2.15 patched - 19709952

But we have some doubts as for the correctness of the patch and we're  
not sure that it won't damage anything.

Please, clarify this item.

Thanks...

Regards, Vladimir

[samba-3.2.15.winbind_cache.size.research.diff]

---

*** samba-3.2.15/source/lib/tdb/common/io.c     2009-09-30 15:24:50.000000000 +0300
--- samba-3.2.15.patched/source/lib/tdb/common/io.c     2009-10-23 17:56:24.000000000 +0300
***************
*** 306,316 ****
   /* must know about any previous expansions by another process */
   tdb->methods->tdb_oob(tdb, tdb->map_size + 1, 1);
 
!       /* always make room for at least 100 more records, and at
!            least 25% more space. Round the database up to a multiple
!            of the page size */
!       new_size = MAX(tdb->map_size + size*100, tdb->map_size * 1.25);
!       size = TDB_ALIGN(new_size, tdb->page_size) - tdb->map_size;
 
   if (!(tdb->flags & TDB_INTERNAL))
      tdb_munmap(tdb);
--- 306,314 ----
   /* must know about any previous expansions by another process */
   tdb->methods->tdb_oob(tdb, tdb->map_size + 1, 1);
 
!        /* always make room for at least 10 more records, and round
!           the database up to a multiple of the page size */
!       size = TDB_ALIGN(tdb->map_size + size*10, tdb->page_size) - tdb->map_size;
 
   if (!(tdb->flags & TDB_INTERNAL))
      tdb_munmap(tdb);

---

[Jeremy Allison]

On Fri, Oct 30, 2009 at 04:50:58PM +0100, valyekhin@... wrote:

> Hello.
>
> After update Samba from v. 3.0.28a to v.  3.2.15 we've noticed that  
> winbibdd_cache.tdb file has been increased up to 10 times. After some  
> testing we got the following file sizes (in bytes)
>
> 6 users in AD:  3.0.28a - 53248, 3.2.15 - 389120
> 1005 users in AD: 0.28a - 1802240, 3.2.15 - 13963264
> 10005 users in AD: 0.28a - 18464768, 3.2.15 - 144846848
>
> We've found out that 3.2.15 uses another algorythm for making room for  
> new records in the tdb file. If 3.0.28a allocates space for at least 10
> more records, 3.2.15 allocates space for at least 100 more records, and
> at least 25% more space.
> We've created a patch which transfers algorythm for allocation space from
> 3.0.28a.
> After using this patch we've got the next results:
> 6 users in AD:  3.0.28a - 53248, 3.2.15 patched - 61440
> 1005 users in AD: 0.28a - 1802240, 3.2.15 patched - 1922916
> 10005 users in AD: 0.28a - 18464768, 3.2.15 patched - 19709952
>
> But we have some doubts as for the correctness of the patch and we're  
> not sure that it won't damage anything.
>
> Please, clarify this item.

The increase in size is by design as it helps prevent fragmentation
in the tdb files. Why is the increased size causing a problem for you ?

Jeremy.

[valyekhin]

Hello.

On Fri, Oct 30, 2009 at 7:39 PM, Jeremy Allison <jra@...> wrote:

> The increase in size is by design as it helps prevent fragmentation
> in the tdb files. Why is the increased size causing a problem for you ?

It is connected with our system volume which has only 1GB.This system
volume store tdb files and for 65000 users became overflow.We have to
think about resize this volume but it is big change (problem with
compatibility).

--
Vladimir Aliokhin