I've found the problem with this now.
It seems to be a bug within JDBM 1.0. This was fixed in JDBMs CVS repository back in February last year - before we branched:
http://jdbm.cvs.sourceforge.net/viewvc/jdbm/jdbm/src/main/jdbm/btree/BTree.java?r1=1.17&r2=1.18I guess we just branched the 1.0 code. Can we just go ahead and apply this patch or should we sync up with the latest JDBM trunk?
Thanks,
Martin
-----Original Message-----
From: Martin Alderson [mailto:
Martin.Alderson@...]
Sent: Thursday 19 March 2009 18:07
To:
dev@...
Subject: [ApacheDS] [JDBM] Index .db files grow from attribute replace
Hi all,
Firstly, apologies if this mail arrives in HTML format. My company is in the middle of switching e-mail systems and may be converting all outgoing mail to HTML at the moment.
I have found that when just replacing an indexed attributes value the JDBM table seems to grow. From doing this 300000 times I find that I have a 66 meg uid.db file which is growing at the rate of around 2.2 meg per 10000 modifications and showing no signs of slowing down.
This is with a stock ApacheDS 1.5.4. The following code should allow you to reproduce the problem:
////////////////////////////////////////////////////////////
import java.util.Hashtable;
import javax.naming.Context;
import javax.naming.directory.BasicAttribute;
import javax.naming.directory.DirContext;
import javax.naming.directory.ModificationItem;
import javax.naming.ldap.InitialLdapContext;
import javax.naming.ldap.LdapContext;
public class StressTestIndex {
public static void main (String[] args) throws Exception {
Hashtable env = new Hashtable();
env.put(Context.SECURITY_PRINCIPAL, "uid=admin,ou=system");
env.put(Context.SECURITY_CREDENTIALS, "secret");
env.put(Context.SECURITY_AUTHENTICATION, "simple");
env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
env.put(Context.PROVIDER_URL, "ldap://localhost:10389/");
LdapContext ldapContext = (LdapContext) new InitialLdapContext(env, null);
for (int i = 0; i < 100000; i++) {
ldapContext.modifyAttributes("uid=admin,ou=system", new ModificationItem[] { new ModificationItem(DirContext.REPLACE_ATTRIBUTE, new BasicAttribute("uid", "admin")) });
}
}
}
////////////////////////////////////////////////////////////
Emmanuel pointed me to the nifty LaunchDiagnosticUI extended request which I send as:
ldapContext.extendedOperation(new LaunchDiagnosticUiRequest(0));
Looking at the JDBM table with this tool it looks like the index only has the one entry in it as it should - so I think the rest of the space in the db file must be wasted space. Does anyone know if the JDBM table can be compacted? If it can should ApacheDS be doing that periodically?
Thanks,
Martin
winmail.dat (6K) 