[
https://issues.apache.org/jira/browse/LUCENE-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724074#action_12724074 ]
Simon Willnauer commented on LUCENE-1717:
-----------------------------------------
Regarding buffered delete's RAM usage, accounting an exact number is quite difficult in this case as there are many strings involved (Terms with field and value) . BufferedDeletes#terms stores <Term, Num> and BufferedDeletes#queries stores <Query, Num> in both cases the value part is easy to account while especially for query the memory consumption is hard to guess similarly the amount of memory a Term takes.
On the other hand I would like to have a notion of memory consumption os BufferedDeletes but the IndexWriters#setRAMBufferSizeMB javaDoc clearly says that this does not include the memory used by buffered deletes. I would rather tend to leave it as it is and make it clear in javadoc / wiki that setMaxBufferedDeleteTerms is the way to go if you run into memory problems. Feels quite ambiguous to estimate the memory of buffered deletes.
bq. I think is a good default once we fix the accounting in IndexWriter to properly account for buffered delete's RAM usage.
is there already an issue to fix the RAM usage?
> IndexWriter does not properly account for the RAM consumed by pending deletes
> -----------------------------------------------------------------------------
>
> Key: LUCENE-1717
> URL:
https://issues.apache.org/jira/browse/LUCENE-1717> Project: Lucene - Java
> Issue Type: Bug
> Affects Versions: 2.4, 2.4.1
> Reporter: Michael McCandless
> Fix For: 2.9
>
> Attachments: BufferedDeletes_beautification.patch
>
>
> IndexWriter, with autoCommit false, is able to carry buffered deletes for quite some time before materializing them to docIDs (thus freeing up RAM used).
> It's only on triggering a merge (or, commit/close) that the deletes are materialized and the RAM is freed.
> I expect this in practice is a smallish amount of RAM, but we should still fix it.
> I don't have a patch yet so if someone wants to grab this, feel free!!
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail:
java-dev-unsubscribe@...
For additional commands, e-mail:
java-dev-help@...
