[jira] Created: (LUCENE-1566) Large Lucene index can hit false OOM due to Sun JRE issue
|
View:
New views
7 Messages
—
Rating Filter:
Alert me
|
| < Prev | 1 - 2 | Next > |
 |
[jira] Commented: (LUCENE-1566) Large Lucene index can hit false OOM due to Sun JRE issue
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/LUCENE-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731880#action_12731880 ] Michael McCandless commented on LUCENE-1566: -------------------------------------------- SimpleFSDirectory is missing from the last patch? > Large Lucene index can hit false OOM due to Sun JRE issue > --------------------------------------------------------- > > Key: LUCENE-1566 > URL: https://issues.apache.org/jira/browse/LUCENE-1566 > Project: Lucene - Java > Issue Type: Bug > Components: Index > Affects Versions: 2.4.1 > Reporter: Michael McCandless > Assignee: Simon Willnauer > Priority: Minor > Fix For: 2.9 > > Attachments: LUCENE-1566.patch, LUCENE-1566.patch, LUCENE_1566_IndexInput.patch, LUCENE_1566_IndexInput_Changes.patch > > > This is not a Lucene issue, but I want to open this so future google > diggers can more easily find it. > There's this nasty bug in Sun's JRE: > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6478546 > The gist seems to be, if you try to read a large (eg 200 MB) number of > bytes during a single RandomAccessFile.read call, you can incorrectly > hit OOM. Lucene does this, with norms, since we read in one byte per > doc per field with norms, as a contiguous array of length maxDoc(). > The workaround was a custom patch to do large file reads as several > smaller reads. > Background here: > http://www.nabble.com/problems-with-large-Lucene-index-td22347854.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscribe@... For additional commands, e-mail: java-dev-help@... |
|
|
[jira] Updated: (LUCENE-1566) Large Lucene index can hit false OOM due to Sun JRE issue
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/LUCENE-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Willnauer updated LUCENE-1566: ------------------------------------ Attachment: LUCENE_1566_IndexInput_Changes.patch added missing SimpleFSDirectory. > Large Lucene index can hit false OOM due to Sun JRE issue > --------------------------------------------------------- > > Key: LUCENE-1566 > URL: https://issues.apache.org/jira/browse/LUCENE-1566 > Project: Lucene - Java > Issue Type: Bug > Components: Index > Affects Versions: 2.4.1 > Reporter: Michael McCandless > Assignee: Simon Willnauer > Priority: Minor > Fix For: 2.9 > > Attachments: LUCENE-1566.patch, LUCENE-1566.patch, LUCENE_1566_IndexInput.patch, LUCENE_1566_IndexInput_Changes.patch, LUCENE_1566_IndexInput_Changes.patch > > > This is not a Lucene issue, but I want to open this so future google > diggers can more easily find it. > There's this nasty bug in Sun's JRE: > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6478546 > The gist seems to be, if you try to read a large (eg 200 MB) number of > bytes during a single RandomAccessFile.read call, you can incorrectly > hit OOM. Lucene does this, with norms, since we read in one byte per > doc per field with norms, as a contiguous array of length maxDoc(). > The workaround was a custom patch to do large file reads as several > smaller reads. > Background here: > http://www.nabble.com/problems-with-large-Lucene-index-td22347854.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscribe@... For additional commands, e-mail: java-dev-help@... |
|
|
[jira] Commented: (LUCENE-1566) Large Lucene index can hit false OOM due to Sun JRE issue
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/LUCENE-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731886#action_12731886 ] Simon Willnauer commented on LUCENE-1566: ----------------------------------------- bq. SimpleFSDirectory is missing from the last patch? ups! :) > Large Lucene index can hit false OOM due to Sun JRE issue > --------------------------------------------------------- > > Key: LUCENE-1566 > URL: https://issues.apache.org/jira/browse/LUCENE-1566 > Project: Lucene - Java > Issue Type: Bug > Components: Index > Affects Versions: 2.4.1 > Reporter: Michael McCandless > Assignee: Simon Willnauer > Priority: Minor > Fix For: 2.9 > > Attachments: LUCENE-1566.patch, LUCENE-1566.patch, LUCENE_1566_IndexInput.patch, LUCENE_1566_IndexInput_Changes.patch, LUCENE_1566_IndexInput_Changes.patch > > > This is not a Lucene issue, but I want to open this so future google > diggers can more easily find it. > There's this nasty bug in Sun's JRE: > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6478546 > The gist seems to be, if you try to read a large (eg 200 MB) number of > bytes during a single RandomAccessFile.read call, you can incorrectly > hit OOM. Lucene does this, with norms, since we read in one byte per > doc per field with norms, as a contiguous array of length maxDoc(). > The workaround was a custom patch to do large file reads as several > smaller reads. > Background here: > http://www.nabble.com/problems-with-large-Lucene-index-td22347854.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscribe@... For additional commands, e-mail: java-dev-help@... |
|
|
[jira] Updated: (LUCENE-1566) Large Lucene index can hit false OOM due to Sun JRE issue
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/LUCENE-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1566: --------------------------------------- Attachment: LUCENE-1566.patch OK I reworked the patch some, tweaking javadocs, changes, etc., and simplifying the loops that read the bytes inside NIOFSDir & SimpleFSDir. I think it's ready to commit. Simon can you take a look? Thanks. > Large Lucene index can hit false OOM due to Sun JRE issue > --------------------------------------------------------- > > Key: LUCENE-1566 > URL: https://issues.apache.org/jira/browse/LUCENE-1566 > Project: Lucene - Java > Issue Type: Bug > Components: Index > Affects Versions: 2.4.1 > Reporter: Michael McCandless > Assignee: Simon Willnauer > Priority: Minor > Fix For: 2.9 > > Attachments: LUCENE-1566.patch, LUCENE-1566.patch, LUCENE-1566.patch, LUCENE_1566_IndexInput.patch, LUCENE_1566_IndexInput_Changes.patch, LUCENE_1566_IndexInput_Changes.patch > > > This is not a Lucene issue, but I want to open this so future google > diggers can more easily find it. > There's this nasty bug in Sun's JRE: > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6478546 > The gist seems to be, if you try to read a large (eg 200 MB) number of > bytes during a single RandomAccessFile.read call, you can incorrectly > hit OOM. Lucene does this, with norms, since we read in one byte per > doc per field with norms, as a contiguous array of length maxDoc(). > The workaround was a custom patch to do large file reads as several > smaller reads. > Background here: > http://www.nabble.com/problems-with-large-Lucene-index-td22347854.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscribe@... For additional commands, e-mail: java-dev-help@... |
|
|
[jira] Commented: (LUCENE-1566) Large Lucene index can hit false OOM due to Sun JRE issue
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/LUCENE-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12732000#action_12732000 ] Simon Willnauer commented on LUCENE-1566: ----------------------------------------- bq. Simon can you take a look? I looked at it closely and did run testcases again. Looks good to me! Thanks for looking at it again! simon > Large Lucene index can hit false OOM due to Sun JRE issue > --------------------------------------------------------- > > Key: LUCENE-1566 > URL: https://issues.apache.org/jira/browse/LUCENE-1566 > Project: Lucene - Java > Issue Type: Bug > Components: Index > Affects Versions: 2.4.1 > Reporter: Michael McCandless > Assignee: Simon Willnauer > Priority: Minor > Fix For: 2.9 > > Attachments: LUCENE-1566.patch, LUCENE-1566.patch, LUCENE-1566.patch, LUCENE_1566_IndexInput.patch, LUCENE_1566_IndexInput_Changes.patch, LUCENE_1566_IndexInput_Changes.patch > > > This is not a Lucene issue, but I want to open this so future google > diggers can more easily find it. > There's this nasty bug in Sun's JRE: > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6478546 > The gist seems to be, if you try to read a large (eg 200 MB) number of > bytes during a single RandomAccessFile.read call, you can incorrectly > hit OOM. Lucene does this, with norms, since we read in one byte per > doc per field with norms, as a contiguous array of length maxDoc(). > The workaround was a custom patch to do large file reads as several > smaller reads. > Background here: > http://www.nabble.com/problems-with-large-Lucene-index-td22347854.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscribe@... For additional commands, e-mail: java-dev-help@... |
|
|
[jira] Commented: (LUCENE-1566) Large Lucene index can hit false OOM due to Sun JRE issue
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/LUCENE-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12732051#action_12732051 ] Michael McCandless commented on LUCENE-1566: -------------------------------------------- OK thanks Simon; I'll commit shortly. > Large Lucene index can hit false OOM due to Sun JRE issue > --------------------------------------------------------- > > Key: LUCENE-1566 > URL: https://issues.apache.org/jira/browse/LUCENE-1566 > Project: Lucene - Java > Issue Type: Bug > Components: Index > Affects Versions: 2.4.1 > Reporter: Michael McCandless > Assignee: Simon Willnauer > Priority: Minor > Fix For: 2.9 > > Attachments: LUCENE-1566.patch, LUCENE-1566.patch, LUCENE-1566.patch, LUCENE_1566_IndexInput.patch, LUCENE_1566_IndexInput_Changes.patch, LUCENE_1566_IndexInput_Changes.patch > > > This is not a Lucene issue, but I want to open this so future google > diggers can more easily find it. > There's this nasty bug in Sun's JRE: > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6478546 > The gist seems to be, if you try to read a large (eg 200 MB) number of > bytes during a single RandomAccessFile.read call, you can incorrectly > hit OOM. Lucene does this, with norms, since we read in one byte per > doc per field with norms, as a contiguous array of length maxDoc(). > The workaround was a custom patch to do large file reads as several > smaller reads. > Background here: > http://www.nabble.com/problems-with-large-Lucene-index-td22347854.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscribe@... For additional commands, e-mail: java-dev-help@... |
|
|
[jira] Resolved: (LUCENE-1566) Large Lucene index can hit false OOM due to Sun JRE issue
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/LUCENE-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-1566. ---------------------------------------- Resolution: Fixed Thanks Simon! > Large Lucene index can hit false OOM due to Sun JRE issue > --------------------------------------------------------- > > Key: LUCENE-1566 > URL: https://issues.apache.org/jira/browse/LUCENE-1566 > Project: Lucene - Java > Issue Type: Bug > Components: Index > Affects Versions: 2.4.1 > Reporter: Michael McCandless > Assignee: Simon Willnauer > Priority: Minor > Fix For: 2.9 > > Attachments: LUCENE-1566.patch, LUCENE-1566.patch, LUCENE-1566.patch, LUCENE_1566_IndexInput.patch, LUCENE_1566_IndexInput_Changes.patch, LUCENE_1566_IndexInput_Changes.patch > > > This is not a Lucene issue, but I want to open this so future google > diggers can more easily find it. > There's this nasty bug in Sun's JRE: > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6478546 > The gist seems to be, if you try to read a large (eg 200 MB) number of > bytes during a single RandomAccessFile.read call, you can incorrectly > hit OOM. Lucene does this, with norms, since we read in one byte per > doc per field with norms, as a contiguous array of length maxDoc(). > The workaround was a custom patch to do large file reads as several > smaller reads. > Background here: > http://www.nabble.com/problems-with-large-Lucene-index-td22347854.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscribe@... For additional commands, e-mail: java-dev-help@... |
| < Prev | 1 - 2 | Next > |
| Free embeddable forum powered by Nabble | Forum Help |
