tag:old.nabble.com,2006:forum-24284Nabble - Lucene - Java2009-11-26T12:43:57Z<p>
<a href="http://lucene.apache.org/java/" target="_top" rel="nofollow">Lucene</a> is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-text search, especially cross-platform.
</p>
<p>
If you have questions about using the Lucene, please post them on the on the <a href="http://www.nabble.com/Lucene---Java-Users-f45.html" target="_top">java-users mailing list</a>.
</p>
<p>
(The <a href="http://www.nabble.com/Lucene---Java-Developer-f46.html" target="_top">java-dev mailing list</a> is for discussions about developing the internals of the Lucene library.)
</p>tag:old.nabble.com,2006:post-26534687Re: best way to ensure IndexWriter won't corrupt the index?2009-11-26T12:43:57Z2009-11-26T12:43:57ZMichael McCandless-2The leftover write.lock is expected on ungraceful shutdown; you just
<br>have to remove it. Or, switch to NativeFSLockFactory, which correctly
<br>detects when the lock is no longer in use.
<br><br>Next time this happens, try running CheckIndex on the index.
<br><br>Are you sure the Luke version you're using is new enough to read 2.4.1 indexes?
<br><br>Mike
<br><br>On Thu, Nov 26, 2009 at 3:35 PM, Khosro Asgharifard
<br><<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534687&i=0" target="_top" rel="nofollow">khosro_question@...</a>> wrote:
<div class='shrinkable-quote'><br>>>>Right, in 2.4, if you kill -9, pull power, OS crashes, etc., it should
<br>>
<br>>>>not corrupt the index.
<br>>
<br>>>>Can you share details on what corruption you see? Is it possible
<br>>>>there are two IndexWriters open on the index at once?
<br>>
<br>> Our apps is multithread,and sometimes when we shutdown the Tomcat ,
<br>> the wrike.lock file exist and does not disappear,and in the subsequent run,the write.lock does not allow
<br>> to index data,so that i must delete it by hand,and after that when i check the index
<br>> by Lukeall ,it tells that "There is no valid Lucene index in this directory".
<br>> And ,in another case,after showting down Tomcat, and running it again more than two times ,write.lock file does not allow to index the data,
<br>> and i must delete it by hand , and after that i check the index by Lukeall ,it tells that "There is no valid Lucene index in this directory".
<br>>
<br>> Khosro.
<br>>
<br>>
<br>>
<br>>>>Mike
<br>>
<br>> On Thu, Nov 26, 2009 at 2:08 PM, Khosro Asgharifard
<br>> <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534687&i=1" target="_top" rel="nofollow">khosro_question@...</a>> wrote:
<br>>> Hello,
<br>>>>>Before 2.4 it was possible that a crash of the OS, or sudden power
<br>>>
<br>>>>>loss to the machine, could corrupt the index. But that's been fixed
<br>>>>>with 2.4.
<br>>>
<br>>> Did you mean that Lucene does not have this issue in 2.4.1?
<br>>> We are running the program that index some data ,and sometime we must shutdown Tomcat,
<br>>> and in some case the index corrupt.This is a probelm in our program,and our data is too huge and
<br>>> we can not run reindeing all data agian.We use Lucene 2.4.0 with Compass.
<br>>>
<br>>> Best Regards
<br>>> Khosro.
<br>>>
<br>>>>>The only known sources of corruption are hardware faults (bad RAM, bad
<br>>>>>disk, etc.), and, accidentally allowing 2 writers to write to the same
<br>>>>>index at once (this will very quickly cause corruption). Lucene's
<br>>>>>write lock normally prevents this from happening.
<br>>>
<br>>>>>kill -9, JRE crashing, OS crashing, power loss, etc., should not cause
<br>>>>>corruption for Lucene >= 2.4.
<br>>>
<br>>>>>Backing up is definitely a good idea -- Lucene's
<br>>>>>SnapshotDeletionPolicy makes it easy to do a hot backup (backup even
<br>>>>>though IndexWriter is still changing the index). There's a paper on
<br>>>>>this (NOTE: I'm the author!) available at
<br>>>>><a href="http://www.manning.com/hatcher3/" target="_top" rel="nofollow">http://www.manning.com/hatcher3/</a> that gives details (look for "Hot
<br>>>>>backups with Lucene (green paper - html)".
<br>>>
<br>>> Mike
<br>>>
<br>>> On Wed, Nov 25, 2009 at 10:37 AM, Max Lynch <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534687&i=2" target="_top" rel="nofollow">ihasmax@...</a>> wrote:
<br>>>> On Wed, Nov 25, 2009 at 9:31 AM, Ian Lea <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534687&i=3" target="_top" rel="nofollow">ian.lea@...</a>> wrote:
<br>>>>
<br>>>>> > What are the typical scenarios when the index will go corrupt?
<br>>>>>
<br>>>>> Dodgy disks.
<br>>>>>
<br>>>>
<br>>>> I also have had index corruption on two occasions. It is not a big deal for
<br>>>> me since my data is fairly real time so the old documents aren't as
<br>>>> important.
<br>>>>
<br>>>> However, I'm running this on a VPS with slicehost, so whether or not they
<br>>>> use dodgy disks is not something I can confirm or even deal with.
<br>>>>
<br>>>> I do need to upgrade to 2.9 from 2.4, but I think one of the reasons for my
<br>>>> index corruption is deleting the index.write file rather than removing the
<br>>>> lock through the Lucene APIs. This seems like it could be a cause of
<br>>>> corruption, correct?
<br>>>>
<br>>>
<br>>> ---------------------------------------------------------------------
<br>>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534687&i=4" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534687&i=5" target="_top" rel="nofollow">java-user-help@...</a>
<br>>>
<br>>>
<br>>>
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534687&i=6" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534687&i=7" target="_top" rel="nofollow">java-user-help@...</a>
<br>>
<br>>
<br>>
</div><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534687&i=8" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534687&i=9" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26534622Re: best way to ensure IndexWriter won't corrupt the index?2009-11-26T12:35:43Z2009-11-26T12:35:43Zkhosro asgharifard>>Right, in 2.4, if you kill -9, pull power, OS crashes, etc., it should
<br><br>>>not corrupt the index.
<br><br>>>Can you share details on what corruption you see? Is it possible
<br>>>there are two IndexWriters open on the index at once?
<br><br>Our apps is multithread,and sometimes when we shutdown the Tomcat ,
<br>the wrike.lock file exist and does not disappear,and in the subsequent run,the write.lock does not allow
<br>to index data,so that i must delete it by hand,and after that when i check the index
<br>by Lukeall ,it tells that "There is no valid Lucene index in this directory".
<br>And ,in another case,after showting down Tomcat, and running it again more than two times ,write.lock file does not allow to index the data,
<br>and i must delete it by hand , and after that i check the index by Lukeall ,it tells that "There is no valid Lucene index in this directory".
<br><br>Khosro.
<br><br><br><br>>>Mike
<br><br>On Thu, Nov 26, 2009 at 2:08 PM, Khosro Asgharifard
<br><<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534622&i=0" target="_top" rel="nofollow">khosro_question@...</a>> wrote:
<div class='shrinkable-quote'><br>> Hello,
<br>>>>Before 2.4 it was possible that a crash of the OS, or sudden power
<br>>
<br>>>>loss to the machine, could corrupt the index. But that's been fixed
<br>>>>with 2.4.
<br>>
<br>> Did you mean that Lucene does not have this issue in 2.4.1?
<br>> We are running the program that index some data ,and sometime we must shutdown Tomcat,
<br>> and in some case the index corrupt.This is a probelm in our program,and our data is too huge and
<br>> we can not run reindeing all data agian.We use Lucene 2.4.0 with Compass.
<br>>
<br>> Best Regards
<br>> Khosro.
<br>>
<br>>>>The only known sources of corruption are hardware faults (bad RAM, bad
<br>>>>disk, etc.), and, accidentally allowing 2 writers to write to the same
<br>>>>index at once (this will very quickly cause corruption). Lucene's
<br>>>>write lock normally prevents this from happening.
<br>>
<br>>>>kill -9, JRE crashing, OS crashing, power loss, etc., should not cause
<br>>>>corruption for Lucene >= 2.4.
<br>>
<br>>>>Backing up is definitely a good idea -- Lucene's
<br>>>>SnapshotDeletionPolicy makes it easy to do a hot backup (backup even
<br>>>>though IndexWriter is still changing the index). There's a paper on
<br>>>>this (NOTE: I'm the author!) available at
<br>>>><a href="http://www.manning.com/hatcher3/" target="_top" rel="nofollow">http://www.manning.com/hatcher3/</a> that gives details (look for "Hot
<br>>>>backups with Lucene (green paper - html)".
<br>>
<br>> Mike
<br>>
<br>> On Wed, Nov 25, 2009 at 10:37 AM, Max Lynch <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534622&i=1" target="_top" rel="nofollow">ihasmax@...</a>> wrote:
<br>>> On Wed, Nov 25, 2009 at 9:31 AM, Ian Lea <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534622&i=2" target="_top" rel="nofollow">ian.lea@...</a>> wrote:
<br>>>
<br>>>> > What are the typical scenarios when the index will go corrupt?
<br>>>>
<br>>>> Dodgy disks.
<br>>>>
<br>>>
<br>>> I also have had index corruption on two occasions. It is not a big deal for
<br>>> me since my data is fairly real time so the old documents aren't as
<br>>> important.
<br>>>
<br>>> However, I'm running this on a VPS with slicehost, so whether or not they
<br>>> use dodgy disks is not something I can confirm or even deal with.
<br>>>
<br>>> I do need to upgrade to 2.9 from 2.4, but I think one of the reasons for my
<br>>> index corruption is deleting the index.write file rather than removing the
<br>>> lock through the Lucene APIs. This seems like it could be a cause of
<br>>> corruption, correct?
<br>>>
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534622&i=3" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534622&i=4" target="_top" rel="nofollow">java-user-help@...</a>
<br>>
<br>>
<br>>
</div><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534622&i=5" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26534622&i=6" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><br> <p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26533869Re: best way to ensure IndexWriter won't corrupt the index?2009-11-26T11:15:48Z2009-11-26T11:15:48ZMichael McCandless-2Right, in 2.4, if you kill -9, pull power, OS crashes, etc., it should
<br>not corrupt the index.
<br><br>Can you share details on what corruption you see? Is it possible
<br>there are two IndexWriters open on the index at once?
<br><br>Mike
<br><br>On Thu, Nov 26, 2009 at 2:08 PM, Khosro Asgharifard
<br><<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26533869&i=0" target="_top" rel="nofollow">khosro_question@...</a>> wrote:
<div class='shrinkable-quote'><br>> Hello,
<br>>>>Before 2.4 it was possible that a crash of the OS, or sudden power
<br>>
<br>>>>loss to the machine, could corrupt the index. But that's been fixed
<br>>>>with 2.4.
<br>>
<br>> Did you mean that Lucene does not have this issue in 2.4.1?
<br>> We are running the program that index some data ,and sometime we must shutdown Tomcat,
<br>> and in some case the index corrupt.This is a probelm in our program,and our data is too huge and
<br>> we can not run reindeing all data agian.We use Lucene 2.4.0 with Compass.
<br>>
<br>> Best Regards
<br>> Khosro.
<br>>
<br>>>>The only known sources of corruption are hardware faults (bad RAM, bad
<br>>>>disk, etc.), and, accidentally allowing 2 writers to write to the same
<br>>>>index at once (this will very quickly cause corruption). Lucene's
<br>>>>write lock normally prevents this from happening.
<br>>
<br>>>>kill -9, JRE crashing, OS crashing, power loss, etc., should not cause
<br>>>>corruption for Lucene >= 2.4.
<br>>
<br>>>>Backing up is definitely a good idea -- Lucene's
<br>>>>SnapshotDeletionPolicy makes it easy to do a hot backup (backup even
<br>>>>though IndexWriter is still changing the index). There's a paper on
<br>>>>this (NOTE: I'm the author!) available at
<br>>>><a href="http://www.manning.com/hatcher3/" target="_top" rel="nofollow">http://www.manning.com/hatcher3/</a> that gives details (look for "Hot
<br>>>>backups with Lucene (green paper - html)".
<br>>
<br>> Mike
<br>>
<br>> On Wed, Nov 25, 2009 at 10:37 AM, Max Lynch <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26533869&i=1" target="_top" rel="nofollow">ihasmax@...</a>> wrote:
<br>>> On Wed, Nov 25, 2009 at 9:31 AM, Ian Lea <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26533869&i=2" target="_top" rel="nofollow">ian.lea@...</a>> wrote:
<br>>>
<br>>>> > What are the typical scenarios when the index will go corrupt?
<br>>>>
<br>>>> Dodgy disks.
<br>>>>
<br>>>
<br>>> I also have had index corruption on two occasions. It is not a big deal for
<br>>> me since my data is fairly real time so the old documents aren't as
<br>>> important.
<br>>>
<br>>> However, I'm running this on a VPS with slicehost, so whether or not they
<br>>> use dodgy disks is not something I can confirm or even deal with.
<br>>>
<br>>> I do need to upgrade to 2.9 from 2.4, but I think one of the reasons for my
<br>>> index corruption is deleting the index.write file rather than removing the
<br>>> lock through the Lucene APIs. This seems like it could be a cause of
<br>>> corruption, correct?
<br>>>
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26533869&i=3" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26533869&i=4" target="_top" rel="nofollow">java-user-help@...</a>
<br>>
<br>>
<br>>
</div><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26533869&i=5" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26533869&i=6" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26533795Re: best way to ensure IndexWriter won't corrupt the index?2009-11-26T11:08:24Z2009-11-26T11:08:24Zkhosro asgharifardHello,
<br>>>Before 2.4 it was possible that a crash of the OS, or sudden power
<br><br>>>loss to the machine, could corrupt the index. But that's been fixed
<br>>>with 2.4.
<br><br>Did you mean that Lucene does not have this issue in 2.4.1?
<br>We are running the program that index some data ,and sometime we must shutdown Tomcat,
<br>and in some case the index corrupt.This is a probelm in our program,and our data is too huge and
<br>we can not run reindeing all data agian.We use Lucene 2.4.0 with Compass.
<br><br>Best Regards
<br>Khosro.
<br><br>>>The only known sources of corruption are hardware faults (bad RAM, bad
<br>>>disk, etc.), and, accidentally allowing 2 writers to write to the same
<br>>>index at once (this will very quickly cause corruption). Lucene's
<br>>>write lock normally prevents this from happening.
<br><br>>>kill -9, JRE crashing, OS crashing, power loss, etc., should not cause
<br>>>corruption for Lucene >= 2.4.
<br><br>>>Backing up is definitely a good idea -- Lucene's
<br>>>SnapshotDeletionPolicy makes it easy to do a hot backup (backup even
<br>>>though IndexWriter is still changing the index). There's a paper on
<br>>>this (NOTE: I'm the author!) available at
<br>>><a href="http://www.manning.com/hatcher3/" target="_top" rel="nofollow">http://www.manning.com/hatcher3/</a> that gives details (look for "Hot
<br>>>backups with Lucene (green paper - html)".
<br><br>Mike
<br><br>On Wed, Nov 25, 2009 at 10:37 AM, Max Lynch <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26533795&i=0" target="_top" rel="nofollow">ihasmax@...</a>> wrote:
<div class='shrinkable-quote'><br>> On Wed, Nov 25, 2009 at 9:31 AM, Ian Lea <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26533795&i=1" target="_top" rel="nofollow">ian.lea@...</a>> wrote:
<br>>
<br>>> > What are the typical scenarios when the index will go corrupt?
<br>>>
<br>>> Dodgy disks.
<br>>>
<br>>
<br>> I also have had index corruption on two occasions. It is not a big deal for
<br>> me since my data is fairly real time so the old documents aren't as
<br>> important.
<br>>
<br>> However, I'm running this on a VPS with slicehost, so whether or not they
<br>> use dodgy disks is not something I can confirm or even deal with.
<br>>
<br>> I do need to upgrade to 2.9 from 2.4, but I think one of the reasons for my
<br>> index corruption is deleting the index.write file rather than removing the
<br>> lock through the Lucene APIs. This seems like it could be a cause of
<br>> corruption, correct?
<br>>
</div><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26533795&i=2" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26533795&i=3" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><br> <p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26531193Re: (LUCENE-1844) Speed up junit tests2009-11-26T07:38:05Z2009-11-26T07:38:05ZErick EricksonDespite my long rambling, I agree that speeding things up is worthwhile. Just<br>not a huge deal for some of us poor peons who are on dinky little 2-core<br>machines and feel inadequate even *talking* to people who have *real*<br>
machines <G>...<br><br>Time to go get ready to eat Turkey....<br><br>Erick<br><br><div class="gmail_quote">On Thu, Nov 26, 2009 at 9:02 AM, Mark Miller <span dir="ltr"><<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=0" target="_top" rel="nofollow">markrmiller@...</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> right - as soon as you have to start running the tests often enough, any decent savings turns into less waiting and more work. Waiting for tests to run is time that could be better spent elsewhere. And many of us runthe tests *a lot* considering how long they take. And we will only keep adding more and will continue to do so.<br>
<br>
Also, many of us *are* on multicore and should be able to benifit from it. I don't dev on anything less than 4 cores these days. It's a life changer :) and cheap currently. I'd like 8.<br>
<br>
- Mark<br>
<br>
<a href="http://www.lucidimagination.com" target="_blank" rel="nofollow">http://www.lucidimagination.com</a> (mobile)<div><div></div><div class="h5"><br>
<br>
On Nov 26, 2009, at 5:24 AM, Michael McCandless <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=1" target="_top" rel="nofollow">lucene@...</a>> wrote:<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I still think there's value to faster tests, even if they don't become<br>
so fast as to enable "fully interactive testing".<br>
<br>
Plus, this is an ongoing goal with time, not a one-time event. As we<br>
create tests we should generally try to maximize coverage and minimize<br>
CPU cost, as long as the effort is smallish.<br>
<br>
Mike<br>
<br>
On Wed, Nov 25, 2009 at 9:32 PM, Erick Erickson <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=2" target="_top" rel="nofollow">erickerickson@...</a>> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I posted a rather long diatribe outlining why I think speed-ups<br>
are a false goal for Lucene. Briefly, I'm convinced that as long<br>
as the tests are run when Hudson builds Lucene, 99% of the<br>
value of unit tests is realized. I suppose this implies that the<br>
hard-core committers agree that as long as failed tests<br>
are caught/corrected within a day things are fine.<br>
<br>
Although coming from a background where unit<br>
tests are not always required, my viewpoint may be<br>
suspect <G>.<br>
<br>
<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=3" target="_top" rel="nofollow">Erick@...</a>....<br>
<br>
On Wed, Nov 25, 2009 at 8:43 PM, Michael McCandless (JIRA)<br>
<<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=4" target="_top" rel="nofollow">jira@...</a>>wrote:<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
[<br>
<a href="https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782716#action_12782716" target="_blank" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782716#action_12782716</a>]<br>
<br>
Michael McCandless commented on LUCENE-1844:<br>
--------------------------------------------<br>
<br>
Will we also speed up back-compat tests?<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Speed up junit tests<br>
--------------------<br>
<br>
Key: LUCENE-1844<br>
URL: <a href="https://issues.apache.org/jira/browse/LUCENE-1844" target="_blank" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844</a><br>
Project: Lucene - Java<br>
Issue Type: Improvement<br>
Reporter: Mark Miller<br>
Attachments: FastCnstScoreQTest.patch,<br>
</blockquote>
hi_junit_test_runtimes.png, LUCENE-1844.patch<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
<br>
As Lucene grows, so does the number of JUnit tests. This is obviously a<br>
</blockquote>
good thing, but it comes with longer and longer test times. Now that we also<br>
run back compat tests in a standard test run, this problem is essentially<br>
doubled.<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
There are some ways this may get better, including running parallel<br>
</blockquote>
tests. You will need the hardware to fully take advantage, but it should be<br>
a nice gain. There is already an issue for this, and Junit 4.6, 4.7 have the<br>
beginnings of something we might be able to count on soon. 4.6 was buggy,<br>
and 4.7 still doesn't come with nice ant integration. Parallel tests will<br>
come though.<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Beyond parallel testing, I think we also need to concentrate on keeping<br>
</blockquote>
our tests lean. We don't want to sacrifice coverage or quality, but I'm sure<br>
there is plenty of fat to skim.<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I've started making a list of some of the longer tests - I think with<br>
</blockquote>
some work we can make our tests much faster - and then with parallelization,<br>
I think we could see some really great gains.<br>
<br>
--<br>
This message is automatically generated by JIRA.<br>
-<br>
You can reply to this email to add a comment to the issue online.<br>
<br>
<br>
---------------------------------------------------------------------<br>
To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=5" target="_top" rel="nofollow">java-dev-unsubscribe@...</a><br>
For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=6" target="_top" rel="nofollow">java-dev-help@...</a><br>
<br>
<br>
</blockquote>
<br>
I posted a rather long diatribe outlining why I think speed-ups<br>
are a false goal for Lucene. Briefly, I'm convinced that as long<br>
as the tests are run when Hudson builds Lucene, 99% of the<br>
value of unit tests is realized. I suppose this implies that the<br>
hard-core committers agree that as long as failed tests<br>
are caught/corrected within a day things are fine.<br>
<br>
Although coming from a background where unit<br>
tests are not always required, my viewpoint may be<br>
suspect <G>.<br>
<br>
<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=7" target="_top" rel="nofollow">Erick@...</a>....<br>
<br>
On Wed, Nov 25, 2009 at 8:43 PM, Michael McCandless (JIRA) <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=8" target="_top" rel="nofollow">jira@...</a>> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
[ <a href="https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782716#action_12782716" target="_blank" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782716#action_12782716</a> ]<br>
<br>
Michael McCandless commented on LUCENE-1844:<br>
--------------------------------------------<br>
<br>
Will we also speed up back-compat tests?<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Speed up junit tests<br>
--------------------<br>
<br>
Key: LUCENE-1844<br>
URL: <a href="https://issues.apache.org/jira/browse/LUCENE-1844" target="_blank" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844</a><br>
Project: Lucene - Java<br>
Issue Type: Improvement<br>
Reporter: Mark Miller<br>
Attachments: FastCnstScoreQTest.patch, hi_junit_test_runtimes.png, LUCENE-1844.patch<br>
<br>
<br>
As Lucene grows, so does the number of JUnit tests. This is obviously a good thing, but it comes with longer and longer test times. Now that we also run back compat tests in a standard test run, this problem is essentially doubled.<br>
There are some ways this may get better, including running parallel tests. You will need the hardware to fully take advantage, but it should be a nice gain. There is already an issue for this, and Junit 4.6, 4.7 have the beginnings of something we might be able to count on soon. 4.6 was buggy, and 4.7 still doesn't come with nice ant integration. Parallel tests will come though.<br>
Beyond parallel testing, I think we also need to concentrate on keeping our tests lean. We don't want to sacrifice coverage or quality, but I'm sure there is plenty of fat to skim.<br>
I've started making a list of some of the longer tests - I think with some work we can make our tests much faster - and then with parallelization, I think we could see some really great gains.<br>
</blockquote>
<br>
--<br>
This message is automatically generated by JIRA.<br>
-<br>
You can reply to this email to add a comment to the issue online.<br>
<br>
<br>
---------------------------------------------------------------------<br>
To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=9" target="_top" rel="nofollow">java-dev-unsubscribe@...</a><br>
For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=10" target="_top" rel="nofollow">java-dev-help@...</a><br>
<br>
</blockquote>
<br>
<br>
<br>
</blockquote>
<br>
---------------------------------------------------------------------<br>
To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=11" target="_top" rel="nofollow">java-dev-unsubscribe@...</a><br>
For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=12" target="_top" rel="nofollow">java-dev-help@...</a><br>
<br>
</blockquote>
<br>
---------------------------------------------------------------------<br>
To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=13" target="_top" rel="nofollow">java-dev-unsubscribe@...</a><br>
For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26531193&i=14" target="_top" rel="nofollow">java-dev-help@...</a><br>
<br>
</div></div></blockquote></div><br>
<p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26530833[jira] Commented: (LUCENE-2037) Allow Junit4 tests in our environment.2009-11-26T07:14:39Z2009-11-26T07:14:39ZJIRA jira@apache.org<br> [ <a href="https://issues.apache.org/jira/browse/LUCENE-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782916#action_12782916" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782916#action_12782916</a> ]
<br><br>Erick Erickson commented on LUCENE-2037:
<br>----------------------------------------
<br><br>Hold off on this patch until I get a chance to submit a new one, we're straightening out LUCENE-1844 interdependencies between patches.....
<br><div class='shrinkable-quote'><br>> Allow Junit4 tests in our environment.
<br>> --------------------------------------
<br>>
<br>> Key: LUCENE-2037
<br>> URL: <a href="https://issues.apache.org/jira/browse/LUCENE-2037" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-2037</a><br>> Project: Lucene - Java
<br>> Issue Type: Improvement
<br>> Components: Other
<br>> Affects Versions: 3.1
<br>> Environment: Development
<br>> Reporter: Erick Erickson
<br>> Assignee: Erick Erickson
<br>> Priority: Minor
<br>> Fix For: 3.1
<br>>
<br>> Attachments: junit-4.7.jar, LUCENE-2037.patch
<br>>
<br>> Original Estimate: 8h
<br>> Remaining Estimate: 8h
<br>>
<br>> Now that we're dropping Java 1.4 compatibility for 3.0, we can incorporate Junit4 in testing. Junit3 and junit4 tests can coexist, so no tests should have to be rewritten. We should start this for the 3.1 release so we can get a clean 3.0 out smoothly.
<br>> It's probably worthwhile to convert a small set of tests as an exemplar.
</div><br>--
<br>This message is automatically generated by JIRA.
<br>-
<br>You can reply to this email to add a comment to the issue online.
<br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26530833&i=0" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26530833&i=1" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26530797[jira] Commented: (LUCENE-1844) Speed up junit tests2009-11-26T07:12:39Z2009-11-26T07:12:39ZJIRA jira@apache.org<br> [ <a href="https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782915#action_12782915" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782915#action_12782915</a> ]
<br><br>Erick Erickson commented on LUCENE-1844:
<br>----------------------------------------
<br><br>OK, fire when ready Gridley. Pretty soon I'll understand when to comment and how to keep from multiple comments....
<br><br>This patch does NOT use the Java5 features like generics etc. I've done that work and it'll be included in the TestCustomScoreQuery changes for JUnit4.....
<br><br><br><div class='shrinkable-quote'><br>> Speed up junit tests
<br>> --------------------
<br>>
<br>> Key: LUCENE-1844
<br>> URL: <a href="https://issues.apache.org/jira/browse/LUCENE-1844" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844</a><br>> Project: Lucene - Java
<br>> Issue Type: Improvement
<br>> Reporter: Mark Miller
<br>> Attachments: FastCnstScoreQTest.patch, hi_junit_test_runtimes.png, LUCENE-1844-Junit3.patch, LUCENE-1844.patch
<br>>
<br>>
<br>> As Lucene grows, so does the number of JUnit tests. This is obviously a good thing, but it comes with longer and longer test times. Now that we also run back compat tests in a standard test run, this problem is essentially doubled.
<br>> There are some ways this may get better, including running parallel tests. You will need the hardware to fully take advantage, but it should be a nice gain. There is already an issue for this, and Junit 4.6, 4.7 have the beginnings of something we might be able to count on soon. 4.6 was buggy, and 4.7 still doesn't come with nice ant integration. Parallel tests will come though.
<br>> Beyond parallel testing, I think we also need to concentrate on keeping our tests lean. We don't want to sacrifice coverage or quality, but I'm sure there is plenty of fat to skim.
<br>> I've started making a list of some of the longer tests - I think with some work we can make our tests much faster - and then with parallelization, I think we could see some really great gains.
</div><br>--
<br>This message is automatically generated by JIRA.
<br>-
<br>You can reply to this email to add a comment to the issue online.
<br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26530797&i=0" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26530797&i=1" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26530749[jira] Updated: (LUCENE-1844) Speed up junit tests2009-11-26T07:08:39Z2009-11-26T07:08:39ZJIRA jira@apache.org<br> [ <a href="https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel</a> ]
<br><br>Erick Erickson updated LUCENE-1844:
<br>-----------------------------------
<br><br> Attachment: LUCENE-1844-Junit3.patch
<br><br>Speeds up TestBooleanMinShouldMatch and TestCustomScoreQuery without using JUnit4
<br><div class='shrinkable-quote'><br>> Speed up junit tests
<br>> --------------------
<br>>
<br>> Key: LUCENE-1844
<br>> URL: <a href="https://issues.apache.org/jira/browse/LUCENE-1844" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844</a><br>> Project: Lucene - Java
<br>> Issue Type: Improvement
<br>> Reporter: Mark Miller
<br>> Attachments: FastCnstScoreQTest.patch, hi_junit_test_runtimes.png, LUCENE-1844-Junit3.patch, LUCENE-1844.patch
<br>>
<br>>
<br>> As Lucene grows, so does the number of JUnit tests. This is obviously a good thing, but it comes with longer and longer test times. Now that we also run back compat tests in a standard test run, this problem is essentially doubled.
<br>> There are some ways this may get better, including running parallel tests. You will need the hardware to fully take advantage, but it should be a nice gain. There is already an issue for this, and Junit 4.6, 4.7 have the beginnings of something we might be able to count on soon. 4.6 was buggy, and 4.7 still doesn't come with nice ant integration. Parallel tests will come though.
<br>> Beyond parallel testing, I think we also need to concentrate on keeping our tests lean. We don't want to sacrifice coverage or quality, but I'm sure there is plenty of fat to skim.
<br>> I've started making a list of some of the longer tests - I think with some work we can make our tests much faster - and then with parallelization, I think we could see some really great gains.
</div><br>--
<br>This message is automatically generated by JIRA.
<br>-
<br>You can reply to this email to add a comment to the issue online.
<br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26530749&i=0" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26530749&i=1" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26530517Re: Inside Lucene2009-11-26T06:50:52Z2009-11-26T06:50:52ZBenjamin HeilbrunnHello,
<br><br>if you are searching for information about lucenes file structure you
<br>can find something here:
<br><a href="http://lucene.apache.org/java/3_0_0/fileformats.html" target="_top" rel="nofollow">http://lucene.apache.org/java/3_0_0/fileformats.html</a><br><br><br>Benjamin
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26530517&i=0" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26530517&i=1" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26530361[jira] Commented: (LUCENE-1930) Scale moderator not in line with sinusoidal projector2009-11-26T06:40:39Z2009-11-26T06:40:39ZJIRA jira@apache.org<br> [ <a href="https://issues.apache.org/jira/browse/LUCENE-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782908#action_12782908" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782908#action_12782908</a> ]
<br><br>Nicolas Helleringer commented on LUCENE-1930:
<br>---------------------------------------------
<br><br><br>Ok ! I did not get that =)
<br><br><br>Of course : add a comment to the issue to tell the project.
<br><br>Best regards,
<br><br>Nicolas
<br><br><br><br><div class='shrinkable-quote'><br>> Scale moderator not in line with sinusoidal projector
<br>> -----------------------------------------------------
<br>>
<br>> Key: LUCENE-1930
<br>> URL: <a href="https://issues.apache.org/jira/browse/LUCENE-1930" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1930</a><br>> Project: Lucene - Java
<br>> Issue Type: Bug
<br>> Components: contrib/spatial
<br>> Affects Versions: 2.9
<br>> Reporter: Nicolas Helleringer
<br>> Attachments: LUCENE-1930.patch
<br>>
<br>>
<br>> Current implementation in spatial Lucene does :
<br>> public double getTierBoxId (double latitude, double longitude) {
<br>> double[] coords = projector.coords(latitude, longitude);
<br>> double id = getBoxId(coords[0]) + (getBoxId(coords[1]) / tierVerticalPosDivider);
<br>> return id ;
<br>> }
<br>> private double getBoxId (double coord){
<br>> return Math.floor(coord / (idd / this.tierLength));
<br>> }
<br>> with
<br>> Double idd = new Double(180);
<br>> in the CartesianTierPlotter constructor.
<br>> But current Sinusoidal Projector set and used in initialisation of CartesianTierPlotter does :
<br>> public double[] coords(double latitude, double longitude) {
<br>> double rlat = Math.toRadians(latitude);
<br>> double rlong = Math.toRadians(longitude);
<br>> double nlat = rlong * Math.cos(rlat);
<br>> double r[] = {nlat, rlong};
<br>> return r;
<br>> }
<br>> Thus we moderate with idd = 180 a coord that is in a Radian space.
<br>> Things to do :
<br>> 1°) fix idd to : double idd= PI;
<br>> 2°) Move idd definition to IProjector interface : The coord space should belong to the projector doing the job. Change the code from CTP to use that new interface.
</div><br>--
<br>This message is automatically generated by JIRA.
<br>-
<br>You can reply to this email to add a comment to the issue online.
<br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26530361&i=0" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26530361&i=1" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26530211Inside Lucene2009-11-26T06:27:46Z2009-11-26T06:27:46Z王巍巍Hi guys,
<br><br> I’ve read Lucene in Action and also implemented a general purpose FTP
<br>search engine with lucene, you can find the source here:
<br><a href="http://code.google.com/p/askrosa/" target="_top" rel="nofollow">http://code.google.com/p/askrosa/</a><br><br>I want to learn more about the implementation of lucene, e.g. index ,storing
<br>structure, compression etc.
<br><br>Could anyone here recommend some books or materials about these topics?
<br><br><br><br><br><br>Best Regards!
<br><br><br><br><br><br>Weiwei Wang
<br>Nanjing University, Nanjing, P.R.China
<br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26530165Books about lucene2009-11-26T06:24:43Z2009-11-26T06:24:43Z王巍巍Hi guys,
<br><br> I’ve read Lucene in Action and also implemented a general purpose FTP
<br>search engine with lucene, you can find the source here: <a href="http://code.google" target="_top" rel="nofollow">http://code.google</a>.
<br>com/p/askrosa/
<br><br>I want to learn more about the implementation of lucene, e.g. index storing
<br>structure, compression etc.
<br><br>Could anyone here recommend some books about these topics?
<br><br>
<br><br>
<br><br>Best Regards!
<br><br>
<br><br>
<br><br>Weiwei Wang
<br><br>Nanjing University, Nanjing, P.R.China
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26530171Re: [jira] Commented: (LUCENE-1844) Speed up junit tests2009-11-26T06:24:07Z2009-11-26T06:24:07Zmarkrmiller<br><br><br><br>On Nov 25, 2009, at 9:32 PM, Erick Erickson <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26530171&i=0" target="_top" rel="nofollow">erickerickson@...</a>>
<br>wrote:
<br>> I suppose this implies that the
<br>> hard-core committers agree that as long as failed tests
<br>> are caught/corrected within a day things are fine.
<br><br>Personally, I don't agree with that. Others are checking out,
<br>updating, and using the tests to help verify and figure out their
<br>changes. Breaks in the build are detrimental to this processes. If it
<br>happens on the rare occassion fine, but you should run the the tests
<br>before committing - it's being polite to others working with the
<br>build. Don't break the build is a rule I'm strongly behind - and it
<br>requires running the tests outside of Hudson.
<br><br>-mark
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26530171&i=1" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26530171&i=2" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26529885Re: (LUCENE-1844) Speed up junit tests2009-11-26T06:02:25Z2009-11-26T06:02:25Zmarkrmiller right - as soon as you have to start running the tests often enough,
<br>any decent savings turns into less waiting and more work. Waiting for
<br>tests to run is time that could be better spent elsewhere. And many of
<br>us runthe tests *a lot* considering how long they take. And we will
<br>only keep adding more and will continue to do so.
<br><br>Also, many of us *are* on multicore and should be able to benifit from
<br>it. I don't dev on anything less than 4 cores these days. It's a life
<br>changer :) and cheap currently. I'd like 8.
<br><br>- Mark
<br><br><a href="http://www.lucidimagination.com" target="_top" rel="nofollow">http://www.lucidimagination.com</a> (mobile)
<br><br>On Nov 26, 2009, at 5:24 AM, Michael McCandless <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=0" target="_top" rel="nofollow">lucene@...</a>
<br> > wrote:
<br><div class='shrinkable-quote'><br>> I still think there's value to faster tests, even if they don't become
<br>> so fast as to enable "fully interactive testing".
<br>>
<br>> Plus, this is an ongoing goal with time, not a one-time event. As we
<br>> create tests we should generally try to maximize coverage and minimize
<br>> CPU cost, as long as the effort is smallish.
<br>>
<br>> Mike
<br>>
<br>> On Wed, Nov 25, 2009 at 9:32 PM, Erick Erickson <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=1" target="_top" rel="nofollow">erickerickson@...</a>
<br>> > wrote:
<br>>> I posted a rather long diatribe outlining why I think speed-ups
<br>>> are a false goal for Lucene. Briefly, I'm convinced that as long
<br>>> as the tests are run when Hudson builds Lucene, 99% of the
<br>>> value of unit tests is realized. I suppose this implies that the
<br>>> hard-core committers agree that as long as failed tests
<br>>> are caught/corrected within a day things are fine.
<br>>>
<br>>> Although coming from a background where unit
<br>>> tests are not always required, my viewpoint may be
<br>>> suspect <G>.
<br>>>
<br>>> <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=2" target="_top" rel="nofollow">Erick@...</a>....
<br>>>
<br>>> On Wed, Nov 25, 2009 at 8:43 PM, Michael McCandless (JIRA)
<br>>> <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=3" target="_top" rel="nofollow">jira@...</a>>wrote:
<br>>>
<br>>>>
<br>>>> [
<br>>>> <a href="https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782716#action_12782716" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782716#action_12782716</a>]
<br>>>>
<br>>>> Michael McCandless commented on LUCENE-1844:
<br>>>> --------------------------------------------
<br>>>>
<br>>>> Will we also speed up back-compat tests?
<br>>>>
<br>>>>> Speed up junit tests
<br>>>>> --------------------
<br>>>>>
<br>>>>> Key: LUCENE-1844
<br>>>>> URL: <a href="https://issues.apache.org/jira/browse/LUCENE-1844" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844</a><br>>>>> Project: Lucene - Java
<br>>>>> Issue Type: Improvement
<br>>>>> Reporter: Mark Miller
<br>>>>> Attachments: FastCnstScoreQTest.patch,
<br>>>> hi_junit_test_runtimes.png, LUCENE-1844.patch
<br>>>>>
<br>>>>>
<br>>>>> As Lucene grows, so does the number of JUnit tests. This is
<br>>>>> obviously a
<br>>>> good thing, but it comes with longer and longer test times. Now
<br>>>> that we also
<br>>>> run back compat tests in a standard test run, this problem is
<br>>>> essentially
<br>>>> doubled.
<br>>>>> There are some ways this may get better, including running parallel
<br>>>> tests. You will need the hardware to fully take advantage, but it
<br>>>> should be
<br>>>> a nice gain. There is already an issue for this, and Junit 4.6,
<br>>>> 4.7 have the
<br>>>> beginnings of something we might be able to count on soon. 4.6 was
<br>>>> buggy,
<br>>>> and 4.7 still doesn't come with nice ant integration. Parallel
<br>>>> tests will
<br>>>> come though.
<br>>>>> Beyond parallel testing, I think we also need to concentrate on
<br>>>>> keeping
<br>>>> our tests lean. We don't want to sacrifice coverage or quality,
<br>>>> but I'm sure
<br>>>> there is plenty of fat to skim.
<br>>>>> I've started making a list of some of the longer tests - I think
<br>>>>> with
<br>>>> some work we can make our tests much faster - and then with
<br>>>> parallelization,
<br>>>> I think we could see some really great gains.
<br>>>>
<br>>>> --
<br>>>> This message is automatically generated by JIRA.
<br>>>> -
<br>>>> You can reply to this email to add a comment to the issue online.
<br>>>>
<br>>>>
<br>>>> ---
<br>>>> ------------------------------------------------------------------
<br>>>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=4" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>>>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=5" target="_top" rel="nofollow">java-dev-help@...</a>
<br>>>>
<br>>>>
<br>>>
<br>>> I posted a rather long diatribe outlining why I think speed-ups
<br>>> are a false goal for Lucene. Briefly, I'm convinced that as long
<br>>> as the tests are run when Hudson builds Lucene, 99% of the
<br>>> value of unit tests is realized. I suppose this implies that the
<br>>> hard-core committers agree that as long as failed tests
<br>>> are caught/corrected within a day things are fine.
<br>>>
<br>>> Although coming from a background where unit
<br>>> tests are not always required, my viewpoint may be
<br>>> suspect <G>.
<br>>>
<br>>> <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=6" target="_top" rel="nofollow">Erick@...</a>....
<br>>>
<br>>> On Wed, Nov 25, 2009 at 8:43 PM, Michael McCandless (JIRA) <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=7" target="_top" rel="nofollow">jira@...</a>
<br>>> > wrote:
<br>>>>
<br>>>> [ <a href="https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782716#action_12782716" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782716#action_12782716</a>
<br>>>> ]
<br>>>>
<br>>>> Michael McCandless commented on LUCENE-1844:
<br>>>> --------------------------------------------
<br>>>>
<br>>>> Will we also speed up back-compat tests?
<br>>>>
<br>>>>> Speed up junit tests
<br>>>>> --------------------
<br>>>>>
<br>>>>> Key: LUCENE-1844
<br>>>>> URL: <a href="https://issues.apache.org/jira/browse/LUCENE-1844" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844</a><br>>>>> Project: Lucene - Java
<br>>>>> Issue Type: Improvement
<br>>>>> Reporter: Mark Miller
<br>>>>> Attachments: FastCnstScoreQTest.patch,
<br>>>>> hi_junit_test_runtimes.png, LUCENE-1844.patch
<br>>>>>
<br>>>>>
<br>>>>> As Lucene grows, so does the number of JUnit tests. This is
<br>>>>> obviously a good thing, but it comes with longer and longer test
<br>>>>> times. Now that we also run back compat tests in a standard test
<br>>>>> run, this problem is essentially doubled.
<br>>>>> There are some ways this may get better, including running
<br>>>>> parallel tests. You will need the hardware to fully take
<br>>>>> advantage, but it should be a nice gain. There is already an
<br>>>>> issue for this, and Junit 4.6, 4.7 have the beginnings of
<br>>>>> something we might be able to count on soon. 4.6 was buggy, and
<br>>>>> 4.7 still doesn't come with nice ant integration. Parallel tests
<br>>>>> will come though.
<br>>>>> Beyond parallel testing, I think we also need to concentrate on
<br>>>>> keeping our tests lean. We don't want to sacrifice coverage or
<br>>>>> quality, but I'm sure there is plenty of fat to skim.
<br>>>>> I've started making a list of some of the longer tests - I think
<br>>>>> with some work we can make our tests much faster - and then with
<br>>>>> parallelization, I think we could see some really great gains.
<br>>>>
<br>>>> --
<br>>>> This message is automatically generated by JIRA.
<br>>>> -
<br>>>> You can reply to this email to add a comment to the issue online.
<br>>>>
<br>>>>
<br>>>> ---
<br>>>> ------------------------------------------------------------------
<br>>>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=8" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>>>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=9" target="_top" rel="nofollow">java-dev-help@...</a>
<br>>>>
<br>>>
<br>>>
<br>>>
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=10" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=11" target="_top" rel="nofollow">java-dev-help@...</a>
<br>>
</div><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=12" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529885&i=13" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26529612[jira] Commented: (LUCENE-1930) Scale moderator not in line with sinusoidal projector2009-11-26T05:43:39Z2009-11-26T05:43:39ZJIRA jira@apache.org<br> [ <a href="https://issues.apache.org/jira/browse/LUCENE-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782896#action_12782896" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782896#action_12782896</a> ]
<br><br>Chris Male commented on LUCENE-1930:
<br>------------------------------------
<br><br>Hi Nicolas,
<br><br>I have every intention to commit my work to Lucene in the next week or so. If you can wait till then, I will include this patch in my patch.
<br><br>Thanks
<br><div class='shrinkable-quote'><br>> Scale moderator not in line with sinusoidal projector
<br>> -----------------------------------------------------
<br>>
<br>> Key: LUCENE-1930
<br>> URL: <a href="https://issues.apache.org/jira/browse/LUCENE-1930" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1930</a><br>> Project: Lucene - Java
<br>> Issue Type: Bug
<br>> Components: contrib/spatial
<br>> Affects Versions: 2.9
<br>> Reporter: Nicolas Helleringer
<br>> Attachments: LUCENE-1930.patch
<br>>
<br>>
<br>> Current implementation in spatial Lucene does :
<br>> public double getTierBoxId (double latitude, double longitude) {
<br>> double[] coords = projector.coords(latitude, longitude);
<br>> double id = getBoxId(coords[0]) + (getBoxId(coords[1]) / tierVerticalPosDivider);
<br>> return id ;
<br>> }
<br>> private double getBoxId (double coord){
<br>> return Math.floor(coord / (idd / this.tierLength));
<br>> }
<br>> with
<br>> Double idd = new Double(180);
<br>> in the CartesianTierPlotter constructor.
<br>> But current Sinusoidal Projector set and used in initialisation of CartesianTierPlotter does :
<br>> public double[] coords(double latitude, double longitude) {
<br>> double rlat = Math.toRadians(latitude);
<br>> double rlong = Math.toRadians(longitude);
<br>> double nlat = rlong * Math.cos(rlat);
<br>> double r[] = {nlat, rlong};
<br>> return r;
<br>> }
<br>> Thus we moderate with idd = 180 a coord that is in a Radian space.
<br>> Things to do :
<br>> 1°) fix idd to : double idd= PI;
<br>> 2°) Move idd definition to IProjector interface : The coord space should belong to the projector doing the job. Change the code from CTP to use that new interface.
</div><br>--
<br>This message is automatically generated by JIRA.
<br>-
<br>You can reply to this email to add a comment to the issue online.
<br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529612&i=0" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529612&i=1" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26529595RE: Is it a lucene bug?2009-11-26T05:41:49Z2009-11-26T05:41:49ZUwe SchindlerRead the documentation of the Document class: if you set a boost for a
<br>document, it is used when indexing the fields and multiplied to each field.
<br>For the document itself no boost value is stored, so you cannot get it (only
<br>so called stored fields are retrievable).
<br><br>-----
<br>Uwe Schindler
<br>H.-H.-Meier-Allee 63, D-28213 Bremen
<br><a href="http://www.thetaphi.de" target="_top" rel="nofollow">http://www.thetaphi.de</a><br>eMail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529595&i=0" target="_top" rel="nofollow">uwe@...</a>
<br><br><div class='shrinkable-quote'><br>> -----Original Message-----
<br>> From: Wilson Wu [mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529595&i=1" target="_top" rel="nofollow">songzi0206@...</a>]
<br>> Sent: Thursday, November 26, 2009 1:01 PM
<br>> To: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529595&i=2" target="_top" rel="nofollow">java-user@...</a>
<br>> Subject: Is it a lucene bug?
<br>>
<br>> Hi,
<br>> Recently, there is a requirement to sort the hits by both the
<br>> scores of documents and the updateTime which is a field of document to
<br>> mark the document's update time. We want the new document in the
<br>> front and also want high score document in the front,in other words,
<br>> we want to mix the score and updateTime, but not first sort by
<br>> one,second by the other. So, I design a time based function f(t) to
<br>> calculte each document boost to write into the index store.
<br>> The result is that I can caculate a value for each document
<br>> based its update time, and the value can influence the document score
<br>> through adjusting the fieldNorm value. But when I lookup the boost
<br>> value through the method document.getBoost() from every document in
<br>> the index store, I found the boost value = 1.0. Which means I can set
<br>> a document's boost value and the boost value can adjust the final
<br>> score, but I can't read the boost value from the document I have
<br>> searched out.
<br>> Is it a bug in lucene? Thanks. I use lucene version 2.4.1.
<br>> PS: Is there any other way to meet my reqirement? I think it is
<br>> not a good idea to adjust document's final score through writing a
<br>> document boost into the index store. Because if I want to open two
<br>> interfaces to the Client: one is sorting documents only by score which
<br>> is just the similarity score and has not been adjusted by boost value
<br>> f(t), the other is sorting by final score which has been adjuested by
<br>> boost value f(t). Thank a lot!
<br>>
<br>> wilson
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529595&i=3" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529595&i=4" target="_top" rel="nofollow">java-user-help@...</a>
</div><br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529595&i=5" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529595&i=6" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26529392Re: Is it a lucene bug?2009-11-26T05:24:48Z2009-11-26T05:24:48ZSavvas-Andreas Moysidishi,
<br><br><br><br>I’m not exactly sure I understand they the type of sorting you are trying to
<br>achieve.
<br><br>You have an updateTime field and you mention that "We want the new document
<br>in the
<br>front and also want high score document in the front".
<br><br>My take on this is that you want to first sort by the updateTime and then by
<br>score but you say this is not the case?
<br><br><br>Instead of calculating a boost value with f(t) can you not calculate and
<br>index the actual value you need for every document?
<br><br>Then you can first sort by this value and then by score?
<br><br><br><br>regards,
<br><br>savvas
<br><br><br>2009/11/26 Wilson Wu <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529392&i=0" target="_top" rel="nofollow">songzi0206@...</a>>
<br><div class='shrinkable-quote'><br>> Hi,
<br>> Recently, there is a requirement to sort the hits by both the
<br>> scores of documents and the updateTime which is a field of document to
<br>> mark the document's update time. We want the new document in the
<br>> front and also want high score document in the front,in other words,
<br>> we want to mix the score and updateTime, but not first sort by
<br>> one,second by the other. So, I design a time based function f(t) to
<br>> calculte each document boost to write into the index store.
<br>> The result is that I can caculate a value for each document
<br>> based its update time, and the value can influence the document score
<br>> through adjusting the fieldNorm value. But when I lookup the boost
<br>> value through the method document.getBoost() from every document in
<br>> the index store, I found the boost value = 1.0. Which means I can set
<br>> a document's boost value and the boost value can adjust the final
<br>> score, but I can't read the boost value from the document I have
<br>> searched out.
<br>> Is it a bug in lucene? Thanks. I use lucene version 2.4.1.
<br>> PS: Is there any other way to meet my reqirement? I think it is
<br>> not a good idea to adjust document's final score through writing a
<br>> document boost into the index store. Because if I want to open two
<br>> interfaces to the Client: one is sorting documents only by score which
<br>> is just the similarity score and has not been adjusted by boost value
<br>> f(t), the other is sorting by final score which has been adjuested by
<br>> boost value f(t). Thank a lot!
<br>>
<br>> wilson
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529392&i=1" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26529392&i=2" target="_top" rel="nofollow">java-user-help@...</a>
<br>>
<br>>
<br></div><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26528733[jira] Commented: (LUCENE-1934) Rework (Float)LatLng implementation and distance calculation2009-11-26T04:28:40Z2009-11-26T04:28:40ZJIRA jira@apache.org<br> [ <a href="https://issues.apache.org/jira/browse/LUCENE-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782873#action_12782873" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782873#action_12782873</a> ]
<br><br>Nicolas Helleringer commented on LUCENE-1934:
<br>---------------------------------------------
<br><br>As Chris work is being released outside Lucene Apache project (<a href="http://www.jteam.nl/news/spatialsolr" target="_top" rel="nofollow">http://www.jteam.nl/news/spatialsolr</a>), should we consider this patch to be re evaluated to commit ?
<br><div class='shrinkable-quote'><br>> Rework (Float)LatLng implementation and distance calculation
<br>> ------------------------------------------------------------
<br>>
<br>> Key: LUCENE-1934
<br>> URL: <a href="https://issues.apache.org/jira/browse/LUCENE-1934" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1934</a><br>> Project: Lucene - Java
<br>> Issue Type: Improvement
<br>> Components: contrib/spatial
<br>> Affects Versions: 2.9
<br>> Reporter: Nicolas Helleringer
<br>> Priority: Minor
<br>> Attachments: LUCENE-1934.patch
<br>>
<br>>
<br>> Clean up of the code and normalisation of the distance calculation to standard
</div><br>--
<br>This message is automatically generated by JIRA.
<br>-
<br>You can reply to this email to add a comment to the issue online.
<br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528733&i=0" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528733&i=1" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26528734[jira] Commented: (LUCENE-1930) Scale moderator not in line with sinusoidal projector2009-11-26T04:28:39Z2009-11-26T04:28:39ZJIRA jira@apache.org<br> [ <a href="https://issues.apache.org/jira/browse/LUCENE-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782872#action_12782872" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782872#action_12782872</a> ]
<br><br>Nicolas Helleringer commented on LUCENE-1930:
<br>---------------------------------------------
<br><br>As Chris work is being released outside Lucene Apache project (<a href="http://www.jteam.nl/news/spatialsolr" target="_top" rel="nofollow">http://www.jteam.nl/news/spatialsolr</a>), should we consider this patch to be re evaluated to commit ?
<br><div class='shrinkable-quote'><br>> Scale moderator not in line with sinusoidal projector
<br>> -----------------------------------------------------
<br>>
<br>> Key: LUCENE-1930
<br>> URL: <a href="https://issues.apache.org/jira/browse/LUCENE-1930" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1930</a><br>> Project: Lucene - Java
<br>> Issue Type: Bug
<br>> Components: contrib/spatial
<br>> Affects Versions: 2.9
<br>> Reporter: Nicolas Helleringer
<br>> Attachments: LUCENE-1930.patch
<br>>
<br>>
<br>> Current implementation in spatial Lucene does :
<br>> public double getTierBoxId (double latitude, double longitude) {
<br>> double[] coords = projector.coords(latitude, longitude);
<br>> double id = getBoxId(coords[0]) + (getBoxId(coords[1]) / tierVerticalPosDivider);
<br>> return id ;
<br>> }
<br>> private double getBoxId (double coord){
<br>> return Math.floor(coord / (idd / this.tierLength));
<br>> }
<br>> with
<br>> Double idd = new Double(180);
<br>> in the CartesianTierPlotter constructor.
<br>> But current Sinusoidal Projector set and used in initialisation of CartesianTierPlotter does :
<br>> public double[] coords(double latitude, double longitude) {
<br>> double rlat = Math.toRadians(latitude);
<br>> double rlong = Math.toRadians(longitude);
<br>> double nlat = rlong * Math.cos(rlat);
<br>> double r[] = {nlat, rlong};
<br>> return r;
<br>> }
<br>> Thus we moderate with idd = 180 a coord that is in a Radian space.
<br>> Things to do :
<br>> 1°) fix idd to : double idd= PI;
<br>> 2°) Move idd definition to IProjector interface : The coord space should belong to the projector doing the job. Change the code from CTP to use that new interface.
</div><br>--
<br>This message is automatically generated by JIRA.
<br>-
<br>You can reply to this email to add a comment to the issue online.
<br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528734&i=0" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528734&i=1" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26528576Re: MergePolicy$MergeException CorruptIndexException in lucene2.4.12009-11-26T04:12:46Z2009-11-26T04:12:46ZjmwapThe process is still running and ops dont want to stop it. As soon as
<br>stops I'll try checkindex.
<br><br>Its created brand new with 2.4.1
<br><br>On Thu, Nov 26, 2009 at 12:42 PM, Michael McCandless
<br><<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528576&i=0" target="_top" rel="nofollow">lucene@...</a>> wrote:
<div class='shrinkable-quote'><br>> I think you're using a JRE that has the fix for the issue found in LUCENE-1282.
<br>>
<br>> Can you run CheckIndex on your index and post the output?
<br>>
<br>> Was this index created from scratch on Lucene 2.4.1? Or, created from
<br>> an earlier Lucene version?
<br>>
<br>> Mike
<br>>
<br>> On Thu, Nov 26, 2009 at 6:03 AM, jm <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528576&i=1" target="_top" rel="nofollow">jmuguruza@...</a>> wrote:
<br>>> or are we really? I think we are on 1.6 update 14 right??
<br>>>
<br>>> sorry Im lost right now on jdk version numbering
<br>>>
<br>>> On Thu, Nov 26, 2009 at 12:01 PM, jm <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528576&i=2" target="_top" rel="nofollow">jmuguruza@...</a>> wrote:
<br>>>> on second thought...I hadnt noticed the jdk numbers properly, we are
<br>>>> using using b28, and JDK 6 Update 10 (b28) is the one fixing this...
<br>>>>
<br>>>> ok forget this then
<br>>>> thanks!
<br>>>>
<br>>>> On Thu, Nov 26, 2009 at 11:55 AM, jm <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528576&i=3" target="_top" rel="nofollow">jmuguruza@...</a>> wrote:
<br>>>>> Hi,
<br>>>>>
<br>>>>> Dont know if this should be here or in java-dev, posting to this one
<br>>>>> first. In one of our installations, we have encountered an exception:
<br>>>>>
<br>>>>> Exception in thread "Lucene Merge Thread #0"
<br>>>>> org.apache.lucene.index.MergePolicy$MergeException:
<br>>>>> org.apache.lucene.index.CorruptIndexException: docs out of order
<br>>>>> (24658 <= 24658 )
<br>>>>> at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309)
<br>>>>> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:286)
<br>>>>> Caused by: org.apache.lucene.index.CorruptIndexException: docs out of
<br>>>>> order (24658 <= 24658 )
<br>>>>> at org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:641)
<br>>>>> at org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:586)
<br>>>>> at org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:547)
<br>>>>> at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:500)
<br>>>>> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:140)
<br>>>>> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4231)
<br>>>>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884)
<br>>>>> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
<br>>>>> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
<br>>>>>
<br>>>>> I have investigated in the list and it looked like
<br>>>>> <a href="https://issues.apache.org/jira/browse/LUCENE-1282" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1282</a>. But we are using
<br>>>>> 2.4.1, and
<br>>>>> C:\Documents and Settings\Administrator>java -version
<br>>>>> java version "1.6.0_14"
<br>>>>> Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
<br>>>>> Java HotSpot(TM) Client VM (build 14.0-b16, mixed mode)
<br>>>>>
<br>>>>> java process launched like this:
<br>>>>> java -server -Xmx1536m ...
<br>>>>>
<br>>>>> So I understand this bug should not be happening??
<br>>>>>
<br>>>>> any idea?
<br>>>>> thanks
<br>>>>> javi
<br>>>>>
<br>>>>
<br>>>
<br>>> ---------------------------------------------------------------------
<br>>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528576&i=4" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528576&i=5" target="_top" rel="nofollow">java-user-help@...</a>
<br>>>
<br>>>
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528576&i=6" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528576&i=7" target="_top" rel="nofollow">java-user-help@...</a>
<br>>
<br>>
</div><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528576&i=8" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528576&i=9" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26528444Is it a lucene bug?2009-11-26T04:01:15Z2009-11-26T04:01:15ZWilson WuHi,
<br> Recently, there is a requirement to sort the hits by both the
<br>scores of documents and the updateTime which is a field of document to
<br>mark the document's update time. We want the new document in the
<br>front and also want high score document in the front,in other words,
<br>we want to mix the score and updateTime, but not first sort by
<br>one,second by the other. So, I design a time based function f(t) to
<br>calculte each document boost to write into the index store.
<br> The result is that I can caculate a value for each document
<br>based its update time, and the value can influence the document score
<br>through adjusting the fieldNorm value. But when I lookup the boost
<br>value through the method document.getBoost() from every document in
<br>the index store, I found the boost value = 1.0. Which means I can set
<br>a document's boost value and the boost value can adjust the final
<br>score, but I can't read the boost value from the document I have
<br>searched out.
<br> Is it a bug in lucene? Thanks. I use lucene version 2.4.1.
<br> PS: Is there any other way to meet my reqirement? I think it is
<br>not a good idea to adjust document's final score through writing a
<br>document boost into the index store. Because if I want to open two
<br>interfaces to the Client: one is sorting documents only by score which
<br>is just the similarity score and has not been adjusted by boost value
<br>f(t), the other is sorting by final score which has been adjuested by
<br>boost value f(t). Thank a lot!
<br><br> wilson
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528444&i=0" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528444&i=1" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26528208Re: MergePolicy$MergeException CorruptIndexException in lucene2.4.12009-11-26T03:42:11Z2009-11-26T03:42:11ZMichael McCandless-2I think you're using a JRE that has the fix for the issue found in LUCENE-1282.
<br><br>Can you run CheckIndex on your index and post the output?
<br><br>Was this index created from scratch on Lucene 2.4.1? Or, created from
<br>an earlier Lucene version?
<br><br>Mike
<br><br>On Thu, Nov 26, 2009 at 6:03 AM, jm <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528208&i=0" target="_top" rel="nofollow">jmuguruza@...</a>> wrote:
<div class='shrinkable-quote'><br>> or are we really? I think we are on 1.6 update 14 right??
<br>>
<br>> sorry Im lost right now on jdk version numbering
<br>>
<br>> On Thu, Nov 26, 2009 at 12:01 PM, jm <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528208&i=1" target="_top" rel="nofollow">jmuguruza@...</a>> wrote:
<br>>> on second thought...I hadnt noticed the jdk numbers properly, we are
<br>>> using using b28, and JDK 6 Update 10 (b28) is the one fixing this...
<br>>>
<br>>> ok forget this then
<br>>> thanks!
<br>>>
<br>>> On Thu, Nov 26, 2009 at 11:55 AM, jm <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528208&i=2" target="_top" rel="nofollow">jmuguruza@...</a>> wrote:
<br>>>> Hi,
<br>>>>
<br>>>> Dont know if this should be here or in java-dev, posting to this one
<br>>>> first. In one of our installations, we have encountered an exception:
<br>>>>
<br>>>> Exception in thread "Lucene Merge Thread #0"
<br>>>> org.apache.lucene.index.MergePolicy$MergeException:
<br>>>> org.apache.lucene.index.CorruptIndexException: docs out of order
<br>>>> (24658 <= 24658 )
<br>>>> at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309)
<br>>>> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:286)
<br>>>> Caused by: org.apache.lucene.index.CorruptIndexException: docs out of
<br>>>> order (24658 <= 24658 )
<br>>>> at org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:641)
<br>>>> at org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:586)
<br>>>> at org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:547)
<br>>>> at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:500)
<br>>>> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:140)
<br>>>> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4231)
<br>>>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884)
<br>>>> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
<br>>>> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
<br>>>>
<br>>>> I have investigated in the list and it looked like
<br>>>> <a href="https://issues.apache.org/jira/browse/LUCENE-1282" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1282</a>. But we are using
<br>>>> 2.4.1, and
<br>>>> C:\Documents and Settings\Administrator>java -version
<br>>>> java version "1.6.0_14"
<br>>>> Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
<br>>>> Java HotSpot(TM) Client VM (build 14.0-b16, mixed mode)
<br>>>>
<br>>>> java process launched like this:
<br>>>> java -server -Xmx1536m ...
<br>>>>
<br>>>> So I understand this bug should not be happening??
<br>>>>
<br>>>> any idea?
<br>>>> thanks
<br>>>> javi
<br>>>>
<br>>>
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528208&i=3" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528208&i=4" target="_top" rel="nofollow">java-user-help@...</a>
<br>>
<br>>
</div><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528208&i=5" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26528208&i=6" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26527752Re: MergePolicy$MergeException CorruptIndexException in lucene2.4.12009-11-26T03:03:30Z2009-11-26T03:03:30Zjmwapor are we really? I think we are on 1.6 update 14 right??
<br><br>sorry Im lost right now on jdk version numbering
<br><br>On Thu, Nov 26, 2009 at 12:01 PM, jm <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527752&i=0" target="_top" rel="nofollow">jmuguruza@...</a>> wrote:
<div class='shrinkable-quote'><br>> on second thought...I hadnt noticed the jdk numbers properly, we are
<br>> using using b28, and JDK 6 Update 10 (b28) is the one fixing this...
<br>>
<br>> ok forget this then
<br>> thanks!
<br>>
<br>> On Thu, Nov 26, 2009 at 11:55 AM, jm <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527752&i=1" target="_top" rel="nofollow">jmuguruza@...</a>> wrote:
<br>>> Hi,
<br>>>
<br>>> Dont know if this should be here or in java-dev, posting to this one
<br>>> first. In one of our installations, we have encountered an exception:
<br>>>
<br>>> Exception in thread "Lucene Merge Thread #0"
<br>>> org.apache.lucene.index.MergePolicy$MergeException:
<br>>> org.apache.lucene.index.CorruptIndexException: docs out of order
<br>>> (24658 <= 24658 )
<br>>> at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309)
<br>>> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:286)
<br>>> Caused by: org.apache.lucene.index.CorruptIndexException: docs out of
<br>>> order (24658 <= 24658 )
<br>>> at org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:641)
<br>>> at org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:586)
<br>>> at org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:547)
<br>>> at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:500)
<br>>> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:140)
<br>>> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4231)
<br>>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884)
<br>>> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
<br>>> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
<br>>>
<br>>> I have investigated in the list and it looked like
<br>>> <a href="https://issues.apache.org/jira/browse/LUCENE-1282" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1282</a>. But we are using
<br>>> 2.4.1, and
<br>>> C:\Documents and Settings\Administrator>java -version
<br>>> java version "1.6.0_14"
<br>>> Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
<br>>> Java HotSpot(TM) Client VM (build 14.0-b16, mixed mode)
<br>>>
<br>>> java process launched like this:
<br>>> java -server -Xmx1536m ...
<br>>>
<br>>> So I understand this bug should not be happening??
<br>>>
<br>>> any idea?
<br>>> thanks
<br>>> javi
<br>>>
<br>>
</div><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527752&i=2" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527752&i=3" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26527721Re: MergePolicy$MergeException CorruptIndexException in lucene2.4.12009-11-26T03:01:18Z2009-11-26T03:01:18Zjmwapon second thought...I hadnt noticed the jdk numbers properly, we are
<br>using using b28, and JDK 6 Update 10 (b28) is the one fixing this...
<br><br>ok forget this then
<br>thanks!
<br><br>On Thu, Nov 26, 2009 at 11:55 AM, jm <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527721&i=0" target="_top" rel="nofollow">jmuguruza@...</a>> wrote:
<div class='shrinkable-quote'><br>> Hi,
<br>>
<br>> Dont know if this should be here or in java-dev, posting to this one
<br>> first. In one of our installations, we have encountered an exception:
<br>>
<br>> Exception in thread "Lucene Merge Thread #0"
<br>> org.apache.lucene.index.MergePolicy$MergeException:
<br>> org.apache.lucene.index.CorruptIndexException: docs out of order
<br>> (24658 <= 24658 )
<br>> at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309)
<br>> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:286)
<br>> Caused by: org.apache.lucene.index.CorruptIndexException: docs out of
<br>> order (24658 <= 24658 )
<br>> at org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:641)
<br>> at org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:586)
<br>> at org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:547)
<br>> at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:500)
<br>> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:140)
<br>> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4231)
<br>> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884)
<br>> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
<br>> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
<br>>
<br>> I have investigated in the list and it looked like
<br>> <a href="https://issues.apache.org/jira/browse/LUCENE-1282" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1282</a>. But we are using
<br>> 2.4.1, and
<br>> C:\Documents and Settings\Administrator>java -version
<br>> java version "1.6.0_14"
<br>> Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
<br>> Java HotSpot(TM) Client VM (build 14.0-b16, mixed mode)
<br>>
<br>> java process launched like this:
<br>> java -server -Xmx1536m ...
<br>>
<br>> So I understand this bug should not be happening??
<br>>
<br>> any idea?
<br>> thanks
<br>> javi
<br>>
</div><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527721&i=1" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527721&i=2" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26527706Re: Need help regarding implementation of autosuggest using jquery2009-11-26T02:59:21Z2009-11-26T02:59:21ZAnshum-2Try this,
<br>Change the code as required:
<br>---------
<br><br><br>import java.io.IOException;
<br><br>import org.apache.lucene.index.CorruptIndexException;
<br>import org.apache.lucene.index.IndexReader;
<br>import org.apache.lucene.index.Term;
<br>import org.apache.lucene.index.TermEnum;
<br><br>/**
<br> * @author anshum
<br> *
<br> */
<br>public class GetTermsToSuggest {
<br><br>private static void getTerms(String inputText) {
<br>IndexReader reader = null;
<br> try {
<br>reader = IndexReader.open("/home/anshum/index/testindex");
<br> String field = "fieldname";
<br>field = field.intern();
<br>TermEnum tenum = reader.terms(new Term("fieldname", ""));
<br> Boolean hasRun = false;
<br>try {
<br>do {
<br> final Term term = tenum.term();
<br>if (term == null || term.field() != field)
<br> break;
<br>final String termText = term.text();
<br>if (termText.startsWith(inputText)) {
<br> System.out.println(termText);
<br>hasRun = true;
<br>} else if (hasRun == true)
<br> break;
<br>} while (tenum.next());
<br>tenum.close();
<br> } catch (IOException e) {
<br>e.printStackTrace();
<br>}
<br> } catch (CorruptIndexException e2) {
<br>e2.printStackTrace();
<br>} catch (IOException e2) {
<br> e2.printStackTrace();
<br>}
<br><br>}
<br><br>/**
<br> * @param args
<br> */
<br> public static void main(String[] args) {
<br>GetTermsToSuggest.getTerms(args[0]);
<br> }
<br>}
<br><br><br>--
<br>Anshum Gupta
<br>Naukri Labs!
<br><a href="http://ai-cafe.blogspot.com" target="_top" rel="nofollow">http://ai-cafe.blogspot.com</a><br><br>The facts expressed here belong to everybody, the opinions to me. The
<br>distinction is yours to draw............
<br><br><br>On Thu, Nov 26, 2009 at 3:19 PM, Uwe Schindler <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527706&i=0" target="_top" rel="nofollow">uwe@...</a>> wrote:
<br><div class='shrinkable-quote'><br>> You can fix this if you just create the initial term not with "", instead
<br>> with your prefix:
<br>> TermEnum tenum = reader.terms(new Term(field,prefix));
<br>>
<br>> And inside the while loop just break out,
<br>>
<br>> if (!termText.startsWith(prefix)) break;
<br>>
<br>> -----
<br>> Uwe Schindler
<br>> H.-H.-Meier-Allee 63, D-28213 Bremen
<br>> <a href="http://www.thetaphi.de" target="_top" rel="nofollow">http://www.thetaphi.de</a><br>> eMail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527706&i=1" target="_top" rel="nofollow">uwe@...</a>
<br>>
<br>>
<br>> > -----Original Message-----
<br>> > From: DHIVYA M [mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527706&i=2" target="_top" rel="nofollow">dhivyakrishnan87@...</a>]
<br>> > Sent: Thursday, November 26, 2009 10:39 AM
<br>> > To: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527706&i=3" target="_top" rel="nofollow">java-user@...</a>
<br>> > Subject: RE: Need help regarding implementation of autosuggest using
<br>> > jquery
<br>> >
<br>> > Sir,
<br>> >
<br>> > Your suggestion was fantastic.
<br>> >
<br>> > I tried the below mentioned code but it is showing me the entire result
<br>> of
<br>> > indexed words starting from the letter that i give as input.
<br>> > Ex:
<br>> > if i give "fo"
<br>> > am getting all the indexes from the word starting with fo upto words
<br>> > starting with z.
<br>> > i.e. it starts displaying from the word matching the search word and ends
<br>> > up with the last word available in the index file.
<br>> >
<br>> > Kindly suggest me a solution for this problem
<br>> >
<br>> > Thanks in advance,
<br>> > Dhivya
<br>> >
<br>> > --- On Wed, 25/11/09, Uwe Schindler <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527706&i=4" target="_top" rel="nofollow">uwe@...</a>> wrote:
<br>> >
<br>> >
<br>> > From: Uwe Schindler <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527706&i=5" target="_top" rel="nofollow">uwe@...</a>>
<br>> > Subject: RE: Need help regarding implementation of autosuggest using
<br>> > jquery
<br>> > To: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527706&i=6" target="_top" rel="nofollow">java-user@...</a>
<br>> > Date: Wednesday, 25 November, 2009, 9:54 AM
<br>> >
<br>> >
<br>> > Hi Dhivya,
<br>> >
<br>> > you can iterate all terms in the index using a TermEnum, that can be
<br>> > retrieved using IndexReader.terms(Term startTerm).
<br>> >
<br>> > If you are interested in all terms from a specific field, position the
<br>> > TermEnum on the first possible term in this field ("") and iterate until
<br>> > the
<br>> > field name changes. As terms in the TermEnum are first ordered by field
<br>> > name
<br>> > then by term text (in UTF-16 order), the loop would look like this:
<br>> >
<br>> > IndexReader reader = ...
<br>> > String field = ....
<br>> > Field = field.intern(); // important for the while loop
<br>> > TermEnum tenum = reader.terms(new Term(field,""));
<br>> > try {
<br>> > do {
<br>> > final Term term = tenum.term();
<br>> > if (term==null || term.field()!=field) break;
<br>> > final String termText = term.text();
<br>> > // do something with the termText
<br>> > } while (tenum.next());
<br>> > } finally {
<br>> > tenum.close();
<br>> > }
<br>> >
<br>> >
<br>> > -----
<br>> > Uwe Schindler
<br>> > H.-H.-Meier-Allee 63, D-28213 Bremen
<br>> > <a href="http://www.thetaphi.de" target="_top" rel="nofollow">http://www.thetaphi.de</a><br>> > eMail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527706&i=7" target="_top" rel="nofollow">uwe@...</a>
<br>> >
<br>> >
<br>> > > -----Original Message-----
<br>> > > From: DHIVYA M [mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527706&i=8" target="_top" rel="nofollow">dhivyakrishnan87@...</a>]
<br>> > > Sent: Wednesday, November 25, 2009 8:06 AM
<br>> > > To: java user
<br>> > > Subject: Need help regarding implementation of autosuggest using jquery
<br>> > >
<br>> > > Hi all,
<br>> > >
<br>> > > Am using lucene 2.3.2 as a search engine in my e-paper site. So that i
<br>> > > want the user to search the news. I achieved that objective but now am
<br>> > > trying to implement autosuggest so that user can pick a choice from the
<br>> > > drop down and no need of typing in the entire sentence or so.
<br>> > >
<br>> > > I have download Jquery for this purpose and am trying to implement it.
<br>> > > The collections of data to refer for the suggestion is given in an
<br>> > > arraylist or jus with in a string.
<br>> > >
<br>> > > But for my application, i need to populate the suggestions with the
<br>> > > indexed words available in the index file created during indexing
<br>> > > operation.
<br>> > >
<br>> > > Can anyone give an idea to read the contents from the index file and
<br>> > make
<br>> > > it available as suggestions? or anyother idea to achieve this
<br>> objective?
<br>> > >
<br>> > > Thanks in advance,
<br>> > > Dhivya
<br>> > >
<br>> > >
<br>> > > The INTERNET now has a personality. YOURS! See your Yahoo!
<br>> > Homepage.
<br>> > > <a href="http://in.yahoo.com/" target="_top" rel="nofollow">http://in.yahoo.com/</a><br>> >
<br>> >
<br>> > ---------------------------------------------------------------------
<br>> > To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527706&i=9" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>> > For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527706&i=10" target="_top" rel="nofollow">java-user-help@...</a>
<br>> >
<br>> >
<br>> >
<br>> >
<br>> > The INTERNET now has a personality. YOURS! See your Yahoo!
<br>> Homepage.
<br>> > <a href="http://in.yahoo.com/" target="_top" rel="nofollow">http://in.yahoo.com/</a><br>>
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527706&i=11" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527706&i=12" target="_top" rel="nofollow">java-user-help@...</a>
<br>>
<br>>
<br></div><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26527663MergePolicy$MergeException CorruptIndexException in lucene2.4.12009-11-26T02:55:14Z2009-11-26T02:55:14ZjmwapHi,
<br><br>Dont know if this should be here or in java-dev, posting to this one
<br>first. In one of our installations, we have encountered an exception:
<br><br>Exception in thread "Lucene Merge Thread #0"
<br>org.apache.lucene.index.MergePolicy$MergeException:
<br>org.apache.lucene.index.CorruptIndexException: docs out of order
<br>(24658 <= 24658 )
<br> at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309)
<br> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:286)
<br>Caused by: org.apache.lucene.index.CorruptIndexException: docs out of
<br>order (24658 <= 24658 )
<br> at org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:641)
<br> at org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:586)
<br> at org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:547)
<br> at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:500)
<br> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:140)
<br> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4231)
<br> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884)
<br> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
<br> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
<br><br>I have investigated in the list and it looked like
<br><a href="https://issues.apache.org/jira/browse/LUCENE-1282" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1282</a>. But we are using
<br>2.4.1, and
<br>C:\Documents and Settings\Administrator>java -version
<br>java version "1.6.0_14"
<br>Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
<br>Java HotSpot(TM) Client VM (build 14.0-b16, mixed mode)
<br><br>java process launched like this:
<br>java -server -Xmx1536m ...
<br><br>So I understand this bug should not be happening??
<br><br>any idea?
<br>thanks
<br>javi
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527663&i=0" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527663&i=1" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26527549Re: Jira emails via Gmail2009-11-26T02:45:17Z2009-11-26T02:45:17ZMichael McCandless-2JRA-3609 gets at yet another improvement I'd like to see in Jira,
<br>which is the ability to customize subject-line for each comment added
<br>to an issue.
<br><br>EG I would have really liked to use the comment 'pathological GC
<br>behavior' for this post:
<br><br> <a href="https://issues.apache.org/jira/browse/LUCENE-2075?focusedCommentId=12782069&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12782069" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-2075?focusedCommentId=12782069&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12782069</a><br><br>I posted the request to the Jira forums, here:
<br><br> <a href="http://forums.atlassian.com/thread.jspa?threadID=39074&tstart=0" target="_top" rel="nofollow">http://forums.atlassian.com/thread.jspa?threadID=39074&tstart=0</a><br><br>But, hideously (thanks to Gmail's overly simplistic thread grouping),
<br>JRA-3609 is completely at odds with fixing Gmail to group the Jira
<br>mails correctly. Sigh.
<br><br>Mike
<br><br>On Wed, Nov 25, 2009 at 1:43 PM, Chris Hostetter
<br><<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527549&i=0" target="_top" rel="nofollow">hossman_lucene@...</a>> wrote:
<div class='shrinkable-quote'><br>>
<br>> : I signed up for a login, and voted for this issue. If others did the same,
<br>> : that might help.
<br>>
<br>> if you read the comments in the issue, there's really nothing that can be
<br>> fixed in Jira to make this work better -- jira already puts an
<br>> In-Reply-To header on all of the messages so that mail clients who do
<br>> threading "correctly" can use them -- the problem is that Gmail isn't
<br>> looking at those headers, and is instead focusing on subject.
<br>>
<br>> voting for JRA-12640 probably won't accomplish much, since the bug is in
<br>> GMail, not Jira -- but voting for JRA-3609 might help since then Jira
<br>> could be customized to keep the subject consistent for all types of
<br>> messages related to a single issue...
<br>>
<br>> <a href="http://jira.atlassian.com/browse/JRA-3609" target="_top" rel="nofollow">http://jira.atlassian.com/browse/JRA-3609</a><br>>
<br>> ...or you could file a bug with GMail asking them to implement the
<br>> defacto standard algorithm for email message threading, using all of the
<br>> various headers that exist for this purpose...
<br>>
<br>> <a href="http://www.jwz.org/doc/threading.html" target="_top" rel="nofollow">http://www.jwz.org/doc/threading.html</a><br>>
<br>>
<br>>
<br>> -Hoss
<br>>
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527549&i=1" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527549&i=2" target="_top" rel="nofollow">java-dev-help@...</a>
<br>>
<br>>
</div><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527549&i=3" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527549&i=4" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26527271[ANNOUNCE] Apache Lucene Java 3.0.0 released2009-11-26T02:24:26Z2009-11-26T02:24:26ZUwe SchindlerHello Lucene users,
<br><br>On behalf of the Lucene dev community (a growing community far larger than
<br>just the committers) I would like to announce the release of Lucene Java
<br>3.0.0:
<br><br>The new version is mostly a cleanup release without any new features. All
<br>deprecations targeted to be removed in version 3.0 were removed. If you are
<br>upgrading from version 2.9.1 of Lucene, you have to fix all deprecation
<br>warnings in your code base to be able to recompile against this version.
<br><br>This is the first Lucene release with Java 5 as a minimum requirement. The
<br>API was cleaned up to make use of Java 5's generics, varargs, enums, and
<br>autoboxing. New users of Lucene are advised to use this version for new
<br>developments, because it has a clean, type safe new API. Upgrading users can
<br>now remove unnecessary casts and add generics to their code, too. If you
<br>have not upgraded your installation to Java 5, please read the file
<br>JRE_VERSION_MIGRATION.txt (please note that this is not related only to this
<br>version of Lucene, it will also happen with any previous release when you
<br>upgrade your Java environment).
<br><br>Lucene 3.0.0 has some changes regarding compressed fields: 2.9.0 already
<br>deprecated compressed fields; support for them was removed now. Lucene 3.0.0
<br>is still able to read indexes with compressed fields, but as soon as merges
<br>occur or the index is optimized, all compressed fields are decompressed and
<br>converted to Field.Store.YES. Because of this, indexes with compressed
<br>fields can suddenly get larger.
<br><br>While we generally try and maintain full backwards compatibility between
<br>major versions, Lucene 3.0.0 has some minor breaks, mostly related to
<br>deprecation removal, pointed out in the 'Changes in backwards compatibility
<br>policy' section of CHANGES.txt. Notable are:
<br><br>- IndexReader.open(Directory) now opens in read-only mode per default (this
<br>method was deprecated because of that in 2.9.X). The same occurs to
<br>IndexSearcher.
<br><br>- Already started in 2.9, core TokenStreams are now made final to enforce
<br>the decorator pattern.
<br><br>- If you interrupt an IndexWriter merge thread, IndexWriter now throws an
<br>unchecked ThreadInterruptedException that extends RuntimeException and
<br>clears the interrupt status.
<br><br><br>See core changes at
<br><a href="http://lucene.apache.org/java/3_0_0/changes/Changes.html" target="_top" rel="nofollow">http://lucene.apache.org/java/3_0_0/changes/Changes.html</a><br>and contrib changes at
<br><a href="http://lucene.apache.org/java/3_0_0/changes/Contrib-Changes.html" target="_top" rel="nofollow">http://lucene.apache.org/java/3_0_0/changes/Contrib-Changes.html</a><br><br>Binary and source distributions are available at
<br><a href="http://www.apache.org/dyn/closer.cgi/lucene/java/" target="_top" rel="nofollow">http://www.apache.org/dyn/closer.cgi/lucene/java/</a><br><br>Lucene artifacts are also available in the Maven2 repository at
<br><a href="http://repo1.maven.org/maven2/org/apache/lucene/" target="_top" rel="nofollow">http://repo1.maven.org/maven2/org/apache/lucene/</a><br><br><br>-----
<br>Uwe Schindler
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527271&i=0" target="_top" rel="nofollow">uschindler@...</a>
<br>Apache Lucene Java Committer
<br>Bremen, Germany
<br><a href="http://lucene.apache.org/java/docs/" target="_top" rel="nofollow">http://lucene.apache.org/java/docs/</a><br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527271&i=1" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527271&i=2" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26527268[ANNOUNCE] Apache Lucene Java 3.0.0 released2009-11-26T02:24:26Z2009-11-26T02:24:26ZUwe SchindlerHello Lucene users,
<br><br>On behalf of the Lucene dev community (a growing community far larger than
<br>just the committers) I would like to announce the release of Lucene Java
<br>3.0.0:
<br><br>The new version is mostly a cleanup release without any new features. All
<br>deprecations targeted to be removed in version 3.0 were removed. If you are
<br>upgrading from version 2.9.1 of Lucene, you have to fix all deprecation
<br>warnings in your code base to be able to recompile against this version.
<br><br>This is the first Lucene release with Java 5 as a minimum requirement. The
<br>API was cleaned up to make use of Java 5's generics, varargs, enums, and
<br>autoboxing. New users of Lucene are advised to use this version for new
<br>developments, because it has a clean, type safe new API. Upgrading users can
<br>now remove unnecessary casts and add generics to their code, too. If you
<br>have not upgraded your installation to Java 5, please read the file
<br>JRE_VERSION_MIGRATION.txt (please note that this is not related only to this
<br>version of Lucene, it will also happen with any previous release when you
<br>upgrade your Java environment).
<br><br>Lucene 3.0.0 has some changes regarding compressed fields: 2.9.0 already
<br>deprecated compressed fields; support for them was removed now. Lucene 3.0.0
<br>is still able to read indexes with compressed fields, but as soon as merges
<br>occur or the index is optimized, all compressed fields are decompressed and
<br>converted to Field.Store.YES. Because of this, indexes with compressed
<br>fields can suddenly get larger.
<br><br>While we generally try and maintain full backwards compatibility between
<br>major versions, Lucene 3.0.0 has some minor breaks, mostly related to
<br>deprecation removal, pointed out in the 'Changes in backwards compatibility
<br>policy' section of CHANGES.txt. Notable are:
<br><br>- IndexReader.open(Directory) now opens in read-only mode per default (this
<br>method was deprecated because of that in 2.9.X). The same occurs to
<br>IndexSearcher.
<br><br>- Already started in 2.9, core TokenStreams are now made final to enforce
<br>the decorator pattern.
<br><br>- If you interrupt an IndexWriter merge thread, IndexWriter now throws an
<br>unchecked ThreadInterruptedException that extends RuntimeException and
<br>clears the interrupt status.
<br><br><br>See core changes at
<br><a href="http://lucene.apache.org/java/3_0_0/changes/Changes.html" target="_top" rel="nofollow">http://lucene.apache.org/java/3_0_0/changes/Changes.html</a><br>and contrib changes at
<br><a href="http://lucene.apache.org/java/3_0_0/changes/Contrib-Changes.html" target="_top" rel="nofollow">http://lucene.apache.org/java/3_0_0/changes/Contrib-Changes.html</a><br><br>Binary and source distributions are available at
<br><a href="http://www.apache.org/dyn/closer.cgi/lucene/java/" target="_top" rel="nofollow">http://www.apache.org/dyn/closer.cgi/lucene/java/</a><br><br>Lucene artifacts are also available in the Maven2 repository at
<br><a href="http://repo1.maven.org/maven2/org/apache/lucene/" target="_top" rel="nofollow">http://repo1.maven.org/maven2/org/apache/lucene/</a><br><br><br>-----
<br>Uwe Schindler
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527268&i=0" target="_top" rel="nofollow">uschindler@...</a>
<br>Apache Lucene Java Committer
<br>Bremen, Germany
<br><a href="http://lucene.apache.org/java/docs/" target="_top" rel="nofollow">http://lucene.apache.org/java/docs/</a><br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527268&i=1" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527268&i=2" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26527261Re: (LUCENE-1844) Speed up junit tests2009-11-26T02:24:06Z2009-11-26T02:24:06ZMichael McCandless-2I still think there's value to faster tests, even if they don't become
<br>so fast as to enable "fully interactive testing".
<br><br>Plus, this is an ongoing goal with time, not a one-time event. As we
<br>create tests we should generally try to maximize coverage and minimize
<br>CPU cost, as long as the effort is smallish.
<br><br>Mike
<br><br>On Wed, Nov 25, 2009 at 9:32 PM, Erick Erickson <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527261&i=0" target="_top" rel="nofollow">erickerickson@...</a>> wrote:
<div class='shrinkable-quote'><br>> I posted a rather long diatribe outlining why I think speed-ups
<br>> are a false goal for Lucene. Briefly, I'm convinced that as long
<br>> as the tests are run when Hudson builds Lucene, 99% of the
<br>> value of unit tests is realized. I suppose this implies that the
<br>> hard-core committers agree that as long as failed tests
<br>> are caught/corrected within a day things are fine.
<br>>
<br>> Although coming from a background where unit
<br>> tests are not always required, my viewpoint may be
<br>> suspect <G>.
<br>>
<br>> <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527261&i=1" target="_top" rel="nofollow">Erick@...</a>....
<br>>
<br>> On Wed, Nov 25, 2009 at 8:43 PM, Michael McCandless (JIRA)
<br>> <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527261&i=2" target="_top" rel="nofollow">jira@...</a>>wrote:
<br>>
<br>>>
<br>>> [
<br>>> <a href="https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782716#action_12782716" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782716#action_12782716</a>]
<br>>>
<br>>> Michael McCandless commented on LUCENE-1844:
<br>>> --------------------------------------------
<br>>>
<br>>> Will we also speed up back-compat tests?
<br>>>
<br>>> > Speed up junit tests
<br>>> > --------------------
<br>>> >
<br>>> > Key: LUCENE-1844
<br>>> > URL: <a href="https://issues.apache.org/jira/browse/LUCENE-1844" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844</a><br>>> > Project: Lucene - Java
<br>>> > Issue Type: Improvement
<br>>> > Reporter: Mark Miller
<br>>> > Attachments: FastCnstScoreQTest.patch,
<br>>> hi_junit_test_runtimes.png, LUCENE-1844.patch
<br>>> >
<br>>> >
<br>>> > As Lucene grows, so does the number of JUnit tests. This is obviously a
<br>>> good thing, but it comes with longer and longer test times. Now that we also
<br>>> run back compat tests in a standard test run, this problem is essentially
<br>>> doubled.
<br>>> > There are some ways this may get better, including running parallel
<br>>> tests. You will need the hardware to fully take advantage, but it should be
<br>>> a nice gain. There is already an issue for this, and Junit 4.6, 4.7 have the
<br>>> beginnings of something we might be able to count on soon. 4.6 was buggy,
<br>>> and 4.7 still doesn't come with nice ant integration. Parallel tests will
<br>>> come though.
<br>>> > Beyond parallel testing, I think we also need to concentrate on keeping
<br>>> our tests lean. We don't want to sacrifice coverage or quality, but I'm sure
<br>>> there is plenty of fat to skim.
<br>>> > I've started making a list of some of the longer tests - I think with
<br>>> some work we can make our tests much faster - and then with parallelization,
<br>>> I think we could see some really great gains.
<br>>>
<br>>> --
<br>>> This message is automatically generated by JIRA.
<br>>> -
<br>>> You can reply to this email to add a comment to the issue online.
<br>>>
<br>>>
<br>>> ---------------------------------------------------------------------
<br>>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527261&i=3" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527261&i=4" target="_top" rel="nofollow">java-dev-help@...</a>
<br>>>
<br>>>
<br>>
<br>> I posted a rather long diatribe outlining why I think speed-ups
<br>> are a false goal for Lucene. Briefly, I'm convinced that as long
<br>> as the tests are run when Hudson builds Lucene, 99% of the
<br>> value of unit tests is realized. I suppose this implies that the
<br>> hard-core committers agree that as long as failed tests
<br>> are caught/corrected within a day things are fine.
<br>>
<br>> Although coming from a background where unit
<br>> tests are not always required, my viewpoint may be
<br>> suspect <G>.
<br>>
<br>> <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527261&i=5" target="_top" rel="nofollow">Erick@...</a>....
<br>>
<br>> On Wed, Nov 25, 2009 at 8:43 PM, Michael McCandless (JIRA) <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527261&i=6" target="_top" rel="nofollow">jira@...</a>> wrote:
<br>>>
<br>>> [ <a href="https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782716#action_12782716" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782716#action_12782716</a> ]
<br>>>
<br>>> Michael McCandless commented on LUCENE-1844:
<br>>> --------------------------------------------
<br>>>
<br>>> Will we also speed up back-compat tests?
<br>>>
<br>>> > Speed up junit tests
<br>>> > --------------------
<br>>> >
<br>>> > Key: LUCENE-1844
<br>>> > URL: <a href="https://issues.apache.org/jira/browse/LUCENE-1844" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-1844</a><br>>> > Project: Lucene - Java
<br>>> > Issue Type: Improvement
<br>>> > Reporter: Mark Miller
<br>>> > Attachments: FastCnstScoreQTest.patch, hi_junit_test_runtimes.png, LUCENE-1844.patch
<br>>> >
<br>>> >
<br>>> > As Lucene grows, so does the number of JUnit tests. This is obviously a good thing, but it comes with longer and longer test times. Now that we also run back compat tests in a standard test run, this problem is essentially doubled.
<br>>> > There are some ways this may get better, including running parallel tests. You will need the hardware to fully take advantage, but it should be a nice gain. There is already an issue for this, and Junit 4.6, 4.7 have the beginnings of something we might be able to count on soon. 4.6 was buggy, and 4.7 still doesn't come with nice ant integration. Parallel tests will come though.
<br>>> > Beyond parallel testing, I think we also need to concentrate on keeping our tests lean. We don't want to sacrifice coverage or quality, but I'm sure there is plenty of fat to skim.
<br>>> > I've started making a list of some of the longer tests - I think with some work we can make our tests much faster - and then with parallelization, I think we could see some really great gains.
<br>>>
<br>>> --
<br>>> This message is automatically generated by JIRA.
<br>>> -
<br>>> You can reply to this email to add a comment to the issue online.
<br>>>
<br>>>
<br>>> ---------------------------------------------------------------------
<br>>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527261&i=7" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527261&i=8" target="_top" rel="nofollow">java-dev-help@...</a>
<br>>>
<br>>
<br>>
<br>>
</div><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527261&i=9" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527261&i=10" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26527172What about a Unicode support release2009-11-26T02:15:21Z2009-11-26T02:15:21ZSimon WillnauerSince 3.0 is almost out of the door and development on Unicode support
<br>for supplementary characters has already started heavily (I like the
<br>word heavy - like uwe) I see a lot of stuff going on in the flex
<br>branch which seems to be a huge change that might take quite a while
<br>to finish / be releaseable. Other nice features like the automation
<br>stuff seems to be ready in the near future and together with the
<br>unicode support this appears to be one of the missing features in
<br>Lucene these days.
<br>I wonder if we should go forward and try to finish these features and
<br>proceed to 3.1. This will not slow down any other development as far
<br>as I can see and many people might be happy with those features.
<br><br>does that seem to be feasible?
<br><br>simon
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527172&i=0" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527172&i=1" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26527168Re: (LUCENE-1844) Speed up junit tests2009-11-26T02:14:55Z2009-11-26T02:14:55ZMichael McCandless-2On Wed, Nov 25, 2009 at 8:29 PM, Erick Erickson <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527168&i=0" target="_top" rel="nofollow">erickerickson@...</a>> wrote:
<br>> IMHO there are other reasons to upgrade to junit4 besides
<br>> parallelization, there are some nice new capabilities. I
<br>> suppose the analogous question is "why upgrade to
<br>> Lucene 2.9?"
<br><br>OK then I think we should do the upgrade, and separately think about
<br>how to take advantage of parallelization... I applied the patch for
<br>1084, and also carried it to the back-compat branch, and upgraded the
<br>junit jar in the lib dir to 4.7, and all tests run fine. I also
<br>separately tested 2037, and it ran fine as well.
<br><div class='shrinkable-quote'><br>> Especially since it's not a matter of upgrading. Junit3 tests run
<br>> just fine under junit4. I've tested after removing the
<br>> junit3 jar from lib, no problem. It even seems to run
<br>> slightly faster, which makes me wonder...
<br>>
<br>> So really, we have the best of both worlds. No work involved in
<br>> using Junit4 with the current tests, but the ability to use the
<br>> new features of Junit4. Although I'm sure there'll be
<br>> *something* that bites us, I have great faith in Murphy.
<br>>
<br>> Kinda reminds me of the Lucene drop-in replacement policy
<br>> <G>.......
</div><br>Heh ;)
<br><div class='shrinkable-quote'><br>> But on the topic of parallelization: I'm not at all sure
<br>> it's worth the effort. As far as I can tell, it really only gets significant
<br>> gains when you have more cores to run with. It's not at all clear
<br>> to me how much time we spend doing I/O in the tests... very little
<br>> I suspect (although I confess I don't know for sure). And if we're
<br>> CPU bound anyway, parallelization doesn't help. Anybody know
<br>> for sure?
<br>>
<br>> And say we did all the work to parallelize all the tests. And say that
<br>> instead of taking 25 minutes on my 3 year old MacBook Pro, we
<br>> got it down to 10 minutes. Who cares? 10 minutes is still too long
<br>> according to the eXtreme Programming (XP) folks, and I sympathize
<br>> with their point of view. Even though I did spend some time trying
<br>> to trim some time.
</div><br>If you have enough cores, you can get decent speedups. I have a scary
<br>python wrapper that runs the tests in parallel (runs 5 threads on an 8
<br>core machine... any more threads seems not to help), by letting each
<br>thread run a separate package, and sub-dividing the big (index,
<br>search) packages w/ a temporary patch. Still it takes ~6.5 minutes to
<br>run the tests on my machine these days, which is still too long for
<br>interactive testing...
<br><div class='shrinkable-quote'><br>> The XP approach to unit testing is to run it almost every time you
<br>> change a line of code. OK, I'm exaggerating, but not by too
<br>> much with the die-hard XP folks. Even at 10 minutes, we can't
<br>> do that.
<br>>
<br>> So, I think the value for Lucene/SOLR comes *not* from running the
<br>> tests 15 times an hour. I think the real value comes from not
<br>> letting errors hide for days/weeks/months/releases. So I'm quite willing
<br>> to let the automated builds catch the unit test failures in unexpected
<br>> places in those instances where I don't run all of the tests before
<br>> a patch is committed. As long as we fix them as soon as they're
<br>> found.
</div><br>Kinda like what's the point of a cell phone having better battery life
<br>when it's already plenty for my daily usage yet not enough for 2 days
<br>usage, ie, I still have to charge it up, nightly.... it's quantized.
<br>Slightly faster tests won't really make a fundamental change in our
<br>development approach.
<br><br>But, still, while this speedup won't make the tests fully interactive,
<br>it's still an important improvement. I run the unit tests an insane
<br>number times, and bringing the time down even "just" by 2 minutes will
<br>make a real difference.
<br><br>> OK, I'm rambling. I'm off for Thanksgiving, and my daughter is
<br>> at her in-laws until tomorrow (they're visiting from CA). So sue
<br>> me <G>.
<br><br>Happy Thanksgiving!!
<br><br>Mike
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527168&i=1" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527168&i=2" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26527156Re: To exit the while loop if match is found2009-11-26T02:13:54Z2009-11-26T02:13:54ZSavvas-Andreas Moysidishi,
<br><br>maybe you could not break but rather collect them in an ordered collection.
<br>If it's a web app you are having you can then pass it up your view layer
<br>wrapped in json?
<br><br>savvas
<br><br>2009/11/26 DHIVYA M <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=0" target="_top" rel="nofollow">dhivyakrishnan87@...</a>>
<br><div class='shrinkable-quote'><br>> Thanks for your suggested code sir.
<br>>
<br>> But it displays only one word.
<br>>
<br>> Ex:
<br>> if the input is "z"
<br>>
<br>> actual output must be
<br>>
<br>> zing
<br>> Zohar
<br>>
<br>> but am getting zing alone
<br>>
<br>> --- On Thu, 26/11/09, Uwe Schindler <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=1" target="_top" rel="nofollow">uwe@...</a>> wrote:
<br>>
<br>>
<br>> From: Uwe Schindler <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=2" target="_top" rel="nofollow">uwe@...</a>>
<br>> Subject: RE: Need help regarding implementation of autosuggest using jquery
<br>> To: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=3" target="_top" rel="nofollow">java-user@...</a>
<br>> Date: Thursday, 26 November, 2009, 9:49 AM
<br>>
<br>>
<br>> You can fix this if you just create the initial term not with "", instead
<br>> with your prefix:
<br>> TermEnum tenum = reader.terms(new Term(field,prefix));
<br>>
<br>> And inside the while loop just break out,
<br>>
<br>> if (!termText.startsWith(prefix)) break;
<br>>
<br>> -----
<br>> Uwe Schindler
<br>> H.-H.-Meier-Allee 63, D-28213 Bremen
<br>> <a href="http://www.thetaphi.de" target="_top" rel="nofollow">http://www.thetaphi.de</a><br>> eMail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=4" target="_top" rel="nofollow">uwe@...</a>
<br>>
<br>>
<br>> > -----Original Message-----
<br>> > From: DHIVYA M [mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=5" target="_top" rel="nofollow">dhivyakrishnan87@...</a>]
<br>> > Sent: Thursday, November 26, 2009 10:39 AM
<br>> > To: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=6" target="_top" rel="nofollow">java-user@...</a>
<br>> > Subject: RE: Need help regarding implementation of autosuggest using
<br>> > jquery
<br>> >
<br>> > Sir,
<br>> >
<br>> > Your suggestion was fantastic.
<br>> >
<br>> > I tried the below mentioned code but it is showing me the entire result
<br>> of
<br>> > indexed words starting from the letter that i give as input.
<br>> > Ex:
<br>> > if i give "fo"
<br>> > am getting all the indexes from the word starting with fo upto words
<br>> > starting with z.
<br>> > i.e. it starts displaying from the word matching the search word and ends
<br>> > up with the last word available in the index file.
<br>> >
<br>> > Kindly suggest me a solution for this problem
<br>> >
<br>> > Thanks in advance,
<br>> > Dhivya
<br>> >
<br>> > --- On Wed, 25/11/09, Uwe Schindler <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=7" target="_top" rel="nofollow">uwe@...</a>> wrote:
<br>> >
<br>> >
<br>> > From: Uwe Schindler <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=8" target="_top" rel="nofollow">uwe@...</a>>
<br>> > Subject: RE: Need help regarding implementation of autosuggest using
<br>> > jquery
<br>> > To: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=9" target="_top" rel="nofollow">java-user@...</a>
<br>> > Date: Wednesday, 25 November, 2009, 9:54 AM
<br>> >
<br>> >
<br>> > Hi Dhivya,
<br>> >
<br>> > you can iterate all terms in the index using a TermEnum, that can be
<br>> > retrieved using IndexReader.terms(Term startTerm).
<br>> >
<br>> > If you are interested in all terms from a specific field, position the
<br>> > TermEnum on the first possible term in this field ("") and iterate until
<br>> > the
<br>> > field name changes. As terms in the TermEnum are first ordered by field
<br>> > name
<br>> > then by term text (in UTF-16 order), the loop would look like this:
<br>> >
<br>> > IndexReader reader = ...
<br>> > String field = ....
<br>> > Field = field.intern(); // important for the while loop
<br>> > TermEnum tenum = reader.terms(new Term(field,""));
<br>> > try {
<br>> > do {
<br>> > final Term term = tenum.term();
<br>> > if (term==null || term.field()!=field) break;
<br>> > final String termText = term.text();
<br>> > // do something with the termText
<br>> > } while (tenum.next());
<br>> > } finally {
<br>> > tenum.close();
<br>> > }
<br>> >
<br>> >
<br>> > -----
<br>> > Uwe Schindler
<br>> > H.-H.-Meier-Allee 63, D-28213 Bremen
<br>> > <a href="http://www.thetaphi.de" target="_top" rel="nofollow">http://www.thetaphi.de</a><br>> > eMail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=10" target="_top" rel="nofollow">uwe@...</a>
<br>> >
<br>> >
<br>> > > -----Original Message-----
<br>> > > From: DHIVYA M [mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=11" target="_top" rel="nofollow">dhivyakrishnan87@...</a>]
<br>> > > Sent: Wednesday, November 25, 2009 8:06 AM
<br>> > > To: java user
<br>> > > Subject: Need help regarding implementation of autosuggest using jquery
<br>> > >
<br>> > > Hi all,
<br>> > >
<br>> > > Am using lucene 2.3.2 as a search engine in my e-paper site. So that i
<br>> > > want the user to search the news. I achieved that objective but now am
<br>> > > trying to implement autosuggest so that user can pick a choice from the
<br>> > > drop down and no need of typing in the entire sentence or so.
<br>> > >
<br>> > > I have download Jquery for this purpose and am trying to implement it.
<br>> > > The collections of data to refer for the suggestion is given in an
<br>> > > arraylist or jus with in a string.
<br>> > >
<br>> > > But for my application, i need to populate the suggestions with the
<br>> > > indexed words available in the index file created during indexing
<br>> > > operation.
<br>> > >
<br>> > > Can anyone give an idea to read the contents from the index file and
<br>> > make
<br>> > > it available as suggestions? or anyother idea to achieve this
<br>> objective?
<br>> > >
<br>> > > Thanks in advance,
<br>> > > Dhivya
<br>> > >
<br>> > >
<br>> > > The INTERNET now has a personality. YOURS! See your Yahoo!
<br>> > Homepage.
<br>> > > <a href="http://in.yahoo.com/" target="_top" rel="nofollow">http://in.yahoo.com/</a><br>> >
<br>> >
<br>> > ---------------------------------------------------------------------
<br>> > To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=12" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>> > For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=13" target="_top" rel="nofollow">java-user-help@...</a>
<br>> >
<br>> >
<br>> >
<br>> >
<br>> > The INTERNET now has a personality. YOURS! See your Yahoo!
<br>> Homepage.
<br>> > <a href="http://in.yahoo.com/" target="_top" rel="nofollow">http://in.yahoo.com/</a><br>>
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=14" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527156&i=15" target="_top" rel="nofollow">java-user-help@...</a>
<br>>
<br>>
<br>>
<br>>
<br>> The INTERNET now has a personality. YOURS! See your Yahoo! Homepage.
<br>> <a href="http://in.yahoo.com/" target="_top" rel="nofollow">http://in.yahoo.com/</a><br></div><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>tag:old.nabble.com,2006:post-26527057[jira] Commented: (LUCENE-2037) Allow Junit4 tests in our environment.2009-11-26T02:05:39Z2009-11-26T02:05:39ZJIRA jira@apache.org<br> [ <a href="https://issues.apache.org/jira/browse/LUCENE-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782823#action_12782823" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782823#action_12782823</a> ]
<br><br>Michael McCandless commented on LUCENE-2037:
<br>--------------------------------------------
<br><br>Can we, for this issue, focus on not altering what's tested, ie, simply cutover to juni4? And then open new issues for, eg, parameterizing tests across different Versions? If we do that, then Erick if you post your latest patch (that cuts over LocalizedTestCase -> new junit4 capabilities), then is that committable?
<br><div class='shrinkable-quote'><br>> Allow Junit4 tests in our environment.
<br>> --------------------------------------
<br>>
<br>> Key: LUCENE-2037
<br>> URL: <a href="https://issues.apache.org/jira/browse/LUCENE-2037" target="_top" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-2037</a><br>> Project: Lucene - Java
<br>> Issue Type: Improvement
<br>> Components: Other
<br>> Affects Versions: 3.1
<br>> Environment: Development
<br>> Reporter: Erick Erickson
<br>> Assignee: Erick Erickson
<br>> Priority: Minor
<br>> Fix For: 3.1
<br>>
<br>> Attachments: junit-4.7.jar, LUCENE-2037.patch
<br>>
<br>> Original Estimate: 8h
<br>> Remaining Estimate: 8h
<br>>
<br>> Now that we're dropping Java 1.4 compatibility for 3.0, we can incorporate Junit4 in testing. Junit3 and junit4 tests can coexist, so no tests should have to be rewritten. We should start this for the 3.1 release so we can get a clean 3.0 out smoothly.
<br>> It's probably worthwhile to convert a small set of tests as an exemplar.
</div><br>--
<br>This message is automatically generated by JIRA.
<br>-
<br>You can reply to this email to add a comment to the issue online.
<br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527057&i=0" target="_top" rel="nofollow">java-dev-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26527057&i=1" target="_top" rel="nofollow">java-dev-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Developer-f46.html" embed="fixTarget[46]" target="_top" >Lucene - Java Developer</a></p>tag:old.nabble.com,2006:post-26526958Re: Is Lucene a good choice for PB scale mailbox search?2009-11-26T01:57:18Z2009-11-26T01:57:18ZIan LeaIf you are planning on using lucene only for searching then you don't
<br>need to store much data at all - just the message id or whatever you
<br>use to identify messages. And there won't be much point in
<br>compressing that.
<br><br>If on the other hand you plan on storing data in lucene, perhaps for
<br>displaying hits on a web page, you might want to compress it. That
<br>will save some space but at the cost of some performance at indexing
<br>and retrieval time. If you are storing, say, From:, To: and Subject:
<br>for display in search results and message body only displayed when
<br>they want to view the message, you could leave the first three
<br>uncompressed and compress the message body.
<br><br>Personally, I only use compression in indexes storing large fields but
<br>with low search/retrieval rate. But my indexes are only a few Gb in
<br>size.
<br><br>Lucene's handling of compressed fields is changing in 3.0 - see the
<br>release notes or 2.9 javadocs for Field.Store.html#COMPRESS
<br><br><br>--
<br>Ian.
<br><br>On Thu, Nov 26, 2009 at 1:34 AM, fulin tang <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26526958&i=0" target="_top" rel="nofollow">tangfulin@...</a>> wrote:
<div class='shrinkable-quote'><br>> Thanks all for the good suggestions !
<br>>
<br>> But any idea of the storage? How can we make the indexes as small as possible?
<br>>
<br>> We know compressing is the only way, but when and where to compress is
<br>> best for search?
<br>>
<br>> Thanks all again!
<br>>
<br>>
<br>> 2009/11/24 Kay Kay <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26526958&i=1" target="_top" rel="nofollow">kaykay.unique@...</a>>:
<br>>> fulin tang wrote:
<br>>>>
<br>>>> We are going to add full-text search for our mailbox service .
<br>>>>
<br>>>> The problem is we have more than 1 PB mails there , and obviously we
<br>>>> don't want to add another PB storage for search service , so we hope
<br>>>> the index data will be small enough for storage while the search keeps
<br>>>> fast .
<br>>>>
<br>>>> The lucky is that every user just search with mails of their own , so
<br>>>> we can split the data into a lot of indexes instead of keeping them in
<br>>>> a big one .
<br>>>>
<br>>>
<br>>> If it is going to be sharded by the 'To' or 'Cc' list - then potentially the
<br>>> mail information is going to be duplicated proportional to the number of
<br>>> people in an email thread. Selecting some other dimension like time, for
<br>>> sharding might be useful to begin with.
<br>>>>
<br>>>> So, after all these concerns , the question is , is lucene a good
<br>>>> choice for this ? or which is the right way to do this ? Does anyone
<br>>>> have done this before ?
<br>>>>
<br>>>
<br>>> With PB of storage - check out solr sharding / katta for prior work in this
<br>>> arena.
<br>>>
<br>>>> All opinions and comments are welcome !
<br>>>>
<br>>>> fulin
<br>>>>
<br>>>>
<br>>>>
<br>>>
<br>>>
<br>>> ---------------------------------------------------------------------
<br>>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26526958&i=2" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26526958&i=3" target="_top" rel="nofollow">java-user-help@...</a>
<br>>>
<br>>>
<br>>
<br>>
<br>>
<br>> --
<br>> 梦的开始挣扎于城市的边缘
<br>> 心的远方执着在脚步的瞬间
<br>> 我的宿命埋藏了寂寞的永远
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26526958&i=4" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26526958&i=5" target="_top" rel="nofollow">java-user-help@...</a>
<br>>
<br>>
</div><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26526958&i=6" target="_top" rel="nofollow">java-user-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26526958&i=7" target="_top" rel="nofollow">java-user-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/Lucene---Java-Users-f45.html" embed="fixTarget[45]" target="_top" >Lucene - Java Users</a></p>