|
»
»
»
Index files not deleted after optimization
|
View:
New views
7 Messages
—
Rating Filter:
Alert me
|
 |
Index files not deleted after optimization
by Ganesh - yahoo
::
Rate this Message:
Hello all,
I am using Lucene 2.4.1 and My app is running inside Tomcat. In Windows, after database optimization, the old db files are not getting deleted. I enabled the info stream and found the below entries. I used ProcessExplorer from SysInternals to view the lock file, but old db files are not locked by any process. Even rename of those files are successful. If i stop the tomcat, old db files are deleted. I could see only the optimized segment files. The same code works fine in Linux. I am facing this issue in Windows XP and 2003. IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.frq": java.io.IOException: Cannot delete <FolderName>\_0.frq>; Will re-try later. IFD [FLUSH]: delete "_0.prx" IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.prx": java.io.IOException: Cannot delete <FolderName>\_0.prx; Will re-try later. IFD [FLUSH]: delete "_0.tis" IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.tis": java.io.IOException: Cannot delete <FolderName>\_0.tis; Will re-try later. Regards Ganesh Send instant messages to your online friends http://in.messenger.yahoo.com --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@... For additional commands, e-mail: java-user-help@... |
|
|
Re: Index files not deleted after optimization
by Michael McCandless-2
::
Rate this Message:
Something must still have these file handles open at the time the
optimization completed. EG do you have a reader open on this index? Mike On Mon, Nov 2, 2009 at 6:54 AM, Ganesh <emailgane@...> wrote: > Hello all, > > I am using Lucene 2.4.1 and My app is running inside Tomcat. > > In Windows, after database optimization, the old db files are not getting deleted. I enabled the info stream and found the below entries. I used ProcessExplorer from SysInternals to view the lock file, but old db files are not locked by any process. Even rename of those files are successful. If i stop the tomcat, old db files are deleted. I could see only the optimized segment files. The same code works fine in Linux. I am facing this issue in Windows XP and 2003. > > IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.frq": java.io.IOException: Cannot delete <FolderName>\_0.frq>; Will re-try later. > IFD [FLUSH]: delete "_0.prx" > IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.prx": java.io.IOException: Cannot delete <FolderName>\_0.prx; Will re-try later. > IFD [FLUSH]: delete "_0.tis" > IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.tis": java.io.IOException: Cannot delete <FolderName>\_0.tis; Will re-try later. > > Regards > Ganesh > > Send instant messages to your online friends http://in.messenger.yahoo.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscribe@... > For additional commands, e-mail: java-user-help@... > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@... For additional commands, e-mail: java-user-help@... |
|
|
Re: Index files not deleted after optimization
by Ganesh - yahoo
::
Rate this Message:
I am reopening the reader and closing the old one. I am not facing this issue in Linux otherwise Linux will show <filename>(deleted) under /proc/pid/fd/.
searcher.getIndexReader().close(); searcher.close(); Regards Ganesh ----- Original Message ----- From: "Michael McCandless" <lucene@...> To: <java-user@...> Sent: Monday, November 02, 2009 6:03 PM Subject: Re: Index files not deleted after optimization Something must still have these file handles open at the time the optimization completed. EG do you have a reader open on this index? Mike On Mon, Nov 2, 2009 at 6:54 AM, Ganesh <emailgane@...> wrote: > Hello all, > > I am using Lucene 2.4.1 and My app is running inside Tomcat. > > In Windows, after database optimization, the old db files are not getting deleted. I enabled the info stream and found the below entries. I used ProcessExplorer from SysInternals to view the lock file, but old db files are not locked by any process. Even rename of those files are successful. If i stop the tomcat, old db files are deleted. I could see only the optimized segment files. The same code works fine in Linux. I am facing this issue in Windows XP and 2003. > > IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.frq": java.io.IOException: Cannot delete <FolderName>\_0.frq>; Will re-try later. > IFD [FLUSH]: delete "_0.prx" > IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.prx": java.io.IOException: Cannot delete <FolderName>\_0.prx; Will re-try later. > IFD [FLUSH]: delete "_0.tis" > IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.tis": java.io.IOException: Cannot delete <FolderName>\_0.tis; Will re-try later. > > Regards > Ganesh > > Send instant messages to your online friends http://in.messenger.yahoo.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscribe@... > For additional commands, e-mail: java-user-help@... > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@... For additional commands, e-mail: java-user-help@... Send instant messages to your online friends http://in.messenger.yahoo.com --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@... For additional commands, e-mail: java-user-help@... |
|
|
Re: Index files not deleted after optimization
by Michael McCandless-2
::
Rate this Message:
It depends on the relative timing.
If the old IndexReader is still open when the optimize completes then the files it has open cannot be deleted. But, if that IndexReader hadn't been reopened in a while, it's possible it did not in fact have the files of the just-merged segments open (in which case the delete would succeed). Mike On Tue, Nov 3, 2009 at 4:17 AM, Ganesh <emailgane@...> wrote: > I am reopening the reader and closing the old one. I am not facing this issue in Linux otherwise Linux will show <filename>(deleted) under /proc/pid/fd/. > > searcher.getIndexReader().close(); > searcher.close(); > > Regards > Ganesh > > ----- Original Message ----- > From: "Michael McCandless" <lucene@...> > To: <java-user@...> > Sent: Monday, November 02, 2009 6:03 PM > Subject: Re: Index files not deleted after optimization > > > Something must still have these file handles open at the time the > optimization completed. > > EG do you have a reader open on this index? > > Mike > > On Mon, Nov 2, 2009 at 6:54 AM, Ganesh <emailgane@...> wrote: >> Hello all, >> >> I am using Lucene 2.4.1 and My app is running inside Tomcat. >> >> In Windows, after database optimization, the old db files are not getting deleted. I enabled the info stream and found the below entries. I used ProcessExplorer from SysInternals to view the lock file, but old db files are not locked by any process. Even rename of those files are successful. If i stop the tomcat, old db files are deleted. I could see only the optimized segment files. The same code works fine in Linux. I am facing this issue in Windows XP and 2003. >> >> IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.frq": java.io.IOException: Cannot delete <FolderName>\_0.frq>; Will re-try later. >> IFD [FLUSH]: delete "_0.prx" >> IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.prx": java.io.IOException: Cannot delete <FolderName>\_0.prx; Will re-try later. >> IFD [FLUSH]: delete "_0.tis" >> IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.tis": java.io.IOException: Cannot delete <FolderName>\_0.tis; Will re-try later. >> >> Regards >> Ganesh >> >> Send instant messages to your online friends http://in.messenger.yahoo.com >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscribe@... >> For additional commands, e-mail: java-user-help@... >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscribe@... > For additional commands, e-mail: java-user-help@... > > Send instant messages to your online friends http://in.messenger.yahoo.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscribe@... > For additional commands, e-mail: java-user-help@... > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@... For additional commands, e-mail: java-user-help@... |
|
|
Re: Index files not deleted after optimization
by Ganesh - yahoo
::
Rate this Message:
My IndexReader and Searcher is open all the time. I am reopening it at constant interval.
Below are the code sequence. 1. DB optimize 2. Close writer 3. Open writer 4. Reopen new reader 5. Close old reader 6. Close old searcher. >>If the old IndexReader is still open when the optimize completes then the files it has open cannot be deleted. If reader should not be opened while doing optimize then User cannot perform search during the time period. Regards Ganesh ----- Original Message ----- From: "Michael McCandless" <lucene@...> To: <java-user@...> Sent: Tuesday, November 03, 2009 3:22 PM Subject: Re: Index files not deleted after optimization It depends on the relative timing. If the old IndexReader is still open when the optimize completes then the files it has open cannot be deleted. But, if that IndexReader hadn't been reopened in a while, it's possible it did not in fact have the files of the just-merged segments open (in which case the delete would succeed). Mike On Tue, Nov 3, 2009 at 4:17 AM, Ganesh <emailgane@...> wrote: > I am reopening the reader and closing the old one. I am not facing this issue in Linux otherwise Linux will show <filename>(deleted) under /proc/pid/fd/. > > searcher.getIndexReader().close(); > searcher.close(); > > Regards > Ganesh > > ----- Original Message ----- > From: "Michael McCandless" <lucene@...> > To: <java-user@...> > Sent: Monday, November 02, 2009 6:03 PM > Subject: Re: Index files not deleted after optimization > > > Something must still have these file handles open at the time the > optimization completed. > > EG do you have a reader open on this index? > > Mike > > On Mon, Nov 2, 2009 at 6:54 AM, Ganesh <emailgane@...> wrote: >> Hello all, >> >> I am using Lucene 2.4.1 and My app is running inside Tomcat. >> >> In Windows, after database optimization, the old db files are not getting deleted. I enabled the info stream and found the below entries. I used ProcessExplorer from SysInternals to view the lock file, but old db files are not locked by any process. Even rename of those files are successful. If i stop the tomcat, old db files are deleted. I could see only the optimized segment files. The same code works fine in Linux. I am facing this issue in Windows XP and 2003. >> >> IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.frq": java.io.IOException: Cannot delete <FolderName>\_0.frq>; Will re-try later. >> IFD [FLUSH]: delete "_0.prx" >> IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.prx": java.io.IOException: Cannot delete <FolderName>\_0.prx; Will re-try later. >> IFD [FLUSH]: delete "_0.tis" >> IFD [FLUSH]: IndexFileDeleter: unable to remove file "_0.tis": java.io.IOException: Cannot delete <FolderName>\_0.tis; Will re-try later. >> >> Regards >> Ganesh >> >> Send instant messages to your online friends http://in.messenger.yahoo.com >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscribe@... >> For additional commands, e-mail: java-user-help@... >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscribe@... > For additional commands, e-mail: java-user-help@... > > Send instant messages to your online friends http://in.messenger.yahoo.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscribe@... > For additional commands, e-mail: java-user-help@... > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@... For additional commands, e-mail: java-user-help@... Send instant messages to your online friends http://in.messenger.yahoo.com --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@... For additional commands, e-mail: java-user-help@... |
|
|
Re: Index files not deleted after optimization
by Michael McCandless-2
::
Rate this Message:
On Tue, Nov 3, 2009 at 9:45 AM, Ganesh <emailgane@...> wrote:
> My IndexReader and Searcher is open all the time. I am reopening it at constant interval. > > Below are the code sequence. > > 1. DB optimize > 2. Close writer > 3. Open writer > 4. Reopen new reader > 5. Close old reader > 6. Close old searcher. Assuming there is an implicit: 0. Open a reader/searcher Then it's expected after step 6 that you have leftover files, because the writers in step 2 and 3 were unable to delete the files since the reader from step 0 still had them open. Once you open a new writer, or commit any changes with the existing writer, the files will be deleted. >>>If the old IndexReader is still open when the optimize completes then the files it has open cannot be deleted. > If reader should not be opened while doing optimize then User cannot perform search during the time period. It's fine to leave a reader open... it just means the old files will persist until the writer is able to delete them. Mike --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@... For additional commands, e-mail: java-user-help@... |
|
|
Re: Index files not deleted after optimization
by Ganesh - yahoo
::
Rate this Message:
Thanks Michael.
Below order helped to solve the issue, 1. DB optimize 2. DB commit 3. Reopen new reader 4. Close old reader 5. Close writer 6. Open writer Regards Ganesh ----- Original Message ----- From: "Michael McCandless" <lucene@...> To: <java-user@...> Sent: Tuesday, November 03, 2009 9:05 PM Subject: Re: Index files not deleted after optimization > On Tue, Nov 3, 2009 at 9:45 AM, Ganesh <emailgane@...> wrote: >> My IndexReader and Searcher is open all the time. I am reopening it at constant interval. >> >> Below are the code sequence. >> >> 1. DB optimize >> 2. Close writer >> 3. Open writer >> 4. Reopen new reader >> 5. Close old reader >> 6. Close old searcher. > > Assuming there is an implicit: > > 0. Open a reader/searcher > > Then it's expected after step 6 that you have leftover files, because > the writers in step 2 and 3 were unable to delete the files since the > reader from step 0 still had them open. Once you open a new writer, > or commit any changes with the existing writer, the files will be > deleted. > >>>>If the old IndexReader is still open when the optimize completes then the files it has open cannot be deleted. >> If reader should not be opened while doing optimize then User cannot perform search during the time period. > > It's fine to leave a reader open... it just means the old files will > persist until the writer is able to delete them. > > Mike > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscribe@... > For additional commands, e-mail: java-user-help@... > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@... For additional commands, e-mail: java-user-help@... |
| Free embeddable forum powered by Nabble | Forum Help |
