How to get rid of unused fields?

[Khin, Gerald]

Hi folks!

 

After deleting documents from the index it can happen that fields become
unused (i.e. no document has this field anymore). And
IndexReader.getFieldNames() still returns these unused fields, even
after optimizing the index. Is there any chance to get rid of these
unused fields?

 

Cheers,

Gerald

 

[hossman]

: After deleting documents from the index it can happen that fields become
: unused (i.e. no document has this field anymore). And
: IndexReader.getFieldNames() still returns these unused fields, even
: after optimizing the index. Is there any chance to get rid of these
: unused fields?

that's odd ... i'm 99% sure that should work (optimizng should perge
unused fields).  

what version of Lucene are you using?  (if you're on the trunk it might be
a bug introduced recently with some of the indexing changes)

Can you post some test code demonstrating the problem?



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@...
For additional commands, e-mail: java-user-help@...

[Uwe Schindler]

The same here, even with trunk from yesterday. If you create a field, it
stays there forever, even after deleting *all* documents from index,
reindexing without the field and optimizing.

Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@...

> -----Original Message-----
> From: Chris Hostetter [mailto:hossman_lucene@...]
> Sent: Friday, July 17, 2009 12:39 AM
> To: java-user@...
> Subject: Re: How to get rid of unused fields?
>
>
> : After deleting documents from the index it can happen that fields become
> : unused (i.e. no document has this field anymore). And
> : IndexReader.getFieldNames() still returns these unused fields, even
> : after optimizing the index. Is there any chance to get rid of these
> : unused fields?
>
> that's odd ... i'm 99% sure that should work (optimizng should perge
> unused fields).
>
> what version of Lucene are you using?  (if you're on the trunk it might be
> a bug introduced recently with some of the indexing changes)
>
> Can you post some test code demonstrating the problem?
>
>
>
> -Hoss
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@...
> For additional commands, e-mail: java-user-help@...

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@...
For additional commands, e-mail: java-user-help@...

[hossman]

: The same here, even with trunk from yesterday. If you create a field, it
: stays there forever, even after deleting *all* documents from index,
: reindexing without the field and optimizing.

Uwe: if you have a quick test case already written can you try it against
2.4 (and maybe 2.3) because i'm pretty sure that use to work.

(the reason i'm confident is because once upon a time, before solr
implemented special logic to make "<delete><query>*:*</query></delete>"
explicitly removing the index directory the FAQ on changing schemas told
people to make sure to optimize before reindexing so the field stats would
be removed.)


-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@...
For additional commands, e-mail: java-user-help@...

[hossman]

: : The same here, even with trunk from yesterday. If you create a field, it
: : stays there forever, even after deleting *all* documents from index,
: : reindexing without the field and optimizing.
:
: Uwe: if you have a quick test case already written can you try it against
: 2.4 (and maybe 2.3) because i'm pretty sure that use to work.

i just tested 2.4, 2.3, and 2.2 .. sure enough you can't seem to get rid
of fields even when you delete all docs.

this seems like a bad thing.  so i'm oging to open an issue and hope that
the big brains of Index Internals can see an easy way to make this work.



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@...
For additional commands, e-mail: java-user-help@...