Sv: Re: Facets

[Henrik Hjalmarsson-2]

So basicly there is no effective way of doing this?
The only solution I've come up with is

Pseudo code:

for( every index )
{
    create term(index, "*")
    create wildcard query with term
    rewrite query to primitive query
    extract terms from primive query
    for( each term extracted )
    {
        create query with term and query string from request
    }
}

for( each query created above )
{
    search with query asking for 1 result.
    check total number of hits for query
}

print result

Something like that. Is that a totaly idiotic way of doing it?


>>> Chris Lu  09-11-04 02:51 >>>
If the query is a very selective one, you can go through the XML
document and do the counting.

If the query is not so selective, which is usually the case, and the
number of matches are large, basically all the values need to be loaded
into memory, or solid state disk, to do a fast counting.

--
Chris Lu
-------------------------
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 minutes: http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
DBSight customer, a shopping comparison site, (anonymous per request) got 2.6 Million Euro funding!


Henrik Hjalmarsson wrote:

> Hello
>  
> I am trying to develop an API for a search application that is using Lucene 2.4.1
> The search application is maintained by RAA (swedish goverment organization that keeps track of historical and cultural data).
>  
> I have gotten a demand for an API method that returns an XML response, listing all the indexes in this application and the number of unique values these indexes can have, filtered by a query that is recieved in the method request.
>  
> The application contains a large amount of indexes and some indexes contains a very large amount of unique values. Is there some way to achive this in an effective way?
>  
> With regards Henrik
>
>  

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@...
For additional commands, e-mail: java-user-help@...

[Jake Mannix]

Can you describe your lucene index a little further?  How many documents are
in it, and
what do you mean by "listing all the indexes in this application and the
number of unique
values these indexes can have, filtered by a query that is recieved in the
method request."

Index means what in this context - it's a pretty overloaded word.  What you
describe in this
pseudocode does not look like it will perform *at all*, so to help you get
this to go faster,
I need to understand what exactly it is you're trying to accomplish - you
have an incoming
query "q" which you want to get facet results filtered by (it's sometimes
call the "driving query"
in the faceting problem), and documents have a field, called "index", which
is what, single
valued, an integer, and you effectively want the count of (q AND index:x)
for all possible
unique values of x (this is the basic faceting problem), or what?

If this is what you want to do, both Solr and bobo-browse, which are both
open source and
built on top of lucene, can do this query fairly performantly for you.

  -jake

On Thu, Nov 5, 2009 at 7:37 AM, Henrik Hjalmarsson <
henrik.hjalmarsson@...> wrote:

> So basicly there is no effective way of doing this?
> The only solution I've come up with is
>
> Pseudo code:
>
> for( every index )
> {
>    create term(index, "*")
>    create wildcard query with term
>    rewrite query to primitive query
>    extract terms from primive query
>    for( each term extracted )
>    {
>        create query with term and query string from request
>    }
> }
>
> for( each query created above )
> {
>    search with query asking for 1 result.
>    check total number of hits for query
> }
>
> print result
>
> Something like that. Is that a totaly idiotic way of doing it?
>
>
> >>> Chris Lu  09-11-04 02:51 >>>
> If the query is a very selective one, you can go through the XML
> document and do the counting.
>
> If the query is not so selective, which is usually the case, and the
> number of matches are large, basically all the values need to be loaded
> into memory, or solid state disk, to do a fast counting.
>
> --
> Chris Lu
> -------------------------
> Instant Scalable Full-Text Search On Any Database/Application
> site: http://www.dbsight.net
> demo: http://search.dbsight.com
> Lucene Database Search in 3 minutes:
> http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
> DBSight customer, a shopping comparison site, (anonymous per request) got
> 2.6 Million Euro funding!
>
>
> Henrik Hjalmarsson wrote:
> > Hello
> >
> > I am trying to develop an API for a search application that is using
> Lucene 2.4.1
> > The search application is maintained by RAA (swedish goverment
> organization that keeps track of historical and cultural data).
> >
> > I have gotten a demand for an API method that returns an XML response,
> listing all the indexes in this application and the number of unique values
> these indexes can have, filtered by a query that is recieved in the method
> request.
> >
> > The application contains a large amount of indexes and some indexes
> contains a very large amount of unique values. Is there some way to achive
> this in an effective way?
> >
> > With regards Henrik
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@...
> For additional commands, e-mail: java-user-help@...
>
>
>