Highlighting is very slow

[Andrew Clegg]

Hi everyone,

I'm experimenting with highlighting for the first time, and it seems shockingly slow for some queries.

For example, this query:

http://server:8080/solr/select/?q=transferase&qt=dismax&version=2.2&start=0&rows=10&indent=on

takes 313ms. But when I add highlighting:

http://server:8080/solr/select/?q=transferase&qt=dismax&version=2.2&start=0&rows=10&indent=on&hl=true&hl.fl=*&fl=id

it takes 305212ms = 5mins!

Some of my documents are slightly large -- the 10 hits for that query contain between 362 bytes and 1.4 megabytes of text each. All fields are stored and indexed, and most are termvectored. But this doesn't seem excessively large!

Has anyone else seen this sort of behaviour before? This is with a nightly from 2009-10-26.

All suggestions would be appreciated. My schema and config files are attached...

schema.xml
solrconfig.xml

Thanks (once again),

Andrew.

[Jaco-4]

Hi,

We had a similar case once (although not with those really long response
times). Fixed by moving to JRE 1.6 and tuning garbage collection.

Bye,

Jaco.

2009/11/3 Andrew Clegg <andrew.clegg@...>

>
> Hi everyone,
>
> I'm experimenting with highlighting for the first time, and it seems
> shockingly slow for some queries.
>
> For example, this query:
>
>
> http://server:8080/solr/select/?q=transferase&qt=dismax&version=2.2&start=0&rows=10&indent=on
>
> takes 313ms. But when I add highlighting:
>
>
> http://server:8080/solr/select/?q=transferase&qt=dismax&version=2.2&start=0&rows=10&indent=on&hl=true&hl.fl=*&fl=id
>
> it takes 305212ms = 5mins!
>
> Some of my documents are slightly large -- the 10 hits for that query
> contain between 362 bytes and 1.4 megabytes of text each. All fields are
> stored and indexed, and most are termvectored. But this doesn't seem
> excessively large!
>
> Has anyone else seen this sort of behaviour before? This is with a nightly
> from 2009-10-26.
>
> All suggestions would be appreciated. My schema and config files are
> attached...
>
> http://old.nabble.com/file/p26160217/schema.xml schema.xml
> http://old.nabble.com/file/p26160217/solrconfig.xml solrconfig.xml
>
> Thanks (once again),
>
> Andrew.
>
> --
> View this message in context:
> http://old.nabble.com/Highlighting-is-very-slow-tp26160217p26160217.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>

[Andrew Clegg]

We're on 1.6 already. Any chance you could share your GC settings?

Thanks,

Andrew.

PS apologies for the duplicate message yesterday, Nabble threw an exception when I posted the first one. And the second one actually.

Jaco-4 wrote:

Hi,

We had a similar case once (although not with those really long response
times). Fixed by moving to JRE 1.6 and tuning garbage collection.

Bye,

Jaco.

2009/11/3 Andrew Clegg <[e-mail]>

>
> Hi everyone,
>
> I'm experimenting with highlighting for the first time, and it seems
> shockingly slow for some queries.
>
> For example, this query:
>
>
> http://server:8080/solr/select/?q=transferase&qt=dismax&version=2.2&start=0&rows=10&indent=on
>
> takes 313ms. But when I add highlighting:
>
>
> http://server:8080/solr/select/?q=transferase&qt=dismax&version=2.2&start=0&rows=10&indent=on&hl=true&hl.fl=*&fl=id
>
> it takes 305212ms = 5mins!
>
> Some of my documents are slightly large -- the 10 hits for that query
> contain between 362 bytes and 1.4 megabytes of text each. All fields are
> stored and indexed, and most are termvectored. But this doesn't seem
> excessively large!
>
> Has anyone else seen this sort of behaviour before? This is with a nightly
> from 2009-10-26.
>
> All suggestions would be appreciated. My schema and config files are
> attached...
>
> http://old.nabble.com/file/p26160217/schema.xml schema.xml
> http://old.nabble.com/file/p26160217/solrconfig.xml solrconfig.xml
>
> Thanks (once again),
>
> Andrew.
>
> --
> View this message in context:
> http://old.nabble.com/Highlighting-is-very-slow-tp26160217p26160217.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>