Request assistance with distributed search multi shard/core setup and configuration

[swjunke]

I've been looking through all the documentation.  I've set up a single solr instance, and one multicore instance.  If someone would be willing to share some configuration examples and/or advise for setting up solr for distributing the search, I would really appreciate it.  I've read that there is a way to do it, but most of the current documentation doesn't provide enough example on what to do with solr.xml, and the solrconfig.xml.  Also, I'm using tomcat 6 for the servlet container.  I deployed the solr 1.4.0 released yesterday.

Thanks
RJ

[Otis Gospodnetic]

RJ,

You may want to take a simpler step - single Solr core (no solr.xml needed) per machine.  Then distributed search really only requires that you specify shard URLs in the URL of the search requests.  In practice/production you rarely benefit from distributed search against multiple cores on the same server anyway.

Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR




________________________________
From: "Turner, Robbin J" <Robbin.J.Turner@...>
To: "solr-user@..." <solr-user@...>
Sent: Tue, November 10, 2009 5:58:52 PM
Subject: Request assistance with distributed search multi shard/core setup and configuration

I've been looking through all the documentation.  I've set up a single solr instance, and one multicore instance.  If someone would be willing to share some configuration examples and/or advise for setting up solr for distributing the search, I would really appreciate it.  I've read that there is a way to do it, but most of the current documentation doesn't provide enough example on what to do with solr.xml, and the solrconfig.xml.  Also, I'm using tomcat 6 for the servlet container.  I deployed the solr 1.4.0 released yesterday.

Thanks
RJ

[swjunke]

I've already done the single Solr, that's why my request.  I read on some site that there is a way to setup the configuration so I can send a query to one solr instance and have it pass it on or distribute it across all the instances?

Btw, thanks for the quick reply.
RJ

-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodnetic@...]
Sent: Tuesday, November 10, 2009 6:02 PM
To: solr-user@...
Subject: Re: Request assistance with distributed search multi shard/core setup and configuration

RJ,

You may want to take a simpler step - single Solr core (no solr.xml needed) per machine.  Then distributed search really only requires that you specify shard URLs in the URL of the search requests.  In practice/production you rarely benefit from distributed search against multiple cores on the same server anyway.

Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR




________________________________
From: "Turner, Robbin J" <Robbin.J.Turner@...>
To: "solr-user@..." <solr-user@...>
Sent: Tue, November 10, 2009 5:58:52 PM
Subject: Request assistance with distributed search multi shard/core setup and configuration

I've been looking through all the documentation.  I've set up a single solr instance, and one multicore instance.  If someone would be willing to share some configuration examples and/or advise for setting up solr for distributing the search, I would really appreciate it.  I've read that there is a way to do it, but most of the current documentation doesn't provide enough example on what to do with solr.xml, and the solrconfig.xml.  Also, I'm using tomcat 6 for the servlet container.  I deployed the solr 1.4.0 released yesterday.

Thanks
RJ

[Otis Gospodnetic]

Right, that's http://wiki.apache.org/solr/DistributedSearch

Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR



----- Original Message ----

> From: "Turner, Robbin J" <Robbin.J.Turner@...>
> To: "solr-user@..." <solr-user@...>
> Sent: Tue, November 10, 2009 6:05:19 PM
> Subject: RE: Request assistance with distributed search multi shard/core  setup and configuration
>
> I've already done the single Solr, that's why my request.  I read on some site
> that there is a way to setup the configuration so I can send a query to one solr
> instance and have it pass it on or distribute it across all the instances?
>
> Btw, thanks for the quick reply.
> RJ
>
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@...]
> Sent: Tuesday, November 10, 2009 6:02 PM
> To: solr-user@...
> Subject: Re: Request assistance with distributed search multi shard/core setup
> and configuration
>
> RJ,
>
> You may want to take a simpler step - single Solr core (no solr.xml needed) per
> machine.  Then distributed search really only requires that you specify shard
> URLs in the URL of the search requests.  In practice/production you rarely
> benefit from distributed search against multiple cores on the same server
> anyway.
>
> Otis
> --
> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
>
>
>
>
> ________________________________
> From: "Turner, Robbin J"
> To: "solr-user@..."
> Sent: Tue, November 10, 2009 5:58:52 PM
> Subject: Request assistance with distributed search multi shard/core setup and
> configuration
>
> I've been looking through all the documentation.  I've set up a single solr
> instance, and one multicore instance.  If someone would be willing to share some
> configuration examples and/or advise for setting up solr for distributing the
> search, I would really appreciate it.  I've read that there is a way to do it,
> but most of the current documentation doesn't provide enough example on what to
> do with solr.xml, and the solrconfig.xml.  Also, I'm using tomcat 6 for the
> servlet container.  I deployed the solr 1.4.0 released yesterday.
>
> Thanks
> RJ

[swjunke]

Thanks, I had already read through this url.  I guess my request was is there a way to setup something that is already part of solr itself to pass the URL[shard...] then having create a custom handler.

thanks

-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodnetic@...]
Sent: Tuesday, November 10, 2009 6:09 PM
To: solr-user@...
Subject: Re: Request assistance with distributed search multi shard/core setup and configuration

Right, that's http://wiki.apache.org/solr/DistributedSearch

Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR



----- Original Message ----

> From: "Turner, Robbin J" <Robbin.J.Turner@...>
> To: "solr-user@..." <solr-user@...>
> Sent: Tue, November 10, 2009 6:05:19 PM
> Subject: RE: Request assistance with distributed search multi
> shard/core  setup and configuration
>
> I've already done the single Solr, that's why my request.  I read on
> some site that there is a way to setup the configuration so I can send
> a query to one solr instance and have it pass it on or distribute it across all the instances?
>
> Btw, thanks for the quick reply.
> RJ
>
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@...]
> Sent: Tuesday, November 10, 2009 6:02 PM
> To: solr-user@...
> Subject: Re: Request assistance with distributed search multi
> shard/core setup and configuration
>
> RJ,
>
> You may want to take a simpler step - single Solr core (no solr.xml
> needed) per machine.  Then distributed search really only requires
> that you specify shard URLs in the URL of the search requests.  In
> practice/production you rarely benefit from distributed search against
> multiple cores on the same server anyway.
>
> Otis
> --
> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
>
>
>
>
> ________________________________
> From: "Turner, Robbin J"
> To: "solr-user@..."
> Sent: Tue, November 10, 2009 5:58:52 PM
> Subject: Request assistance with distributed search multi shard/core
> setup and configuration
>
> I've been looking through all the documentation.  I've set up a single
> solr instance, and one multicore instance.  If someone would be
> willing to share some configuration examples and/or advise for setting
> up solr for distributing the search, I would really appreciate it.  
> I've read that there is a way to do it, but most of the current
> documentation doesn't provide enough example on what to do with
> solr.xml, and the solrconfig.xml.  Also, I'm using tomcat 6 for the servlet container.  I deployed the solr 1.4.0 released yesterday.
>
> Thanks
> RJ

[Otis Gospodnetic]

Hm, I don't follow.  You don't need to create a custom (request) handler to make use of Solr's distributed search.

Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR



----- Original Message ----

> From: "Turner, Robbin J" <Robbin.J.Turner@...>
> To: "solr-user@..." <solr-user@...>
> Sent: Tue, November 10, 2009 6:41:32 PM
> Subject: RE: Request assistance with distributed search multi shard/core   setup and configuration
>
> Thanks, I had already read through this url.  I guess my request was is there a
> way to setup something that is already part of solr itself to pass the
> URL[shard...] then having create a custom handler.
>
> thanks
>
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@...]
> Sent: Tuesday, November 10, 2009 6:09 PM
> To: solr-user@...
> Subject: Re: Request assistance with distributed search multi shard/core setup
> and configuration
>
> Right, that's http://wiki.apache.org/solr/DistributedSearch
>
> Otis
> --
> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
>
>
>
> ----- Original Message ----
> > From: "Turner, Robbin J"
> > To: "solr-user@..."
> > Sent: Tue, November 10, 2009 6:05:19 PM
> > Subject: RE: Request assistance with distributed search multi
> > shard/core  setup and configuration
> >
> > I've already done the single Solr, that's why my request.  I read on
> > some site that there is a way to setup the configuration so I can send
> > a query to one solr instance and have it pass it on or distribute it across
> all the instances?
> >
> > Btw, thanks for the quick reply.
> > RJ
> >
> > -----Original Message-----
> > From: Otis Gospodnetic [mailto:otis_gospodnetic@...]
> > Sent: Tuesday, November 10, 2009 6:02 PM
> > To: solr-user@...
> > Subject: Re: Request assistance with distributed search multi
> > shard/core setup and configuration
> >
> > RJ,
> >
> > You may want to take a simpler step - single Solr core (no solr.xml
> > needed) per machine.  Then distributed search really only requires
> > that you specify shard URLs in the URL of the search requests.  In
> > practice/production you rarely benefit from distributed search against
> > multiple cores on the same server anyway.
> >
> > Otis
> > --
> > Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> > Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
> >
> >
> >
> >
> > ________________________________
> > From: "Turner, Robbin J"
> > To: "solr-user@..."
> > Sent: Tue, November 10, 2009 5:58:52 PM
> > Subject: Request assistance with distributed search multi shard/core
> > setup and configuration
> >
> > I've been looking through all the documentation.  I've set up a single
> > solr instance, and one multicore instance.  If someone would be
> > willing to share some configuration examples and/or advise for setting
> > up solr for distributing the search, I would really appreciate it.  
> > I've read that there is a way to do it, but most of the current
> > documentation doesn't provide enough example on what to do with
> > solr.xml, and the solrconfig.xml.  Also, I'm using tomcat 6 for the servlet
> container.  I deployed the solr 1.4.0 released yesterday.
> >
> > Thanks
> > RJ

[Lance Norskog-2]

DS requires a bunch of shard names in the url. That's all. Note that a
ds does not use the data of the solr you call.

You can create an entry point for your distributed search by adding a
new <requestHandler> element in solrconfig.xml. You would add the
shard list parameter to the "defaults" list. Do not have it call the
same requesthandler path- you'll get an infinite loop.

On Tue, Nov 10, 2009 at 6:44 PM, Otis Gospodnetic
<otis_gospodnetic@...> wrote:

> Hm, I don't follow.  You don't need to create a custom (request) handler to make use of Solr's distributed search.
>
> Otis
> --
> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
>
>
>
> ----- Original Message ----
>> From: "Turner, Robbin J" <Robbin.J.Turner@...>
>> To: "solr-user@..." <solr-user@...>
>> Sent: Tue, November 10, 2009 6:41:32 PM
>> Subject: RE: Request assistance with distributed search multi shard/core   setup and configuration
>>
>> Thanks, I had already read through this url.  I guess my request was is there a
>> way to setup something that is already part of solr itself to pass the
>> URL[shard...] then having create a custom handler.
>>
>> thanks
>>
>> -----Original Message-----
>> From: Otis Gospodnetic [mailto:otis_gospodnetic@...]
>> Sent: Tuesday, November 10, 2009 6:09 PM
>> To: solr-user@...
>> Subject: Re: Request assistance with distributed search multi shard/core setup
>> and configuration
>>
>> Right, that's http://wiki.apache.org/solr/DistributedSearch
>>
>> Otis
>> --
>> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
>> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
>>
>>
>>
>> ----- Original Message ----
>> > From: "Turner, Robbin J"
>> > To: "solr-user@..."
>> > Sent: Tue, November 10, 2009 6:05:19 PM
>> > Subject: RE: Request assistance with distributed search multi
>> > shard/core  setup and configuration
>> >
>> > I've already done the single Solr, that's why my request.  I read on
>> > some site that there is a way to setup the configuration so I can send
>> > a query to one solr instance and have it pass it on or distribute it across
>> all the instances?
>> >
>> > Btw, thanks for the quick reply.
>> > RJ
>> >
>> > -----Original Message-----
>> > From: Otis Gospodnetic [mailto:otis_gospodnetic@...]
>> > Sent: Tuesday, November 10, 2009 6:02 PM
>> > To: solr-user@...
>> > Subject: Re: Request assistance with distributed search multi
>> > shard/core setup and configuration
>> >
>> > RJ,
>> >
>> > You may want to take a simpler step - single Solr core (no solr.xml
>> > needed) per machine.  Then distributed search really only requires
>> > that you specify shard URLs in the URL of the search requests.  In
>> > practice/production you rarely benefit from distributed search against
>> > multiple cores on the same server anyway.
>> >
>> > Otis
>> > --
>> > Sematext is hiring -- http://sematext.com/about/jobs.html?mls
>> > Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
>> >
>> >
>> >
>> >
>> > ________________________________
>> > From: "Turner, Robbin J"
>> > To: "solr-user@..."
>> > Sent: Tue, November 10, 2009 5:58:52 PM
>> > Subject: Request assistance with distributed search multi shard/core
>> > setup and configuration
>> >
>> > I've been looking through all the documentation.  I've set up a single
>> > solr instance, and one multicore instance.  If someone would be
>> > willing to share some configuration examples and/or advise for setting
>> > up solr for distributing the search, I would really appreciate it.
>> > I've read that there is a way to do it, but most of the current
>> > documentation doesn't provide enough example on what to do with
>> > solr.xml, and the solrconfig.xml.  Also, I'm using tomcat 6 for the servlet
>> container.  I deployed the solr 1.4.0 released yesterday.
>> >
>> > Thanks
>> > RJ
>
>

--
Lance Norskog
goksron@...