Accessing an Index from a shared location

[JusteAvantToi]

Hi all,

I am new on using Nutch and I found that Nutch is really good. I have a problem and hope somebody can shed a light.

I have built an index and a web application that makes use of that index. I plan to have two web application servers running the application. Since I do not want to replicate the application and the index  on each web application server, I put the application and the index on a shared location and configure nutch-site.xml as follow:

<property>
  <name>searcher.dir</name>
  <value>\\111.111.111.111\folder\index</value>
  <description> Path to root of crawl</description>
</property>

<property>
  <name>plugin.folders</name>
  <value>\\111.111.111.111\folder\plugins</value><description>
</property>

However it seems that my application can not find the index. I have checked that the web application server have access to the shared location.

Is there something that I missed here? Does Nutch allow us to put the index on a network location?

Thank you.


JustAvantToi

[Andrzej Bialecki]

JusteAvantToi wrote:

> Hi all,
>
> I am new on using Nutch and I found that Nutch is really good. I have a
> problem and hope somebody can shed a light.
>
> I have built an index and a web application that makes use of that index. I
> plan to have two web application servers running the application. Since I do
> not want to replicate the application and the index  on each web application
> server, I put the application and the index on a shared location and
> configure nutch-site.xml as follow:
>
> <property>
>   <name>searcher.dir</name>
>   <value>\\111.111.111.111\folder\index</value>
>   <description> Path to root of crawl</description>
> </property>
>
> <property>
>   <name>plugin.folders</name>
>   <value>\\111.111.111.111\folder\plugins</value><description>
> </property>
>
> However it seems that my application can not find the index. I have checked
> that the web application server have access to the shared location.
>
> Is there something that I missed here? Does Nutch allow us to put the index
> on a network location?

UNC paths are not supported in Java - you need to mount this location as
a local volume.


--
Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com

[JusteAvantToi]

Andrzej Bialecki wrote:

JusteAvantToi wrote:
> Hi all,
>
> I am new on using Nutch and I found that Nutch is really good. I have a
> problem and hope somebody can shed a light.
>
> I have built an index and a web application that makes use of that index. I
> plan to have two web application servers running the application. Since I do
> not want to replicate the application and the index  on each web application
> server, I put the application and the index on a shared location and
> configure nutch-site.xml as follow:
>
> <property>
>   <name>searcher.dir</name>
>   <value>\\111.111.111.111\folder\index</value>
>   <description> Path to root of crawl</description>
> </property>
>
> <property>
>   <name>plugin.folders</name>
>   <value>\\111.111.111.111\folder\plugins</value><description>
> </property>
>
> However it seems that my application can not find the index. I have checked
> that the web application server have access to the shared location.
>
> Is there something that I missed here? Does Nutch allow us to put the index
> on a network location?

UNC paths are not supported in Java - you need to mount this location as
a local volume.


--
Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com

Hi Andrzej,

Thanks for your reply. I created a simple Java program to list all file in this path \\xxx.xxx.xxx.xxx\temp, and the program is able to list down all files in that directory. So it is able to interpret the UNC path.

So I am getting confused now. Any pointer is greatly appreciated.

Thanks,

JusteAvantToi