Spell check suggestion and correct way of implementation and some Questions

View: New views
6 Messages — Rating Filter:   Alert me  

Spell check suggestion and correct way of implementation and some Questions

by darniz :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Solr masters,

I just wanted to confirm what i read is the best and optimal way of doing spell check correction.
i am using index based spell check.
For this as the doc says
Create a field ex. my_text with a fieldType my_text_fl which has all the filters defined.

Copy all the othe fields data to the field my_text using copy features.

In the solr config file in the searchComponent which is spellcheck add the below code

  <searchComponent name="spellcheck" class="solr.SpellCheckComponent">

    <lst name="spellchecker">
      <str name="name">mySpellChecker</str>
      <str name="field">spell_text</str>
      <!-- Use a different Distance Measure -->
      <str name="distanceMeasure">org.apache.lucene.search.spell.JaroWinklerDistance</str>
      <str name="spellcheckIndexDir">./mySpellCheckerDataIndex</str>
    </lst>
  </searchComponent>

build the dictionary using the get command

Question. Should i build the dictionlary only once and after that as new words are indexed the dictionary will be updated. Or i to do that manually over certain interval.


add the spellcheck component to the handler in my case as of now standard requets handler. I might also start adding some more dismax handlers depending on my requirement
  <requestHandler name="standard" class="solr.SearchHandler" default="true">
    <!-- default values for query parameters -->
     <lst name="defaults">
       <str name="echoParams">explicit</str>
       <!--
       <int name="rows">10</int>
       <str name="fl">*</str>
       <str name="version">2.1</str>
        -->
     </lst>
     <arr name="last-components">
        <str>spellcheck</str>
     </arr>
  </requestHandler>

run the query with parameter spell.check=true, and also specify against which dictionary you want to run spell check again in my case my spellcheck.dictionary parameter is mySpellChecker.



Please let know if this the best way since i have to start implementing it.

thanks
darniz




Re: Spell check suggestion and correct way of implementation and some Questions

by Shalin Shekhar Mangar :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Wed, Oct 28, 2009 at 8:57 PM, darniz <rnizamuddin@...> wrote:

>
> Question. Should i build the dictionlary only once and after that as new
> words are indexed the dictionary will be updated. Or i to do that manually
> over certain interval.
>
>
No. The dictionary is built only when spellcheck.build=true is specified as
a request parameter. You will need to explicitly send spellcheck.build=true
again when the document changes or you can use the buildOnCommit or
buildOnOptimize parameters to re-build the index automatically.

http://wiki.apache.org/solr/SpellCheckComponent#Building_on_Commits


>
> add the spellcheck component to the handler in my case as of now standard
> requets handler. I might also start adding some more dismax handlers
> depending on my requirement
>  <requestHandler name="standard" class="solr.SearchHandler" default="true">
>    <!-- default values for query parameters -->
>     <lst name="defaults">
>       <str name="echoParams">explicit</str>
>       <!--
>       <int name="rows">10</int>
>       <str name="fl">*</str>
>       <str name="version">2.1</str>
>        -->
>     </lst>
>     <arr name="last-components">
>        <str>spellcheck</str>
>     </arr>
>  </requestHandler>
>
> run the query with parameter spell.check=true, and also specify against
> which dictionary you want to run spell check again in my case my
> spellcheck.dictionary parameter is mySpellChecker.
>
>
The parameter is spellcheck=true not spell.check=true. If you do not give a
name to your dictionary then you do not need to add the
spellcheck.dictionary parameter.

--
Regards,
Shalin Shekhar Mangar.

Re: Spell check suggestion and correct way of implementation and some Questions

by darniz :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello everybody
i am able to use spell checker but i have some questions if someone can answer this
if i search free text word waranty then i get back suggestion warranty which is fine.
but if do a search on field for example
description:waranty the output collation element is description:warranty which i dont want i want to get back only the text ie warranty.

We are using collation to return back the results since if a user types three words then we use collation in the response element to display the spelling suggestion.

Any advice

darniz



Re: Spell check suggestion and correct way of implementation and some Questions

by darniz :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello everybody
i am able to use spell checker but i have some questions if someone can answer this
if i search free text word waranty then i get back suggestion warranty which is fine.
but if do a search on field for example
description:waranty the output collation element is description:warranty which i dont want i want to get back only the text ie warranty.

We are using collation to return back the results since if a user types three words then we use collation in the response element to display the spelling suggestion.

Any advice

darniz

Re: Spell check suggestion and correct way of implementation and some Questions

by darniz :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks

i included the  buildOncommit and buildOnOptimize as true and indexed some documents and it automatically builds the dictionary.

Are there any performance issues we should be aware of, with this approach.

Rashid

Re: Spell check suggestion and correct way of implementation and some Questions

by Shalin Shekhar Mangar :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Wed, Nov 4, 2009 at 12:31 AM, darniz <rnizamuddin@...> wrote:

>
> Thanks
>
> i included the  buildOncommit and buildOnOptimize as true and indexed some
> documents and it automatically builds the dictionary.
>
> Are there any performance issues we should be aware of, with this approach.
>
>
Well, it depends. Each commit/optimize will re-create the spell check index
with those options. So, it is best if you test it out with your index,
queries and load.


--
Regards,
Shalin Shekhar Mangar.