MoreLikeThis and filtering/restricting on "target" fields

[gonzoprosperity]

I am trying to use MoreLikeThis (both the component and handler,
trying combinations) and I would like to give it an input document
reference which has a "source" field to analyze and then get back
other documents which have a given field that is used by MLT.

My dataset is composed of documents like:

# Doc 1
id:Article:99
type_s:Article
body_t: the body of the article...

# Doc 2
id:Article:646
types_s:Article
body_t: another article...

# Doc 3
id:Community:44
type_s:Community
description_t: description of this community...

# Doc 4
id:Community:34874
type_s:Community
description_t: another description....

# Doc 5
id:BlogPost:2384
type_s:BlogPost
body_t: contents of some blog post

So I would like to say, "given an article (e.g. id:"Article:99" which
has a field "body_t" that should be analyze), give more related
Communities, and you will want to search on "description_t" for your
analysis".'

When I run a basic query like:

(using raw URL values for clarity, but they are encoded in reality)

http://localhost:9007/solr/mlt?q=id:WikiArticle:948&mlt.fl=body_t

then I get back a ton of other articles. Which is fine if my target
type was Article.

So how I can I say "search on field A for your analysis of the input
document, but for related terms use field B, filtered by type_s"

It seems that I can really only specify one field via mlt.fl

I have tried using MLT as a search component so that it has access to
filter queries (via fq) but I cannot seem to get it to give me any
data other than more of the same, that is, I can get a ton of Articles
back but not other "content types".

Am I just trying to do too much?

Thanks
/Cody

[Alexey-34]

Hi Cody,

> I have tried using MLT as a search component so that it has access to
> filter queries (via fq) but I cannot seem to get it to give me any
> data other than more of the same, that is, I can get a ton of Articles
> back but not other "content types".
Filter query ( fq ) should work, for example add fq=type_s:BlogPost OR
type_s:Community

http://localhost:9007/solr/mlt?q=id:WikiArticle:948&mlt.fl=body_t&mlt.qf=body_t^1.0&fq=type_s:BlogPost
OR type_s:Community

Alex

On Fri, Nov 6, 2009 at 1:44 AM, Cody Caughlan <toolbag@...> wrote:

> I am trying to use MoreLikeThis (both the component and handler,
> trying combinations) and I would like to give it an input document
> reference which has a "source" field to analyze and then get back
> other documents which have a given field that is used by MLT.
>
> My dataset is composed of documents like:
>
> # Doc 1
> id:Article:99
> type_s:Article
> body_t: the body of the article...
>
> # Doc 2
> id:Article:646
> types_s:Article
> body_t: another article...
>
> # Doc 3
> id:Community:44
> type_s:Community
> description_t: description of this community...
>
> # Doc 4
> id:Community:34874
> type_s:Community
> description_t: another description....
>
> # Doc 5
> id:BlogPost:2384
> type_s:BlogPost
> body_t: contents of some blog post
>
> So I would like to say, "given an article (e.g. id:"Article:99" which
> has a field "body_t" that should be analyze), give more related
> Communities, and you will want to search on "description_t" for your
> analysis".'
>
> When I run a basic query like:
>
> (using raw URL values for clarity, but they are encoded in reality)
>
> http://localhost:9007/solr/mlt?q=id:WikiArticle:948&mlt.fl=body_t
>
> then I get back a ton of other articles. Which is fine if my target
> type was Article.
>
> So how I can I say "search on field A for your analysis of the input
> document, but for related terms use field B, filtered by type_s"
>
> It seems that I can really only specify one field via mlt.fl
>
> I have tried using MLT as a search component so that it has access to
> filter queries (via fq) but I cannot seem to get it to give me any
> data other than more of the same, that is, I can get a ton of Articles
> back but not other "content types".
>
> Am I just trying to do too much?
>
> Thanks
> /Cody
>

[Cody Caughlan-3]

Thanks Alex, that did the trick. Fortunately, I had a copyField
directive which has a 'normalized' version of all the various
documents in one field, so I was able to use that field in the field
list and then just filter on type via the fq parameter - works like a
charm.

Thanks!

On Fri, Nov 6, 2009 at 12:46 AM, Alexey Serba <aserba@...> wrote:

> Hi Cody,
>
>> I have tried using MLT as a search component so that it has access to
>> filter queries (via fq) but I cannot seem to get it to give me any
>> data other than more of the same, that is, I can get a ton of Articles
>> back but not other "content types".
> Filter query ( fq ) should work, for example add fq=type_s:BlogPost OR
> type_s:Community
>
> http://localhost:9007/solr/mlt?q=id:WikiArticle:948&mlt.fl=body_t&mlt.qf=body_t^1.0&fq=type_s:BlogPost
> OR type_s:Community
>
> Alex
>
> On Fri, Nov 6, 2009 at 1:44 AM, Cody Caughlan <toolbag@...> wrote:
>> I am trying to use MoreLikeThis (both the component and handler,
>> trying combinations) and I would like to give it an input document
>> reference which has a "source" field to analyze and then get back
>> other documents which have a given field that is used by MLT.
>>
>> My dataset is composed of documents like:
>>
>> # Doc 1
>> id:Article:99
>> type_s:Article
>> body_t: the body of the article...
>>
>> # Doc 2
>> id:Article:646
>> types_s:Article
>> body_t: another article...
>>
>> # Doc 3
>> id:Community:44
>> type_s:Community
>> description_t: description of this community...
>>
>> # Doc 4
>> id:Community:34874
>> type_s:Community
>> description_t: another description....
>>
>> # Doc 5
>> id:BlogPost:2384
>> type_s:BlogPost
>> body_t: contents of some blog post
>>
>> So I would like to say, "given an article (e.g. id:"Article:99" which
>> has a field "body_t" that should be analyze), give more related
>> Communities, and you will want to search on "description_t" for your
>> analysis".'
>>
>> When I run a basic query like:
>>
>> (using raw URL values for clarity, but they are encoded in reality)
>>
>> http://localhost:9007/solr/mlt?q=id:WikiArticle:948&mlt.fl=body_t
>>
>> then I get back a ton of other articles. Which is fine if my target
>> type was Article.
>>
>> So how I can I say "search on field A for your analysis of the input
>> document, but for related terms use field B, filtered by type_s"
>>
>> It seems that I can really only specify one field via mlt.fl
>>
>> I have tried using MLT as a search component so that it has access to
>> filter queries (via fq) but I cannot seem to get it to give me any
>> data other than more of the same, that is, I can get a ton of Articles
>> back but not other "content types".
>>
>> Am I just trying to do too much?
>>
>> Thanks
>> /Cody
>>
>