[jira] Created: (NUTCH-740) Configuration option to override default language for fetched pages.

Configuration option to override default language for fetched pages.
--------------------------------------------------------------------

                 Key: NUTCH-740
                 URL: https://issues.apache.org/jira/browse/NUTCH-740
             Project: Nutch
          Issue Type: Improvement
          Components: fetcher
    Affects Versions: 1.0.0, 0.9.0
            Reporter: Marcin Okraszewski
             Fix For: 1.0.0


By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/NUTCH-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Marcin Okraszewski updated NUTCH-740:
-------------------------------------

    Attachment: AcceptLanguage.patch

The patch which allows overriding of "Accept-Language" header. The patch is done on 1.0 code.

> Configuration option to override default language for fetched pages.
> --------------------------------------------------------------------
>
>                 Key: NUTCH-740
>                 URL: https://issues.apache.org/jira/browse/NUTCH-740
>             Project: Nutch
>          Issue Type: Improvement
>          Components: fetcher
>    Affects Versions: 0.9.0, 1.0.0
>            Reporter: Marcin Okraszewski
>             Fix For: 1.0.0
>
>         Attachments: AcceptLanguage.patch
>
>
> By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/NUTCH-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Otis Gospodnetic updated NUTCH-740:
-----------------------------------

             Priority: Minor  (was: Major)
    Affects Version/s:     (was: 0.9.0)
        Fix Version/s:     (was: 1.0.0)
                       1.1
             Assignee: Otis Gospodnetic

> Configuration option to override default language for fetched pages.
> --------------------------------------------------------------------
>
>                 Key: NUTCH-740
>                 URL: https://issues.apache.org/jira/browse/NUTCH-740
>             Project: Nutch
>          Issue Type: Improvement
>          Components: fetcher
>    Affects Versions: 1.0.0
>            Reporter: Marcin Okraszewski
>            Assignee: Otis Gospodnetic
>            Priority: Minor
>             Fix For: 1.1
>
>         Attachments: AcceptLanguage.patch
>
>
> By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/NUTCH-740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717061#action_12717061 ]

Doğacan Güney commented on NUTCH-740:
-------------------------------------

+1 for the idea but your patch does not apply to trunk.

> Configuration option to override default language for fetched pages.
> --------------------------------------------------------------------
>
>                 Key: NUTCH-740
>                 URL: https://issues.apache.org/jira/browse/NUTCH-740
>             Project: Nutch
>          Issue Type: Improvement
>          Components: fetcher
>    Affects Versions: 1.0.0
>            Reporter: Marcin Okraszewski
>            Assignee: Otis Gospodnetic
>            Priority: Minor
>             Fix For: 1.1
>
>         Attachments: AcceptLanguage.patch
>
>
> By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/NUTCH-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Marcin Okraszewski updated NUTCH-740:
-------------------------------------

    Attachment: AcceptLanguage_trunk_2009-06-09.patch

It does apply, but with Fuzz factor set to 2. Here is the ported patch.

> Configuration option to override default language for fetched pages.
> --------------------------------------------------------------------
>
>                 Key: NUTCH-740
>                 URL: https://issues.apache.org/jira/browse/NUTCH-740
>             Project: Nutch
>          Issue Type: Improvement
>          Components: fetcher
>    Affects Versions: 1.0.0
>            Reporter: Marcin Okraszewski
>            Assignee: Otis Gospodnetic
>            Priority: Minor
>             Fix For: 1.1
>
>         Attachments: AcceptLanguage.patch, AcceptLanguage_trunk_2009-06-09.patch
>
>
> By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.