« Return to Thread: Plugins: when to perform web service requests, on fetch or on index?

Re: Plugins: when to perform web service requests, on fetch or on index?

by caezar :: Rate this Message:

Reply to Author | View in Thread

Hi,

Thank you for the response. Parsed data is not used in calls. Only page URL. So performance will be better if perform this requests on parsing stage?
Stefan Dlugolinsky wrote:
Hello,

I don't know how v 1.0 differs from v 0.9, but in v 0.9, I would do
those service requests in the stage of indexation (extension point
IndexingFilter), where you have several data prepared from previous
stage (by parsers, etc.), so you can use this data in the requests.
But it depends on what you exactly want, whether you want to use
parsed data in the requests. If not, you can call webservice requests
earlier from parsing stage (extension point Parse).

Here is something about core extension points:
http://wiki.apache.org/nutch/AboutPlugins

Steve

 « Return to Thread: Plugins: when to perform web service requests, on fetch or on index?