Plugins: when to perform web service requests, on fetch or on index?
Hi All,
I'm writing several nutch plugins, which will perform a requests to some webservices for pages being indexed and store retrieved data in index. The question is: on what stage of crawling it is better to perform these webservice requests: on fetching or on indexing (in HtmlParseFilter or in IndexingFilter), in terms of performance, of course?
Nutch version is 1.0, indexer is SolrIndexer.
Thanks.