Hi, I try to REcrawl (with a shell-script. I have already a webDB...) a
website (with some links to other webpage, .html, .doc, .pdf, ...) but this
error occured:
...
Generator: Selecting best-scoring urls due for fetch.
Generator: starting
Generator: segment: crawl/segments/20090703140431
Generator: filtering: true
Generator: topN: 10
Generator: jobtracker is 'local', generating exactly one partition.
Generator: 0 records selected for fetching, exiting ...
Fetcher: Your 'http.agent.name' value should be listed first in
'http.robots.agents' property.
Fetcher: starting
Fetcher: segment: crawl/segments/20090703111416
Exception in thread "main" java.io.IOException: Segment already fetched!
at org.apache.nutch.fetcher.
FetcherOutputFormat.checkOutputSpecs(FetcherOutputFormat.java:50)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:793)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1142)
at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:969)
at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:1003)
...
someone have any idea to resolve this problem?
thx to all
Maurizio [
croci.maurizio@...]
