Re: Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out.

> anyone help? so disappointed.
>
> On Fri, Jul 10, 2009 at 4:29 PM, lei wang <nutchmaillist@...> wrote:
>
>> Yes, I am also occuring to  this problem. Can anyone help?
>>
>>
>> On Sun, Jul 5, 2009 at 11:33 PM, xiao yang <yangxiao9901@...> wrote:
>>
>>> I often get this error message while crawling the intranet
>>> Is it the network problem? What can I do for it?
>>>
>>> $bin/nutch crawl urls -dir crawl -depth 3 -topN 4
>>>
>>> crawl started in: crawl
>>> rootUrlDir = urls
>>> threads = 10
>>> depth = 3
>>> topN = 4
>>> Injector: starting
>>> Injector: crawlDb: crawl/crawldb
>>> Injector: urlDir: urls
>>> Injector: Converting injected urls to crawl db entries.
>>> Injector: Merging injected urls into crawl db.
>>> Injector: done
>>> Generator: Selecting best-scoring urls due for fetch.
>>> Generator: starting
>>> Generator: segment: crawl/segments/20090705212324
>>> Generator: filtering: true
>>> Generator: topN: 4
>>> Generator: Partitioning selected urls by host, for politeness.
>>> Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out.
>>> Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out.
>>> Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out.
>>> Exception in thread "main" java.io.IOException: Job failed!
>>>    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)
>>>    at org.apache.nutch.crawl.Generator.generate(Generator.java:524)
>>>    at org.apache.nutch.crawl.Generator.generate(Generator.java:409)
>>>    at org.apache.nutch.crawl.Crawl.main(Crawl.java:116)
>>>
>>
>