Authenticity of URLs from DMOZ

[Gaurang Patel]

Hey,

Can anyone tell what could be the reason for following which happened while fetching data using bin/nutch fetch:

My AVG Antivirus is detecting virus threats while Nutch fetches pages from available urls of crawldb. I injected DMOZ Open Directory urls to crawldb. Antivirus already detected 4 threats within only half an hour after start of fetching.

Is there any other way(any source other than DMOZ) to get list of whole web urls ? Or is there an automatic way to avoid such harrmful urls from being fetched? Let me know asap.


Regards,
Gaurang