« Return to Thread: Failed to Crawl the page

Re: Failed to Crawl the page

by Ahmed Ashour :: Rate this Message:

Hi Chris,

Latest snapshot doesn't give errors with me, http://build.canoo.com/htmlunit/artifacts/

        WebClient webClient = new WebClient();
        webClient.setThrowExceptionOnFailingStatusCode(false);
        CollectingAlertHandler alerts = new CollectingAlertHandler();
        try {
            webClient.setAlertHandler(alerts);
            HtmlPage page = webClient.getPage("http://www.buy.com/sr/searchresults.aspx?qu=sony&qxt=home&display=col&&resultCount=5&gsap=1");
            System.out.println(page.asXml());
        }
        catch(Exception e) {
            e.printStackTrace();
        }
        finally {
            for (String x: alerts.getCollectedAlerts()) {
                System.out.println(x);
            }
        }

Yours,

Ahmed

From: chris daniel <achris1985@...>
To: htmlunit-user@...
Sent: Friday, April 27, 2012 4:14 PM
Subject: Re: [Htmlunit-user] Failed to Crawl the page

Please find my error log
Apr 27, 2012 6:42:32 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2012 6:42:34 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2012 6:42:34 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2012 6:42:35 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2012 6:42:35 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2012 6:42:36 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2012 6:42:36 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2012 6:42:37 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'application/x-javascript'.
Apr 27, 2012 6:42:38 PM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'text/javascript'.
Apr 27, 2012 6:42:38 PM com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine handleJavaScriptException
INFO: Caught script exception
======= EXCEPTION START ========
EcmaError: lineNumber=[4817] column=[0] lineSource=[<no source>] name=[TypeError] sourceName=[script in http://www.kmart.com/shc/s/dap_10151_10104_DAP_Shop+Internationally+with+Sears?countryCd=IE from (4794, 32) to (4978, 10)] message=[TypeError: Cannot call method "write" of null (script in http://www.kmart.com/shc/s/dap_10151_10104_DAP_Shop+Internationally+with+Sears?countryCd=IE from (4794, 32) to (4978, 10)#4817)]
com.gargoylesoftware.htmlunit.ScriptException: TypeError: Cannot call method "write" of null (script in http://www.kmart.com/shc/s/dap_10151_10104_DAP_Shop+Internationally+with+Sears?countryCd=IE from (4794, 32) to (4978, 10)#4817)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:595)

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Htmlunit-user mailing list
Htmlunit-user@...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

« Return to Thread: Failed to Crawl the page