[NUTCH-807] JSParseFilter produces malformed URL

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.0.0
• Component/s: parser

Environment

Redhat 2.6.18-128.1.6.el5PAE i686 i686 i386 GNU/Linux

• Assignee: Unassigned
• Reporter: Minyao Zhu
• Created: 02/Apr/10 07:31
• Updated: 09/Dec/12 07:39
• Resolved: 09/Dec/12 07:39

Description

This is found when crawling site: http://zhidao.baidu.com/ ( a Chinese language site )

It appears this page contains javascripts which confused JSParseFilter, which produced URL like this:

http://zhidao.baidu.com/){if(A===46){baidu.hide(

Not sure the impact/scope of this issue in general. The observation for this specific site is, much less pages got crawled.

Thanks.