[NUTCH-1328] a problem with regex-normalize.xml

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.4
• Component/s: parser
• Labels: parse
• Assignee: Unassigned
• Reporter: behnam nikbakht
• Created: 03/Apr/12 05:12
• Updated: 10/Jul/12 21:20
• Resolved: 10/Jul/12 21:20

Description

there is a regex-pattern in regex-normalize.xml:
<pattern>([;_]?((?i)l|j|bv_)?((?i)sid|phpsessid|sessionid)=.*?)(?|&|#|$)</pattern>
that remove session ids from urls, but there is some sites, like:
http://www.mehrnews.com/fa
that have urls, like:
http://www.mehrnews.com/fa/newsdetail.aspx?NewsID=1567539
and with this pattern, this url converted to an invalid url:
http://www.mehrnews.com/fa/newsdetail.aspx?New

Issue Links

• duplicates: NUTCH-706 Url regex normalizer: default pattern for session id removal not to match "newsId"