[NUTCH-1408] RobotRulesParser main doesn't take URL's

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.5
• Fix Version/s: 1.6
• Patch Info: Patch Available
• Assignee: Markus Jelsma
• Reporter: Markus Jelsma
• Created: 22/Jun/12 11:40
• Updated: 22/May/13 03:53
• Resolved: 25/Jun/12 14:42

Description

lib-http's org.apache.nutch.protocol.http.api.RobotRulesParser main() takes a robot file and an URL file according to its usage output. It, however, expects URI paths not URL's and will therefore never work if an input contains URL's.