[NUTCH-2985] Disable plugin urlfilter-validator by default

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.19
• Fix Version/s: 1.20
• Component/s: configuration, urlfilter
• Patch Info: Patch Available
• Assignee: Sebastian Nagel
• Reporter: Sebastian Nagel
• Created: 24/Feb/23 15:22
• Updated: 13/Mar/24 14:51
• Resolved: 06/Mar/23 10:51

Description

The plugin urlfilter-validator is activated by default (in nutch-default.xml) but has two major issues which may confuse users of Nutch:

• single-part domain names (localhost, etc.) are not allowed (NUTCH-2973)
• IPv6 host names are rejected as invalid (NUTCH-2705)

What about disabling it by default to overcome these issues?

Issue Links

• supercedes: NUTCH-2973 Single domain names (eg https://localnet) can't be crawled - filtering fails
• links to: Github PR #759
• links to: GitHub Pull Request #759