[NUTCH-69] fetcher.threads.per.host ignored

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Component/s: fetcher
• Assignee: Unassigned
• Reporter: Matthias Jaekle
• Created: 08/Jul/05 23:18
• Updated: 18/Apr/07 15:46
• Resolved: 08/Jul/05 23:38

Description

Fetcher ignores 'maximum threads per host'.
If you fetch less domains with multiple threads, some webservers feel attacked or could not serve you any more.
So you loose lots of existing pages in your segments.