[NUTCH-407] Make Nutch crawling parent directories for file protocol configurable

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 0.8
• Assignee: Andrzej Bialecki
• Reporter: Thorsten Scherler
• Created: 24/Nov/06 13:23
• Updated: 11/Sep/10 03:49
• Resolved: 27/Nov/06 09:38

Description

http://www.mail-archive.com/[e-mail]/msg06698.html

I am looking into fixing some very weird behavior of the file protocol.
I am using 0.8.

Researching this topic I found
http://www.mail-archive.com/[e-mail]/msg06536.html
and
http://www.folge2.de/tp/search/1/crawling-the-local-filesystem-with-nutch

I am on Ubuntu but I have the same problem that nutch is going down the
tree (including parents) and not up (including children from the root
url).

Further I would vote to make the fetch-parents optional and defined per
a property whether I would like this not very intuitive "feature".

Issue Links

• is part of: NUTCH-905 Configurable file protocol parent directory crawling