[NUTCH-1370] Expose exact number of urls injected @runtime

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: nutchgora, 1.5
• Fix Version/s: 1.6, 2.2
• Component/s: injector
• Patch Info: Patch Available
• Assignee: Lewis John McGibbney
• Reporter: Lewis John McGibbney
• Created: 22/May/12 09:23
• Updated: 22/May/13 03:53
• Resolved: 22/Nov/12 14:57

Description

Example: When using trunk, currently we see

2012-05-22 09:04:00,239 INFO  crawl.Injector - Injector: starting at 2012-05-22 09:04:00
2012-05-22 09:04:00,239 INFO  crawl.Injector - Injector: crawlDb: crawl/crawldb
2012-05-22 09:04:00,239 INFO  crawl.Injector - Injector: urlDir: urls
2012-05-22 09:04:00,253 INFO  crawl.Injector - Injector: Converting injected urls to crawl db entries.
2012-05-22 09:04:00,955 INFO  plugin.PluginRepository - Plugins: looking in:

I would like to see

2012-05-22 09:04:00,239 INFO  crawl.Injector - Injector: starting at 2012-05-22 09:04:00
2012-05-22 09:04:00,239 INFO  crawl.Injector - Injector: crawlDb: crawl/crawldb
2012-05-22 09:04:00,239 INFO  crawl.Injector - Injector: urlDir: urls
2012-05-22 09:04:00,253 INFO  crawl.Injector - Injector: Injected N urls to crawl/crawldb
2012-05-22 09:04:00,253 INFO  crawl.Injector - Injector: Converting injected urls to crawl db entries.
2012-05-22 09:04:00,955 INFO  plugin.PluginRepository - Plugins: looking in:

This would make debugging easier and would help those who end up getting

2012-05-22 09:04:04,850 WARN  crawl.Generator - Generator: 0 records selected for fetching, exiting ...

Issue Links

• incorporates: NUTCH-1471 make explicit which datastore urls are injected to