Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-1370

Expose exact number of urls injected @runtime

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: nutchgora, 1.5
    • Fix Version/s: 1.6, 2.2
    • Component/s: injector
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      Example: When using trunk, currently we see

      2012-05-22 09:04:00,239 INFO  crawl.Injector - Injector: starting at 2012-05-22 09:04:00
      2012-05-22 09:04:00,239 INFO  crawl.Injector - Injector: crawlDb: crawl/crawldb
      2012-05-22 09:04:00,239 INFO  crawl.Injector - Injector: urlDir: urls
      2012-05-22 09:04:00,253 INFO  crawl.Injector - Injector: Converting injected urls to crawl db entries.
      2012-05-22 09:04:00,955 INFO  plugin.PluginRepository - Plugins: looking in:
      

      I would like to see

      2012-05-22 09:04:00,239 INFO  crawl.Injector - Injector: starting at 2012-05-22 09:04:00
      2012-05-22 09:04:00,239 INFO  crawl.Injector - Injector: crawlDb: crawl/crawldb
      2012-05-22 09:04:00,239 INFO  crawl.Injector - Injector: urlDir: urls
      2012-05-22 09:04:00,253 INFO  crawl.Injector - Injector: Injected N urls to crawl/crawldb
      2012-05-22 09:04:00,253 INFO  crawl.Injector - Injector: Converting injected urls to crawl db entries.
      2012-05-22 09:04:00,955 INFO  plugin.PluginRepository - Plugins: looking in:
      

      This would make debugging easier and would help those who end up getting

      2012-05-22 09:04:04,850 WARN  crawl.Generator - Generator: 0 records selected for fetching, exiting ...
      

        Attachments

        1. NUTCH-1370-1.x.patch
          2 kB
          Sebastian Nagel
        2. NUTCH-1370-2.x.patch
          8 kB
          Lewis John McGibbney
        3. NUTCH-1370-2.x-v2.patch
          9 kB
          Lewis John McGibbney
        4. NUTCH-1370-2.x-v3.patch
          8 kB
          Sebastian Nagel

          Issue Links

            Activity

              People

              • Assignee:
                lewismc Lewis John McGibbney
                Reporter:
                lewismc Lewis John McGibbney
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: