Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-2910

FetchItemQueues overloaded constructor also interprets fetcher timeout as -1 e.g. no-timeout.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Not A Problem
    • 1.18
    • 1.19
    • fetcher
    • None

    Description

      The FetchItemQueues overloaded constructor attempts to obtain the NON-EXISTENT fetcher.timelimit configuration property.

      this.timelimit = conf.getLong("fetcher.timelimit", -1);
      

      As you can see a default value of -1 is provided. The first parameter is however wrong. It should instead reference the following configuration property.

      <property>
        <name>fetcher.timelimit.mins</name>
        <value>-1</value>
        <description>This is the number of minutes allocated to the fetching.
        Once this value is reached, any remaining entry from the input URL list is skipped 
        and all active queues are emptied. The default value of -1 deactivates the time limit.
        </description>
      </property>
      

      Note, fetcher.timelimit.mins

      I think that this essentially means the Fetcher has no time limit which is ofcourse not desired.

      Attachments

        Activity

          People

            lewismc Lewis John McGibbney
            lewismc Lewis John McGibbney
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: