Nutch
  1. Nutch
  2. NUTCH-1355

nutchgora Configure minimum throughput for fetcher

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: nutchgora
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      Like trunk, nutchgora should also have a feature to configure the fetcher with a minimum throughput. (See NUTCH-1067 for the work done by Markus).

      It's implemented in almost the same way, except that the number of times throughput falls below threshold is measured sequentially. (The counter is reset when throughput is healthy again; this should work even better against temporary dips).

      Defaults to disabled. Will commit later today if there is no objection.

        Activity

        Hide
        Ferdy Galema added a comment -

        committed

        Show
        Ferdy Galema added a comment - committed
        Hide
        Hudson added a comment -

        Integrated in Nutch-nutchgora #248 (See https://builds.apache.org/job/Nutch-nutchgora/248/)
        NUTCH-1355 nutchgora Configure minimum throughput for fetcher (Revision 1335063)

        Result = SUCCESS
        ferdy :
        Files :

        • /nutch/branches/nutchgora/CHANGES.txt
        • /nutch/branches/nutchgora/conf/nutch-default.xml
        • /nutch/branches/nutchgora/src/java/org/apache/nutch/fetcher/FetcherReducer.java
        Show
        Hudson added a comment - Integrated in Nutch-nutchgora #248 (See https://builds.apache.org/job/Nutch-nutchgora/248/ ) NUTCH-1355 nutchgora Configure minimum throughput for fetcher (Revision 1335063) Result = SUCCESS ferdy : Files : /nutch/branches/nutchgora/CHANGES.txt /nutch/branches/nutchgora/conf/nutch-default.xml /nutch/branches/nutchgora/src/java/org/apache/nutch/fetcher/FetcherReducer.java

          People

          • Assignee:
            Unassigned
            Reporter:
            Ferdy Galema
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development