Nutch
  1. Nutch
  2. NUTCH-1409

Remove deprecated properties db.{default,max}.fetch.interval, generate.max.per.host.by.ip

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3, 1.10
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      1) Remove deprecated properties from nutch-default.xml (generate.max.per.host and db.default.fetch.interval).

      2) The already removed properties generate.max.per.host.by.ip and db.max.fetch.interval are still used in source code.

      1. NUTCH-1409-trunk-v2.patch
        2 kB
        Sebastian Nagel
      2. NUTCH-1409-2x.patch
        1 kB
        Sebastian Nagel
      3. NUTCH-1409.patch
        5 kB
        Matthias Agethle

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Nutch-trunk #2749 (See https://builds.apache.org/job/Nutch-trunk/2749/)
          NUTCH-1409 remove deprecated properties db.

          {default,max}

          .fetch.interval, generate.max.per.host.by.ip (snagel: http://svn.apache.org/viewvc/nutch/trunk/?view=rev&rev=1619934)

          • /nutch/branches/2.x/CHANGES.txt
          • /nutch/branches/2.x/src/java/org/apache/nutch/crawl/AbstractFetchSchedule.java
          • /nutch/trunk/CHANGES.txt
          • /nutch/trunk/src/java/org/apache/nutch/crawl/CrawlDbReducer.java
          • /nutch/trunk/src/java/org/apache/nutch/crawl/Generator.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Nutch-trunk #2749 (See https://builds.apache.org/job/Nutch-trunk/2749/ ) NUTCH-1409 remove deprecated properties db. {default,max} .fetch.interval, generate.max.per.host.by.ip (snagel: http://svn.apache.org/viewvc/nutch/trunk/?view=rev&rev=1619934 ) /nutch/branches/2.x/CHANGES.txt /nutch/branches/2.x/src/java/org/apache/nutch/crawl/AbstractFetchSchedule.java /nutch/trunk/CHANGES.txt /nutch/trunk/src/java/org/apache/nutch/crawl/CrawlDbReducer.java /nutch/trunk/src/java/org/apache/nutch/crawl/Generator.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Nutch-nutchgora #1127 (See https://builds.apache.org/job/Nutch-nutchgora/1127/)
          NUTCH-1409 remove deprecated properties db.

          {default,max}

          .fetch.interval, generate.max.per.host.by.ip (snagel: http://svn.apache.org/viewvc/nutch/branches/2.x/?view=rev&rev=1619934)

          • /nutch/branches/2.x/CHANGES.txt
          • /nutch/branches/2.x/src/java/org/apache/nutch/crawl/AbstractFetchSchedule.java
          • /nutch/trunk/CHANGES.txt
          • /nutch/trunk/src/java/org/apache/nutch/crawl/CrawlDbReducer.java
          • /nutch/trunk/src/java/org/apache/nutch/crawl/Generator.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Nutch-nutchgora #1127 (See https://builds.apache.org/job/Nutch-nutchgora/1127/ ) NUTCH-1409 remove deprecated properties db. {default,max} .fetch.interval, generate.max.per.host.by.ip (snagel: http://svn.apache.org/viewvc/nutch/branches/2.x/?view=rev&rev=1619934 ) /nutch/branches/2.x/CHANGES.txt /nutch/branches/2.x/src/java/org/apache/nutch/crawl/AbstractFetchSchedule.java /nutch/trunk/CHANGES.txt /nutch/trunk/src/java/org/apache/nutch/crawl/CrawlDbReducer.java /nutch/trunk/src/java/org/apache/nutch/crawl/Generator.java
          Hide
          Sebastian Nagel added a comment -

          Committed to trunk and 2.x, r1619934. Thanks!

          Show
          Sebastian Nagel added a comment - Committed to trunk and 2.x, r1619934. Thanks!
          Hide
          Sebastian Nagel added a comment -

          Most of the changes in Matthias Agethle's patch have been made by NUTCH-1514. Patches for the remaining changes: will commit now since properties are deprecated for years now. Renamed title: also affects Java files and nutch-default.xml was already cleaned-up with NUTCH-1514.

          Show
          Sebastian Nagel added a comment - Most of the changes in Matthias Agethle 's patch have been made by NUTCH-1514 . Patches for the remaining changes: will commit now since properties are deprecated for years now. Renamed title: also affects Java files and nutch-default.xml was already cleaned-up with NUTCH-1514 .
          Hide
          Lewis John McGibbney added a comment -

          I am not sure about the logging for this one. If we are to remove the properties, thereby denying users the choice to override them, then why do we need to log that they should use some other settings instead? Saying that, this one is nearly ready though.
          @Matthias, off the top of your head, I wonder if you were able to check out 2.x and comment on the code? Thank you very much for the patch.

          Show
          Lewis John McGibbney added a comment - I am not sure about the logging for this one. If we are to remove the properties, thereby denying users the choice to override them, then why do we need to log that they should use some other settings instead? Saying that, this one is nearly ready though. @Matthias, off the top of your head, I wonder if you were able to check out 2.x and comment on the code? Thank you very much for the patch.
          Hide
          Matthias Agethle added a comment -

          Patch for trunk (rev 1352896)

          Show
          Matthias Agethle added a comment - Patch for trunk (rev 1352896)

            People

            • Assignee:
              Unassigned
              Reporter:
              Matthias Agethle
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development