Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-1409

Remove deprecated properties db.{default,max}.fetch.interval, generate.max.per.host.by.ip

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3, 1.10
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      1) Remove deprecated properties from nutch-default.xml (generate.max.per.host and db.default.fetch.interval).

      2) The already removed properties generate.max.per.host.by.ip and db.max.fetch.interval are still used in source code.

      1. NUTCH-1409.patch
        5 kB
        Matthias Agethle
      2. NUTCH-1409-2x.patch
        1 kB
        Sebastian Nagel
      3. NUTCH-1409-trunk-v2.patch
        2 kB
        Sebastian Nagel

        Issue Links

          Activity

          Hide
          magethle Matthias Agethle added a comment -

          Patch for trunk (rev 1352896)

          Show
          magethle Matthias Agethle added a comment - Patch for trunk (rev 1352896)
          Hide
          lewismc Lewis John McGibbney added a comment -

          I am not sure about the logging for this one. If we are to remove the properties, thereby denying users the choice to override them, then why do we need to log that they should use some other settings instead? Saying that, this one is nearly ready though.
          @Matthias, off the top of your head, I wonder if you were able to check out 2.x and comment on the code? Thank you very much for the patch.

          Show
          lewismc Lewis John McGibbney added a comment - I am not sure about the logging for this one. If we are to remove the properties, thereby denying users the choice to override them, then why do we need to log that they should use some other settings instead? Saying that, this one is nearly ready though. @Matthias, off the top of your head, I wonder if you were able to check out 2.x and comment on the code? Thank you very much for the patch.
          Hide
          wastl-nagel Sebastian Nagel added a comment -

          Most of the changes in Matthias Agethle's patch have been made by NUTCH-1514. Patches for the remaining changes: will commit now since properties are deprecated for years now. Renamed title: also affects Java files and nutch-default.xml was already cleaned-up with NUTCH-1514.

          Show
          wastl-nagel Sebastian Nagel added a comment - Most of the changes in Matthias Agethle 's patch have been made by NUTCH-1514 . Patches for the remaining changes: will commit now since properties are deprecated for years now. Renamed title: also affects Java files and nutch-default.xml was already cleaned-up with NUTCH-1514 .
          Hide
          wastl-nagel Sebastian Nagel added a comment -

          Committed to trunk and 2.x, r1619934. Thanks!

          Show
          wastl-nagel Sebastian Nagel added a comment - Committed to trunk and 2.x, r1619934. Thanks!
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Nutch-nutchgora #1127 (See https://builds.apache.org/job/Nutch-nutchgora/1127/)
          NUTCH-1409 remove deprecated properties db.

          {default,max}

          .fetch.interval, generate.max.per.host.by.ip (snagel: http://svn.apache.org/viewvc/nutch/branches/2.x/?view=rev&rev=1619934)

          • /nutch/branches/2.x/CHANGES.txt
          • /nutch/branches/2.x/src/java/org/apache/nutch/crawl/AbstractFetchSchedule.java
          • /nutch/trunk/CHANGES.txt
          • /nutch/trunk/src/java/org/apache/nutch/crawl/CrawlDbReducer.java
          • /nutch/trunk/src/java/org/apache/nutch/crawl/Generator.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Nutch-nutchgora #1127 (See https://builds.apache.org/job/Nutch-nutchgora/1127/ ) NUTCH-1409 remove deprecated properties db. {default,max} .fetch.interval, generate.max.per.host.by.ip (snagel: http://svn.apache.org/viewvc/nutch/branches/2.x/?view=rev&rev=1619934 ) /nutch/branches/2.x/CHANGES.txt /nutch/branches/2.x/src/java/org/apache/nutch/crawl/AbstractFetchSchedule.java /nutch/trunk/CHANGES.txt /nutch/trunk/src/java/org/apache/nutch/crawl/CrawlDbReducer.java /nutch/trunk/src/java/org/apache/nutch/crawl/Generator.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Nutch-trunk #2749 (See https://builds.apache.org/job/Nutch-trunk/2749/)
          NUTCH-1409 remove deprecated properties db.

          {default,max}

          .fetch.interval, generate.max.per.host.by.ip (snagel: http://svn.apache.org/viewvc/nutch/trunk/?view=rev&rev=1619934)

          • /nutch/branches/2.x/CHANGES.txt
          • /nutch/branches/2.x/src/java/org/apache/nutch/crawl/AbstractFetchSchedule.java
          • /nutch/trunk/CHANGES.txt
          • /nutch/trunk/src/java/org/apache/nutch/crawl/CrawlDbReducer.java
          • /nutch/trunk/src/java/org/apache/nutch/crawl/Generator.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Nutch-trunk #2749 (See https://builds.apache.org/job/Nutch-trunk/2749/ ) NUTCH-1409 remove deprecated properties db. {default,max} .fetch.interval, generate.max.per.host.by.ip (snagel: http://svn.apache.org/viewvc/nutch/trunk/?view=rev&rev=1619934 ) /nutch/branches/2.x/CHANGES.txt /nutch/branches/2.x/src/java/org/apache/nutch/crawl/AbstractFetchSchedule.java /nutch/trunk/CHANGES.txt /nutch/trunk/src/java/org/apache/nutch/crawl/CrawlDbReducer.java /nutch/trunk/src/java/org/apache/nutch/crawl/Generator.java

            People

            • Assignee:
              Unassigned
              Reporter:
              magethle Matthias Agethle
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development