Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-1625

IndexerMapReduce skips FETCH_NOTMODIFIED

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Won't Fix
    • 1.7
    • None
    • indexer
    • None
    • Patch Available

    Description

      IndexerMapReduce has the option to skip DB_NOTMODIFIED but legacy code also skips FETCH_NOTMODIFIED and the latter is not optional. We can keep the check but that should also include FETCH_NOTMODIFIED. Relying on FETCH_NOTMODIFIED isn't very useful anyway because since 1.5 orso we can safely rely on DB_NOTMODIFIED as it is properly set in the CrawlDBReducer.

      Attachments

        1. NUTCH-1625.patch
          0.7 kB
          Markus Jelsma
        2. NUTCH-1625.patch
          0.8 kB
          Markus Jelsma

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. NUTCH-1416 IndexerMapReduce can index older version of a document instead of latest one

          • Critical - Crashes, loss of data, severe memory leak.
          • Reopened

          Activity

            People

              markus17 Markus Jelsma
              markus17 Markus Jelsma
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: