Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-683

NUTCH-676 broke CrawlDbMerger

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 1.0.0
    • 1.0.0
    • None
    • None

    Description

      Switch to hadoop's MapWritable broke CrawlDbMerger. Part of the reason is that we reuse the same MapWritable instance during reduce
      which apparently is a big no-no for hadoop's MapWritable. Also, hadoop's MapWritable#putAll doesn't work (I think.... see HADOOP-5142),
      so we should also work around that.

      Attachments

        1. crawldbmerger_v2.patch
          2 kB
          Dogacan Guney

        Activity

          People

            dogacan Dogacan Guney
            dogacan Dogacan Guney
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: