Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-2518

Must check return value of job.waitForCompletion()

    XMLWordPrintableJSON

    Details

      Description

      The return value of job.waitForCompletion() of the new MapReduce API (NUTCH-2375) must always be checked. If it's not true, the job has been failed or killed. Accordingly, the program

      • should not proceed with further jobs/steps
      • must clean-up temporary data, unlock CrawlDB, etc.
      • exit with non-zero exit value, so that scripts running the crawl workflow can handle the failure

      Cf. NUTCH-2076, NUTCH-2442, NUTCH-2375 PR #221.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                wastl-nagel Sebastian Nagel
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: