Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4517

unstable dfs when running jobs on 0.18.1

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed

      Description

      2 attempts of a job using 6000 maps, 1900 reduces

      1.st attempt: failed during reduce phase after 22 hours with 31 dead datanodes most of which became unresponsive due to an exception; dfs lost blocks
      2nd attempt: failed during map phase after 5 hours with 5 dead datanodes due to exception; dfs lost blocks responsible for job failure.

      I will post typical datanode exception and attach thread dump.

        Attachments

        1. 4517_20081024.patch
          3 kB
          Tsz-wo Sze
        2. 4517_20081024b_0.18.patch
          3 kB
          Tsz-wo Sze
        3. 4517_20081024c_0.18.patch
          3 kB
          Tsz-wo Sze
        4. 4517_20081024d_0.18.patch
          3 kB
          Tsz-wo Sze
        5. 4517_20081024d.patch
          3 kB
          Tsz-wo Sze
        6. 4517_20081027_0.18.patch
          4 kB
          Tsz-wo Sze
        7. 4517_20081027.patch
          4 kB
          Tsz-wo Sze
        8. datanode.out
          301 kB
          Christian Kunz

          Activity
          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              szetszwo Tsz-wo Sze
              Reporter:
              ckunz Christian Kunz

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment