Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-378

DFSClient should track failures by datanode across all streams

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Original description: Rather than tracking the total number of times DFSInputStream failed to talk to a datanode for a particular block, such failures and the the list of datanodes involved should be scoped to individual blocks. In particular, the "deadnode" list should be a map of blocks to a list of failed nodes, the latter reset and the nodes retried per the existing semantics.

      [see comment below for new thinking, left this comment to give context to discussion]

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-127 DFSClient block read failures cause open DFSInputStream to become unusable

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-1911 infinite loop in dfs -cat command.

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Task - A task that needs to be done. HDFS-656 Clarify error handling and retry semantics for DFS read path

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Activity

            People

              Unassigned Unassigned
              cdouglas Christopher Douglas
              Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated: