Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10285 Storage Policy Satisfier in HDFS
  3. HDFS-11032

[SPS]: Handling of block movement failure at the coordinator datanode

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: HDFS-10285, 3.2.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The idea of this jira is to discuss and implement an efficient failure(block movement failure) handling logic at the datanode cooridnator. Code reference.

      Following are the possible errors during block movement:

      1. Block pinned - no retries marked as success/no-retry to NN. It is not possible to relocate this block to another datanode.
      2. Network errors(IOException) - no retries maked as failure/retry to NN.
      3. No disk space(IOException) - no retries maked as failure/retry to NN.
      4. Gen_Stamp mismatches - no retries marked as failure/retry to NN. Could be a case that the file might have re-opened.

        Attachments

        1. HDFS-11032-HDFS-10285-00.patch
          16 kB
          Rakesh Radhakrishnan
        2. HDFS-11032-HDFS-10285-01.patch
          16 kB
          Rakesh Radhakrishnan

          Issue Links

            Activity

              People

              • Assignee:
                rakeshr Rakesh Radhakrishnan
                Reporter:
                rakeshr Rakesh Radhakrishnan
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: