Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10285 Storage Policy Satisfier in HDFS
  3. HDFS-11032

[SPS]: Handling of block movement failure at the coordinator datanode

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: HDFS-10285, 3.2.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The idea of this jira is to discuss and implement an efficient failure(block movement failure) handling logic at the datanode cooridnator. Code reference.

      Following are the possible errors during block movement:

      1. Block pinned - no retries marked as success/no-retry to NN. It is not possible to relocate this block to another datanode.
      2. Network errors(IOException) - no retries maked as failure/retry to NN.
      3. No disk space(IOException) - no retries maked as failure/retry to NN.
      4. Gen_Stamp mismatches - no retries marked as failure/retry to NN. Could be a case that the file might have re-opened.

        Attachments

        1. HDFS-11032-HDFS-10285-00.patch
          16 kB
          Rakesh R
        2. HDFS-11032-HDFS-10285-01.patch
          16 kB
          Rakesh R

          Issue Links

            Activity

              People

              • Assignee:
                rakeshr Rakesh R
                Reporter:
                rakeshr Rakesh R
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: