Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1476

listCorruptFileBlocks should be functional while the name node is still in safe mode

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This would allow us to detect whether missing blocks can be fixed using Raid and if that is the case exit safe mode earlier.

      One way to make listCorruptFileBlocks available before the name node has exited from safe mode would be to perform a scan of the blocks map on each call to listCorruptFileBlocks to determine if there are any blocks with no replicas. This scan could be parallelized by dividing the space of block IDs into multiple intervals than can be scanned independently.

      1. HDFS-1476.2.patch
        16 kB
        Patrick Kling
      2. HDFS-1476.3.patch
        16 kB
        Patrick Kling
      3. HDFS-1476.4.patch
        16 kB
        Patrick Kling
      4. HDFS-1476.5.patch
        17 kB
        Patrick Kling
      5. HDFS-1476.patch
        16 kB
        Patrick Kling

        Issue Links

          Activity

          Patrick Kling created issue -
          Patrick Kling made changes -
          Field Original Value New Value
          Attachment HDFS-1476.patch [ 12459751 ]
          Patrick Kling made changes -
          Assignee Patrick Kling [ pkling ]
          Patrick Kling made changes -
          Attachment HDFS-1476.2.patch [ 12459756 ]
          Patrick Kling made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Patrick Kling made changes -
          Attachment HDFS-1476.3.patch [ 12459935 ]
          Patrick Kling made changes -
          Attachment HDFS-1476.4.patch [ 12460054 ]
          Patrick Kling made changes -
          Attachment HDFS-1476.5.patch [ 12465963 ]
          Hairong Kuang made changes -
          Hadoop Flags [Reviewed]
          Fix Version/s 0.23.0 [ 12315571 ]
          Affects Version/s 0.23.0 [ 12315571 ]
          Component/s name-node [ 12312926 ]
          Hairong Kuang made changes -
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Todd Lipcon made changes -
          Link This issue relates to HDFS-2826 [ HDFS-2826 ]

            People

            • Assignee:
              Patrick Kling
              Reporter:
              Patrick Kling
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development