Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0-alpha2
    • None
    • None

    Description

      When a node is transitioned to/stay in/transitioned out of maintenance state, we need to make sure blocks w.r.t. that nodes are properly handled.

      • When nodes are put into maintenance, it will first go to ENTERING_MAINTENANCE, and make sure blocks are minimally replicated before the nodes are transitioned to IN_MAINTENANCE.
      • Do not replica blocks when nodes are in maintenance states. Maintenance replica will remain in BlockMaps and thus is still considered valid from block replication point of view. In other words, putting a node to “maintenance” mode won’t trigger BlockManager to replicate its blocks.
      • Do not invalidate replicas on node under maintenance. After any file's replication factor is reduced, NN needs to invalidate some replicas. It should exclude nodes under maintenance in the handling.
      • Do not put IN_MAINTENANCE replicas in LocatedBlock for read operation.
      • Do not allocate any new block on nodes under maintenance.
      • Have Balancer exclude nodes under maintenance.
      • Exclude nodes under maintenance for DN cache.

      Attachments

        1. HDFS-9390.patch
          121 kB
          Ming Ma
        2. HDFS-9390-2.patch
          123 kB
          Ming Ma
        3. HDFS-9390-3.patch
          131 kB
          Ming Ma
        4. HDFS-9390-4.patch
          132 kB
          Ming Ma
        5. HDFS-9390-5.patch
          133 kB
          Ming Ma
        6. HDFS-9390-branch-2.002.patch
          124 kB
          Ming Ma
        7. HDFS-9390-branch-2.patch
          124 kB
          Ming Ma

        Activity

          People

            mingma Ming Ma
            mingma Ming Ma
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: