Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1623 High Availability Framework for HDFS NN
  3. HDFS-2691

HA: Tests and fixes for pipeline targets and replica recovery

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: HA branch (HDFS-1623)
    • Fix Version/s: HA branch (HDFS-1623)
    • Component/s: ha
    • Labels:
      None

      Description

      Currently there are some TODOs around pipeline/recovery code in the HA branch. For example, commitBlockSynchronization only gets sent to the active NN which may have failed over by that point. So, we need to write some tests here and figure out what the correct behavior is.

      Another related area is the treatment of targets in the pipeline. When a pipeline is created, the active NN adds the "expected locations" to the BlockInfoUnderConstruction, but the DN identifiers aren't logged with the OP_ADD. So after a failover, the BlockInfoUnderConstruction will have no targets and I imagine replica recovery would probably trigger some issues.

        Attachments

        1. hdfs-2691.txt
          37 kB
          Todd Lipcon
        2. hdfs-2691.txt
          36 kB
          Todd Lipcon
        3. hdfs-2691.txt
          33 kB
          Todd Lipcon
        4. hdfs-2691.txt
          10 kB
          Todd Lipcon

          Issue Links

            Activity

              People

              • Assignee:
                tlipcon Todd Lipcon
                Reporter:
                tlipcon Todd Lipcon
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: