Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4539

Streaming Edits to a Backup Node.

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Introduced backup node which maintains the up-to-date state of the namespace by receiving edits from the namenode, and checkpoint node, which creates checkpoints of the name space. These facilities replace the secondary namenode.

      Description

      Currently Secondary name-node acts as mere checkpointer.
      Secondary name-node should be transformed into a standby name-node (SNN).
      The long term goal is to make it a warm standby.
      The purpose of this issue is to provide real time streaming of edits to SNN so that it contained the up-to-date namespace state.

        Attachments

        1. BackupNode.patch
          183 kB
          Konstantin Shvachko
        2. BackupNode.patch
          183 kB
          Konstantin Shvachko
        3. BackupNode.patch
          183 kB
          Konstantin Shvachko
        4. BackupNode.patch
          177 kB
          Konstantin Shvachko
        5. BackupNode.patch
          177 kB
          Konstantin Shvachko
        6. BackupNode.patch
          163 kB
          Konstantin Shvachko
        7. image001.gif
          8 kB
          Konstantin Shvachko
        8. StreamEditsToBN.pdf
          67 kB
          Konstantin Shvachko
        9. StreamEditsToSNN.htm
          31 kB
          Konstantin Shvachko

          Issue Links

          incorporates

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-5511 Add Apache License to EditLogBackupOutputStream

          • Major - Major loss of function.
          • Closed
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-2337 Trash never closes FileSystem

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-1572 Checkpointer should trigger checkpoint with specified period.

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-1583 Improve backup-node sync performance by wrapping RPC parameters

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HADOOP-5119 FSEditLog should not use FSNamesystem.getFSNamesystem()

          • Major - Major loss of function.
          • Closed

          Sub-task - The sub-task of the issue HADOOP-5559 BackupStorage should not use FSNamesystem.getFSNamesystem()

          • Major - Major loss of function.
          • Closed

          Sub-task - The sub-task of the issue HBASE-2098 [EC2] Build a HA cluster

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HDFS-192 TestBackupNode sometimes fails

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-311 Modifications to enable multiple types of logging

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-5502 Backup and checkpoint nodes should be documented

          • Major - Major loss of function.
          • Closed

            Activity

              People

              • Assignee:
                shv Konstantin Shvachko
                Reporter:
                shv Konstantin Shvachko
              • Votes:
                0 Vote for this issue
                Watchers:
                33 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: