Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5704

Change OP_UPDATE_BLOCKS with a new OP_ADD_BLOCK

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Add a new editlog record (OP_ADD_BLOCK) that only records allocation of the new block instead of the entire block list, on every block allocation.

      Description

      Currently every time a block a allocated, the entire list of blocks are written in the editlog in OP_UPDATE_BLOCKS operation. This has n^2 growth issue. The total size of editlog records for a file with large number of blocks could be huge.

      The goal of this jira is discuss adding a different editlog record that only records allocation of block and not the entire block list, on every block allocation.

        Attachments

        1. HDFS-5704.000.patch
          50 kB
          Jing Zhao
        2. HDFS-5704.001.patch
          50 kB
          Jing Zhao
        3. editsStored
          4 kB
          Jing Zhao
        4. HDFS-5704.002.patch
          54 kB
          Jing Zhao
        5. HDFS-5704.003.patch
          51 kB
          Jing Zhao
        6. HDFS-5704-branch2.patch
          45 kB
          Jing Zhao
        7. editsStored
          4 kB
          Jing Zhao

        Issue Links

        requires

        Bug - A problem which impairs or prevents the functions of the product. HDFS-5777 Update LayoutVersion for the new editlog op OP_ADD_BLOCK

        • Major - Major loss of function.
        • Closed

          Activity
          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              jingzhao Jing Zhao Assign to me
              Reporter:
              sureshms Suresh Srinivas

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment