Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5704

Change OP_UPDATE_BLOCKS with a new OP_ADD_BLOCK

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Add a new editlog record (OP_ADD_BLOCK) that only records allocation of the new block instead of the entire block list, on every block allocation.

      Description

      Currently every time a block a allocated, the entire list of blocks are written in the editlog in OP_UPDATE_BLOCKS operation. This has n^2 growth issue. The total size of editlog records for a file with large number of blocks could be huge.

      The goal of this jira is discuss adding a different editlog record that only records allocation of block and not the entire block list, on every block allocation.

        Attachments

        1. editsStored
          4 kB
          Jing Zhao
        2. HDFS-5704-branch2.patch
          45 kB
          Jing Zhao
        3. HDFS-5704.003.patch
          51 kB
          Jing Zhao
        4. HDFS-5704.002.patch
          54 kB
          Jing Zhao
        5. editsStored
          4 kB
          Jing Zhao
        6. HDFS-5704.001.patch
          50 kB
          Jing Zhao
        7. HDFS-5704.000.patch
          50 kB
          Jing Zhao

          Issue Links

            Activity

              People

              • Assignee:
                jingzhao Jing Zhao
                Reporter:
                sureshms Suresh Srinivas
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: