Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-512

Set block id as the key to Block

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.21.0
    • 0.21.0
    • None
    • None
    • Incompatible change, Reviewed

    Description

      Currently the key to Block is block id + generation stamp. I would propose to change it to be only block id. This is based on the following properties of the dfs cluster:
      1. On each datanode only one replica of block exists. Therefore there is only one generation of a block.
      2. NameNode has only one entry for a block in its blocks map.

      With this change, search for a block/replica's meta information is easier since most of the time we know a block's id but may not know its generation stamp.

      Attachments

        1. blockKey.patch
          25 kB
          Hairong Kuang
        2. blockIdAsKey.patch
          20 kB
          Konstantin Shvachko
        3. blockIdAsKey.patch
          23 kB
          Konstantin Shvachko

        Issue Links

          blocks

          Sub-task - The sub-task of the issue HDFS-509 Redesign DataNode volumeMap to include all types of Replicas

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-2656 Support for upgrading existing cluster to facilitate appends to HDFS files

          • Major - Major loss of function.
          • Closed

          Activity

            People

              shv Konstantin Shvachko
              hairong Hairong Kuang
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: