Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      Currently the key to Block is block id + generation stamp. I would propose to change it to be only block id. This is based on the following properties of the dfs cluster:
      1. On each datanode only one replica of block exists. Therefore there is only one generation of a block.
      2. NameNode has only one entry for a block in its blocks map.

      With this change, search for a block/replica's meta information is easier since most of the time we know a block's id but may not know its generation stamp.

      1. blockKey.patch
        25 kB
        Hairong Kuang
      2. blockIdAsKey.patch
        20 kB
        Konstantin Shvachko
      3. blockIdAsKey.patch
        23 kB
        Konstantin Shvachko

        Issue Links

          Activity

          Hairong Kuang created issue -
          Hairong Kuang made changes -
          Field Original Value New Value
          Link This issue blocks HDFS-509 [ HDFS-509 ]
          Hairong Kuang made changes -
          Attachment blockKey.patch [ 12414968 ]
          Tsz Wo Nicholas Sze made changes -
          Link This issue relates to HADOOP-2656 [ HADOOP-2656 ]
          Hairong Kuang made changes -
          Hadoop Flags [Incompatible change]
          Fix Version/s Append Branch [ 12314142 ]
          Fix Version/s 0.21.0 [ 12314046 ]
          Affects Version/s Append Branch [ 12314142 ]
          Affects Version/s 0.21.0 [ 12314046 ]
          Konstantin Shvachko made changes -
          Attachment blockIdAsKey.patch [ 12422175 ]
          Konstantin Shvachko made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.21.0 [ 12314046 ]
          Affects Version/s Append Branch [ 12314142 ]
          Assignee Hairong Kuang [ hairong ] Konstantin Shvachko [ shv ]
          Fix Version/s 0.21.0 [ 12314046 ]
          Fix Version/s Append Branch [ 12314142 ]
          Konstantin Shvachko made changes -
          Attachment blockIdAsKey.patch [ 12422973 ]
          Konstantin Shvachko made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Konstantin Shvachko made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Konstantin Shvachko made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Incompatible change] [Incompatible change, Reviewed]
          Resolution Fixed [ 1 ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Konstantin Shvachko
              Reporter:
              Hairong Kuang
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development