Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9566

Remove expensive 'BlocksMap#getStorages(Block b, final DatanodeStorage.State state)' method

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 3.0.0-alpha1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      HDFS-5318 added a BlocksMap#getStorages(Block, State) which is based on iterables and predicates. The method is very expensive compared to a simple comparison/continue.

      1. HDFS-9566.branch-2.patch
        4 kB
        Daryn Sharp
      2. HDFS-9566.patch
        4 kB
        Daryn Sharp

        Activity

        Hide
        daryn Daryn Sharp added a comment -

        Just removes it.

        Show
        daryn Daryn Sharp added a comment - Just removes it.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        +1

        Show
        arpitagarwal Arpit Agarwal added a comment - +1
        Hide
        vinayrpet Vinayakumar B added a comment -

        Committed to trunk, branch-2 and branch-2.8

        Thanks Daryn Sharp
        Thanks Arpit Agarwal and Mingliang Liu for reviews.

        Show
        vinayrpet Vinayakumar B added a comment - Committed to trunk, branch-2 and branch-2.8 Thanks Daryn Sharp Thanks Arpit Agarwal and Mingliang Liu for reviews.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9213 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9213/)
        HDFS-9566. Remove expensive 'BlocksMap#getStorages(Block b, final (vinayakumarb: rev e418bd1fb0568ce7ae22f588fea2dd9c95567383)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlocksMap.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9213 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9213/ ) HDFS-9566 . Remove expensive 'BlocksMap#getStorages(Block b, final (vinayakumarb: rev e418bd1fb0568ce7ae22f588fea2dd9c95567383) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlocksMap.java

          People

          • Assignee:
            daryn Daryn Sharp
            Reporter:
            daryn Daryn Sharp
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development