Details

    • Hadoop Flags:
      Reviewed

      Description

      HDFS-5377 updates the datanode to send one storage report per storage directory as part of heartbeat messages.

      This patch is to track corresponding changes to the NameNode.

      1. h5398.03.patch
        33 kB
        Arpit Agarwal
      2. h5398.02.patch
        31 kB
        Arpit Agarwal

        Issue Links

          Activity

          Hide
          Arpit Agarwal added a comment -

          Thanks Nicholas! I committed this to branch HDFS-2832.

          Show
          Arpit Agarwal added a comment - Thanks Nicholas! I committed this to branch HDFS-2832 .
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
          Hide
          Arpit Agarwal added a comment -

          Thanks Nicholas, addressed all your feedback.

          Show
          Arpit Agarwal added a comment - Thanks Nicholas, addressed all your feedback.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Patch looks good. Some minor comments:

          • remove the following constructor since it is only used by another constructor.
              public DatanodeDescriptor(DatanodeID nodeID, 
                                        int xceiverCount,
                                        int failedVolumes) {
                ...
              }
            
          • DatanodeDescriptor.getStorageReports() and DatanodeStorageInfo.getStorageReport() are only used in tests. They can be moved to BlockManagerTestUtil.
          • There are quite a few "new StorageReport[ 0]". How about adding an EMPTY_ARRAY so that it could reduce unnecessary object creations? I.e.
            // StorageReport.java
            public static StorageReport[] EMPTY_ARRAY = {};
            
          Show
          Tsz Wo Nicholas Sze added a comment - Patch looks good. Some minor comments: remove the following constructor since it is only used by another constructor. public DatanodeDescriptor(DatanodeID nodeID, int xceiverCount, int failedVolumes) { ... } DatanodeDescriptor.getStorageReports() and DatanodeStorageInfo.getStorageReport() are only used in tests. They can be moved to BlockManagerTestUtil. There are quite a few "new StorageReport[ 0]". How about adding an EMPTY_ARRAY so that it could reduce unnecessary object creations? I.e. // StorageReport.java public static StorageReport[] EMPTY_ARRAY = {};
          Hide
          Arpit Agarwal added a comment -

          Namenode changes to process one storage report per directory.

          See HDFS-5377 for corresponding changes to the Datanode.

          Show
          Arpit Agarwal added a comment - Namenode changes to process one storage report per directory. See HDFS-5377 for corresponding changes to the Datanode.

            People

            • Assignee:
              Arpit Agarwal
              Reporter:
              Arpit Agarwal
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development