Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10752

Several log refactoring/improvement suggestion in HDFS

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.2
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      As per conversation with Vrushali C, we merged HDFS-10749, HDFS-10750, HDFS-10751, HDFS-10753, under this issue.


      HDFS-10749
      Method invocation in logs can be replaced by variable
      Similar to the fix for HDFS-409. In file:

      hadoop-rel-release-2.7.2/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java

      In code block:

      lastQueuedSeqno = currentPacket.getSeqno();
      if (DFSClient.LOG.isDebugEnabled()) {
          DFSClient.LOG.debug("Queued packet " + currentPacket.getSeqno());
      }
      

      currentPacket.getSeqno() is better to be replaced by variable lastQueuedSeqno.


      HDFS-10750
      Similar to the fix for AVRO-115. In file:

      hadoop-rel-release-2.7.2/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java

      in line 695, the logging code:

      LOG.info(getRole() + " RPC up at: " + rpcServer.getRpcAddress());
      

      In the same class, there is a method in line 907:

        /**
         * @return NameNode RPC address
         */
        public InetSocketAddress getNameNodeAddress() {
          return rpcServer.getRpcAddress();
        }
      

      We can tell that rpcServer.getRpcAddress() could be replaced by method getNameNodeAddress() for the case of readability and simplicity


      HDFS-10751
      Similar to the fix for AVRO-115. In file:

      hadoop-rel-release-2.7.2/hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtxCache.java

      in line 72, the logging code:

      LOG.trace("openFileMap size:" + openFileMap.size());
      

      In the same class, there is a method in line 189:

        int size() {
          return openFileMap.size();
        }
      

      We can tell that openFileMap.size() could be replaced by method size() for the case of readability and simplicity


      Print variable in byte
      Similar to the fix for HBASE-623, in file:

      hadoop-rel-release-2.7.2/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java

      In the following method, the log printed variable data (in byte[]). A possible fix is add Bytes.toString(data).

      /**
         * Write the batch of edits to the local copy of the edit logs.
         */
        private void logEditsLocally(long firstTxId, int numTxns, byte[] data) {
          long expectedTxId = editLog.getLastWrittenTxId() + 1;
          Preconditions.checkState(firstTxId == expectedTxId,
              "received txid batch starting at %s but expected txn %s",
              firstTxId, expectedTxId);
          editLog.setNextTxId(firstTxId + numTxns - 1);
          editLog.logEdit(data.length, data);
          editLog.logSync();
        }
      


      HDFS-10753
      MethodInvocation replaced by variable due to toString method
      Similar to the fix in HADOOP-6419, in file:

      hadoop-rel-release-2.7.2/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CorruptReplicasMap.java

      in line 76, the blk.getBlockName() method invocation is invoked on variable blk. "blk" is the class instance of Block.

      void addToCorruptReplicasMap(Block blk, DatanodeDescriptor dn,
            String reason, Reason reasonCode) {
      ...
      NameNode.blockStateChangeLog.info(
                "BLOCK NameSystem.addToCorruptReplicasMap: {} added as corrupt on "
                    + "{} by {} {}", blk.getBlockName(), dn, Server.getRemoteIp(),
                reasonText);
      

      In file: hadoop-rel-release-2.7.2/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java

        @Override
        public String toString() {
          return getBlockName() + "_" + getGenerationStamp();
        }
      

      The toString() method contain not only getBlockName() but also getGenerationStamp which may be helpful for debugging purpose. Therefore blk.getBlockName() can be replaced by blk

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hanishakoneru Hanisha Koneru
                Reporter:
                chenfsd Nemo Chen
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: