Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-4444

Add space between total transaction time and number of transactions in FSEditLog#printStatistics

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha, 3.0.0
    • Fix Version/s: 1.2.0, 2.0.3-alpha, 0.23.7
    • Component/s: None
    • Labels:
      None

      Description

      Currently, when we log statistics, we see something like

      13/01/25 23:16:59 INFO namenode.FSNamesystem: Number of transactions: 0 Total time for transactions(ms): 0Number of transactions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0
      

      Notice how the value for total transactions time and "Number of transactions batched in Syncs" needs a space to separate them.

      FSEditLog#printStatistics:

        private void printStatistics(boolean force) {
          long now = now();
          if (lastPrintTime + 60000 > now && !force) {
            return;
          }
          lastPrintTime = now;
          StringBuilder buf = new StringBuilder();
          buf.append("Number of transactions: ");
          buf.append(numTransactions);
          buf.append(" Total time for transactions(ms): ");
          buf.append(totalTimeTransactions);
          buf.append("Number of transactions batched in Syncs: ");
          buf.append(numTransactionsBatchedInSync);
          buf.append(" Number of syncs: ");
          buf.append(editLogStream.getNumSync());
          buf.append(" SyncTimes(ms): ");
          buf.append(journalSet.getSyncTimes());
          LOG.info(buf);
        }
      
      1. HDFS-4444.patch.branch-1
        0.7 kB
        Stephen Chu
      2. HDFS-4444.patch.001
        0.9 kB
        Stephen Chu

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #513 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/513/)
          HDFS-4444. Add space between total transaction time and number of transactions in FSEditLog#printStatistics. (Stephen Chu via tgraves) (Revision 1441652)

          Result = SUCCESS
          tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441652
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #513 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/513/ ) HDFS-4444 . Add space between total transaction time and number of transactions in FSEditLog#printStatistics. (Stephen Chu via tgraves) (Revision 1441652) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441652 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1328 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1328/)
          HDFS-4444. Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559)

          Result = SUCCESS
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1328 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1328/ ) HDFS-4444 . Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1300 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1300/)
          HDFS-4444. Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559)

          Result = SUCCESS
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1300 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1300/ ) HDFS-4444 . Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Yarn-trunk #111 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/111/)
          HDFS-4444. Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559)

          Result = SUCCESS
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          Show
          Hudson added a comment - Integrated in Hadoop-Yarn-trunk #111 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/111/ ) HDFS-4444 . Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #3287 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3287/)
          HDFS-4444. Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559)

          Result = SUCCESS
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          Show
          Hudson added a comment - Integrated in Hadoop-trunk-Commit #3287 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3287/ ) HDFS-4444 . Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          Hide
          Suresh Srinivas added a comment -

          Committed the patch to trunk, branch-1 and branch-2.

          Thank you Stephen!

          Show
          Suresh Srinivas added a comment - Committed the patch to trunk, branch-1 and branch-2. Thank you Stephen!
          Hide
          Suresh Srinivas added a comment - - edited

          +1 for the change. I will commit it shortly.

          Show
          Suresh Srinivas added a comment - - edited +1 for the change. I will commit it shortly.
          Hide
          Chris Nauroth added a comment -

          +1 for the branch-1 patch as well

          Thanks, Stephen!

          Show
          Chris Nauroth added a comment - +1 for the branch-1 patch as well Thanks, Stephen!
          Hide
          Stephen Chu added a comment -

          I forgot to "Cancel Patch" before uploading the branch-1 patch. I manually verified applying the patch to branch-1 as well as the fix.

          Show
          Stephen Chu added a comment - I forgot to "Cancel Patch" before uploading the branch-1 patch. I manually verified applying the patch to branch-1 as well as the fix.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12566702/HDFS-4444.patch.branch-1
          against trunk revision .

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3887//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566702/HDFS-4444.patch.branch-1 against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3887//console This message is automatically generated.
          Hide
          Stephen Chu added a comment -

          Thanks, Chris!

          I uploaded a patch for branch-1. I manually verified that it corrects the log message.

          Show
          Stephen Chu added a comment - Thanks, Chris! I uploaded a patch for branch-1. I manually verified that it corrects the log message.
          Hide
          Chris Nauroth added a comment -

          +1

          I built and ran the patch locally on trunk and confirmed that it corrected the log message. I see that this same patch applies cleanly to both trunk and branch-2.

          The bug also exists on branch-1 if you want to port it over there.

          Thanks!

          Show
          Chris Nauroth added a comment - +1 I built and ran the patch locally on trunk and confirmed that it corrected the log message. I see that this same patch applies cleanly to both trunk and branch-2. The bug also exists on branch-1 if you want to port it over there. Thanks!
          Hide
          Stephen Chu added a comment -

          No new unit tests are necessary to cover this case (just an extra whitespace character in the log).

          I verified this manually.

          Show
          Stephen Chu added a comment - No new unit tests are necessary to cover this case (just an extra whitespace character in the log). I verified this manually.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12566620/HDFS-4444.patch.001
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3886//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3886//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566620/HDFS-4444.patch.001 against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3886//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3886//console This message is automatically generated.
          Hide
          Stephen Chu added a comment -

          Uploaded a patch that adds a single whitespace character.

          Show
          Stephen Chu added a comment - Uploaded a patch that adds a single whitespace character.

            People

            • Assignee:
              Stephen Chu
              Reporter:
              Stephen Chu
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development