[HDFS-4444] Add space between total transaction time and number of transactions in FSEditLog#printStatistics - ASF JIRA

Details

Type: Bug
Status: Closed
Priority: Trivial
Resolution: Fixed
Affects Version/s: 2.0.0-alpha, 3.0.0
Fix Version/s: 1.2.0, 2.0.3-alpha, 0.23.7
Component/s: None
Labels:
None

Hadoop Flags:

Reviewed
Target Version/s:

2.0.0-alpha, 3.0.0

Description

Currently, when we log statistics, we see something like

13/01/25 23:16:59 INFO namenode.FSNamesystem: Number of transactions: 0 Total time for transactions(ms): 0Number of transactions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0

Notice how the value for total transactions time and "Number of transactions batched in Syncs" needs a space to separate them.

FSEditLog#printStatistics:

  private void printStatistics(boolean force) {
    long now = now();
    if (lastPrintTime + 60000 > now && !force) {
      return;
    }
    lastPrintTime = now;
    StringBuilder buf = new StringBuilder();
    buf.append("Number of transactions: ");
    buf.append(numTransactions);
    buf.append(" Total time for transactions(ms): ");
    buf.append(totalTimeTransactions);
    buf.append("Number of transactions batched in Syncs: ");
    buf.append(numTransactionsBatchedInSync);
    buf.append(" Number of syncs: ");
    buf.append(editLogStream.getNumSync());
    buf.append(" SyncTimes(ms): ");
    buf.append(journalSet.getSyncTimes());
    LOG.info(buf);
  }

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending

Attachments

HDFS-4444.patch.branch-1

28/Jan/13 01:25

0.7 kB

Stephen Chu
HDFS-4444.patch.001

26/Jan/13 07:44

0.9 kB

Stephen Chu

Issue Links

is related to

HDFS-4782 backport edit log corruption toleration to branch-1-win

Resolved

Activity

Descending order - Click to sort in ascending order

Hide

Permalink

Hudson added a comment - 02/Feb/13 12:40

Integrated in Hadoop-Hdfs-0.23-Build #513 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/513/)
~~HDFS-4444~~. Add space between total transaction time and number of transactions in FSEditLog#printStatistics. (Stephen Chu via tgraves) (Revision 1441652)

Result = SUCCESS
tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441652
Files :

/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java

Show

Hudson added a comment - 02/Feb/13 12:40 Integrated in Hadoop-Hdfs-0.23-Build #513 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/513/ ) HDFS-4444 . Add space between total transaction time and number of transactions in FSEditLog#printStatistics. (Stephen Chu via tgraves) (Revision 1441652) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441652 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java

Hide

Permalink

Hudson added a comment - 29/Jan/13 14:04

Integrated in Hadoop-Mapreduce-trunk #1328 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1328/)
~~HDFS-4444~~. Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559)

Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559
Files :

/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java

Show

Hudson added a comment - 29/Jan/13 14:04 Integrated in Hadoop-Mapreduce-trunk #1328 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1328/ ) HDFS-4444 . Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java

Hide

Permalink

Hudson added a comment - 29/Jan/13 13:03

Integrated in Hadoop-Hdfs-trunk #1300 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1300/)
~~HDFS-4444~~. Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559)

Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559
Files :

/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java

Show

Hudson added a comment - 29/Jan/13 13:03 Integrated in Hadoop-Hdfs-trunk #1300 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1300/ ) HDFS-4444 . Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java

Hide

Permalink

Hudson added a comment - 29/Jan/13 10:46

Integrated in Hadoop-Yarn-trunk #111 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/111/)
~~HDFS-4444~~. Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559)

Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559
Files :

/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java

Show

Hudson added a comment - 29/Jan/13 10:46 Integrated in Hadoop-Yarn-trunk #111 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/111/ ) HDFS-4444 . Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java

Hide

Permalink

Hudson added a comment - 28/Jan/13 19:04

Integrated in Hadoop-trunk-Commit #3287 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3287/)
~~HDFS-4444~~. Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559)

Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559
Files :

/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java

Show

Hudson added a comment - 28/Jan/13 19:04 Integrated in Hadoop-trunk-Commit #3287 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3287/ ) HDFS-4444 . Add space between total transaction time and number of transactions in FSEditLog#printStatistics. Contributed by Stephen Chu. (Revision 1439559) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1439559 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java

Hide

Permalink

Suresh Srinivas added a comment - 28/Jan/13 19:00

Committed the patch to trunk, branch-1 and branch-2.

Thank you Stephen!

Show

Suresh Srinivas added a comment - 28/Jan/13 19:00 Committed the patch to trunk, branch-1 and branch-2. Thank you Stephen!

Hide

Permalink

Suresh Srinivas added a comment - 28/Jan/13 18:53 - edited

+1 for the change. I will commit it shortly.

Show

Suresh Srinivas added a comment - 28/Jan/13 18:53 - edited +1 for the change. I will commit it shortly.

Hide

Permalink

Chris Nauroth added a comment - 28/Jan/13 03:50

+1 for the branch-1 patch as well

Thanks, Stephen!

Show

Chris Nauroth added a comment - 28/Jan/13 03:50 +1 for the branch-1 patch as well Thanks, Stephen!

Hide

Permalink

Stephen Chu added a comment - 28/Jan/13 01:57

I forgot to "Cancel Patch" before uploading the branch-1 patch. I manually verified applying the patch to branch-1 as well as the fix.

Show

Stephen Chu added a comment - 28/Jan/13 01:57 I forgot to "Cancel Patch" before uploading the branch-1 patch. I manually verified applying the patch to branch-1 as well as the fix.

Hide

Permalink

Hadoop QA added a comment - 28/Jan/13 01:32

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12566702/HDFS-4444.patch.branch-1
against trunk revision .

-1 patch. The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3887//console

This message is automatically generated.

Show

Hadoop QA added a comment - 28/Jan/13 01:32 -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566702/HDFS-4444.patch.branch-1 against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3887//console This message is automatically generated.

Hide

Permalink

Stephen Chu added a comment - 28/Jan/13 01:25

Thanks, Chris!

I uploaded a patch for branch-1. I manually verified that it corrects the log message.

Show

Stephen Chu added a comment - 28/Jan/13 01:25 Thanks, Chris! I uploaded a patch for branch-1. I manually verified that it corrects the log message.

Hide

Permalink

Chris Nauroth added a comment - 27/Jan/13 23:51

I built and ran the patch locally on trunk and confirmed that it corrected the log message. I see that this same patch applies cleanly to both trunk and branch-2.

The bug also exists on branch-1 if you want to port it over there.

Thanks!

Show

Chris Nauroth added a comment - 27/Jan/13 23:51 +1 I built and ran the patch locally on trunk and confirmed that it corrected the log message. I see that this same patch applies cleanly to both trunk and branch-2. The bug also exists on branch-1 if you want to port it over there. Thanks!

Hide

Permalink

Stephen Chu added a comment - 26/Jan/13 09:50

No new unit tests are necessary to cover this case (just an extra whitespace character in the log).

I verified this manually.

Show

Stephen Chu added a comment - 26/Jan/13 09:50 No new unit tests are necessary to cover this case (just an extra whitespace character in the log). I verified this manually.

Hide

Permalink

Hadoop QA added a comment - 26/Jan/13 09:24

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12566620/HDFS-4444.patch.001
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

+1 contrib tests. The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3886//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3886//console

This message is automatically generated.

Show

Hadoop QA added a comment - 26/Jan/13 09:24 -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566620/HDFS-4444.patch.001 against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3886//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3886//console This message is automatically generated.

Hide

Permalink

Stephen Chu added a comment - 26/Jan/13 07:44

Uploaded a patch that adds a single whitespace character.

Show

Stephen Chu added a comment - 26/Jan/13 07:44 Uploaded a patch that adds a single whitespace character.

People

Assignee:

Stephen Chu

Reporter:

Stephen Chu

Votes:

0 Vote for this issue

Watchers:

5 Start watching this issue

Dates

Created:

26/Jan/13 07:37

Updated:

03/May/13 04:10

Resolved:

28/Jan/13 19:00

Development

Agile

View on Board