Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.98.0
    • Fix Version/s: 0.96.2, 0.98.1, 0.99.0
    • Component/s: regionserver
    • Labels:
      None

      Description

      The FlushHandler thread in MemStoreFlusher class uses default thread name (Thread -XX). This is un-intentional and also confusing in case when there are multiple handlers.

      Current Stack trace looks like this:

      "Thread-18" prio=10 tid=0x00007f4e8cb21800 nid=0x356e waiting on condition [0x00007f4e6d49a000]
         java.lang.Thread.State: TIMED_WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000004e5684b00> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
      	at java.util.concurrent.DelayQueue.poll(DelayQueue.java:201)
      	at java.util.concurrent.DelayQueue.poll(DelayQueue.java:39)
      	at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:228)
      	at java.lang.Thread.run(Thread.java:662)
      
      1. 10563-v1.patch
        1 kB
        Himanshu Vashishtha

        Activity

        Himanshu Vashishtha created issue -
        Hide
        Lars Hofhansl added a comment -

        not an issue in 0.94?

        Show
        Lars Hofhansl added a comment - not an issue in 0.94?
        Himanshu Vashishtha made changes -
        Field Original Value New Value
        Attachment 10563-v1.patch [ 12629483 ]
        Hide
        Himanshu Vashishtha added a comment -

        A simple fix.

        Yeah, should be an issue in 0.94 also. I haven't checked it yet. Shall I check and upload a 94 patch too ? Thanks Lars.

        Show
        Himanshu Vashishtha added a comment - A simple fix. Yeah, should be an issue in 0.94 also. I haven't checked it yet. Shall I check and upload a 94 patch too ? Thanks Lars.
        Hide
        Himanshu Vashishtha added a comment -

        With this patch, the stack trace for FlushHandler looks like:

        "MemStoreFlusher.0" prio=10 tid=0x00007f65c862b000 nid=0x20f4 waiting on condition [0x00007f65c7af9000]
           java.lang.Thread.State: TIMED_WAITING (parking)
                at sun.misc.Unsafe.park(Native Method)
                - parking to wait for  <0x00000000f6939a38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
                at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
                at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
                at java.util.concurrent.DelayQueue.poll(DelayQueue.java:201)
                at java.util.concurrent.DelayQueue.poll(DelayQueue.java:39)
                at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:252)
                at java.lang.Thread.run(Thread.java:662)
        

        The naming is consistent with the other threads (syncers in FSHLog).

        mvn test -Dtest=Test*MemStore* passes.

        Show
        Himanshu Vashishtha added a comment - With this patch, the stack trace for FlushHandler looks like: "MemStoreFlusher.0" prio=10 tid=0x00007f65c862b000 nid=0x20f4 waiting on condition [0x00007f65c7af9000] java.lang. Thread .State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f6939a38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025) at java.util.concurrent.DelayQueue.poll(DelayQueue.java:201) at java.util.concurrent.DelayQueue.poll(DelayQueue.java:39) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:252) at java.lang. Thread .run( Thread .java:662) The naming is consistent with the other threads (syncers in FSHLog). mvn test -Dtest=Test*MemStore* passes.
        Himanshu Vashishtha made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Anoop Sam John added a comment -

        +1

        Show
        Anoop Sam John added a comment - +1
        Hide
        Himanshu Vashishtha added a comment -

        Yes, 94 also has this issue. I will upload a patch for 94 once hadoop-qa has run/passed (else it will fail trying 94-patch on trunk).

        Show
        Himanshu Vashishtha added a comment - Yes, 94 also has this issue. I will upload a patch for 94 once hadoop-qa has run/passed (else it will fail trying 94-patch on trunk).
        Hide
        Lars Hofhansl added a comment -

        +1
        Thanks for doing 0.94 as well.

        Show
        Lars Hofhansl added a comment - +1 Thanks for doing 0.94 as well.
        Hide
        Himanshu Vashishtha added a comment -

        I looked more closely and 94 doesn't has this issue
        HRS sets a proper name to it (it got removed in HBASE-6466). So, this is post-94 only.
        Thanks for reviewing Lars and Anoop.

        Show
        Himanshu Vashishtha added a comment - I looked more closely and 94 doesn't has this issue HRS sets a proper name to it (it got removed in HBASE-6466 ). So, this is post-94 only. Thanks for reviewing Lars and Anoop.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12629483/10563-v1.patch
        against trunk revision .
        ATTACHMENT ID: 12629483

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

        +1 hadoop1.1. The patch compiles against the hadoop 1.1 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12629483/10563-v1.patch against trunk revision . ATTACHMENT ID: 12629483 +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop1.1 . The patch compiles against the hadoop 1.1 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8731//console This message is automatically generated.
        Hide
        Matteo Bertozzi added a comment -

        committed to 0.96, 0.98 and trunk

        Show
        Matteo Bertozzi added a comment - committed to 0.96, 0.98 and trunk
        Matteo Bertozzi made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Assignee Himanshu Vashishtha [ v.himanshu ]
        Fix Version/s 0.96.2 [ 12325658 ]
        Fix Version/s 0.98.1 [ 12325664 ]
        Fix Version/s 0.99.0 [ 12325675 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK #4926 (See https://builds.apache.org/job/HBase-TRUNK/4926/)
        HBASE-10563 Set name for FlushHandler thread (Himanshu Vashishtha) (mbertozzi: rev 1569370)

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #4926 (See https://builds.apache.org/job/HBase-TRUNK/4926/ ) HBASE-10563 Set name for FlushHandler thread (Himanshu Vashishtha) (mbertozzi: rev 1569370) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.98 #164 (See https://builds.apache.org/job/HBase-0.98/164/)
        HBASE-10563 Set name for FlushHandler thread (Himanshu Vashishtha) (mbertozzi: rev 1569372)

        • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.98 #164 (See https://builds.apache.org/job/HBase-0.98/164/ ) HBASE-10563 Set name for FlushHandler thread (Himanshu Vashishtha) (mbertozzi: rev 1569372) /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in hbase-0.96 #298 (See https://builds.apache.org/job/hbase-0.96/298/)
        HBASE-10563 Set name for FlushHandler thread (Himanshu Vashishtha) (mbertozzi: rev 1569371)

        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        Hudson added a comment - SUCCESS: Integrated in hbase-0.96 #298 (See https://builds.apache.org/job/hbase-0.96/298/ ) HBASE-10563 Set name for FlushHandler thread (Himanshu Vashishtha) (mbertozzi: rev 1569371) /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #153 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/153/)
        HBASE-10563 Set name for FlushHandler thread (Himanshu Vashishtha) (mbertozzi: rev 1569372)

        • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #153 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/153/ ) HBASE-10563 Set name for FlushHandler thread (Himanshu Vashishtha) (mbertozzi: rev 1569372) /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in hbase-0.96-hadoop2 #207 (See https://builds.apache.org/job/hbase-0.96-hadoop2/207/)
        HBASE-10563 Set name for FlushHandler thread (Himanshu Vashishtha) (mbertozzi: rev 1569371)

        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        Hudson added a comment - FAILURE: Integrated in hbase-0.96-hadoop2 #207 (See https://builds.apache.org/job/hbase-0.96-hadoop2/207/ ) HBASE-10563 Set name for FlushHandler thread (Himanshu Vashishtha) (mbertozzi: rev 1569371) /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #93 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/93/)
        HBASE-10563 Set name for FlushHandler thread (Himanshu Vashishtha) (mbertozzi: rev 1569370)

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #93 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/93/ ) HBASE-10563 Set name for FlushHandler thread (Himanshu Vashishtha) (mbertozzi: rev 1569370) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
        Hide
        Enis Soztutar added a comment -

        Closing this issue after 0.99.0 release.

        Show
        Enis Soztutar added a comment - Closing this issue after 0.99.0 release.
        Enis Soztutar made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Himanshu Vashishtha
            Reporter:
            Himanshu Vashishtha
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development