HBase
  1. HBase
  2. HBASE-9880

client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96 by HBASE-9867

    Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.96.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      It looks like the backport of HBASE-9867 broke 0.96 build (fine on trunk). This was my patch. Let me fix.

      1. 9880.v1.patch
        0.6 kB
        Nicolas Liochon

        Issue Links

          Activity

          Hide
          Nicolas Liochon added a comment -

          It seems to be a real bug in the AsyncProcess, unrelated to the patch. I don't know why it happens only now.

            private void backgroundFlushCommits(boolean synchronous) throws
                InterruptedIOException, RetriesExhaustedWithDetailsException {
              if (!synchronous && this.writeAsyncBuffer.isEmpty()) return;    //<< this is the fix
          

          (ps: I'm not a jira-jacker , just that I had this failure in a patch I'm going to push soon - hopefully -).

          Show
          Nicolas Liochon added a comment - It seems to be a real bug in the AsyncProcess, unrelated to the patch. I don't know why it happens only now. private void backgroundFlushCommits( boolean synchronous) throws InterruptedIOException, RetriesExhaustedWithDetailsException { if (!synchronous && this .writeAsyncBuffer.isEmpty()) return ; //<< this is the fix (ps: I'm not a jira-jacker , just that I had this failure in a patch I'm going to push soon - hopefully -).
          Hide
          Nicolas Liochon added a comment -

          trunk has this already. Not clear why 0.96 does not. There is no difference between the two branches for AsyncProcess.

          $ git diff 0.96 clean -- hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
          diff --git a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
          index 21447e0..00de79e 100644
          --- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
          +++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
          @@ -207,7 +207,7 @@ public class HTable implements HTableInterface {
               this.pool = getDefaultExecutor(this.configuration);
               this.finishSetup();
             }
          -
          +   
             public static ThreadPoolExecutor getDefaultExecutor(Configuration conf) {
               int maxThreads = conf.getInt("hbase.htable.threads.max", Integer.MAX_VALUE);
               if (maxThreads == 0) {
          @@ -911,7 +911,6 @@ public class HTable implements HTableInterface {
              */
             private void backgroundFlushCommits(boolean synchronous) throws
                 InterruptedIOException, RetriesExhaustedWithDetailsException {
          -    if (this.writeAsyncBuffer.isEmpty()) return;
           
               try {
                 do {
          @@ -1230,7 +1229,7 @@ public class HTable implements HTableInterface {
                 return;
               }
               flushCommits();
          -    if (cleanupPoolOnClose && this.pool != null) {
          +    if (cleanupPoolOnClose) {
                 this.pool.shutdown();
               }
               if (cleanupConnectionOnClose) {
          
          Show
          Nicolas Liochon added a comment - trunk has this already. Not clear why 0.96 does not. There is no difference between the two branches for AsyncProcess. $ git diff 0.96 clean -- hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java diff --git a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java index 21447e0..00de79e 100644 --- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java +++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java @@ -207,7 +207,7 @@ public class HTable implements HTableInterface { this.pool = getDefaultExecutor(this.configuration); this.finishSetup(); } - + public static ThreadPoolExecutor getDefaultExecutor(Configuration conf) { int maxThreads = conf.getInt("hbase.htable.threads.max", Integer.MAX_VALUE); if (maxThreads == 0) { @@ -911,7 +911,6 @@ public class HTable implements HTableInterface { */ private void backgroundFlushCommits(boolean synchronous) throws InterruptedIOException, RetriesExhaustedWithDetailsException { - if (this.writeAsyncBuffer.isEmpty()) return; try { do { @@ -1230,7 +1229,7 @@ public class HTable implements HTableInterface { return; } flushCommits(); - if (cleanupPoolOnClose && this.pool != null) { + if (cleanupPoolOnClose) { this.pool.shutdown(); } if (cleanupConnectionOnClose) {
          Hide
          stack added a comment -

          +1 on the patch.

          Did I fumble the application of HBASE-9867 to 0.96 leaving this line in?

          Show
          stack added a comment - +1 on the patch. Did I fumble the application of HBASE-9867 to 0.96 leaving this line in?
          Hide
          Nicolas Liochon added a comment -

          Yes, actually it's in your patch in HBASE-9867.
          So at the end it's a good news, the test caught the issue.

          Show
          Nicolas Liochon added a comment - Yes, actually it's in your patch in HBASE-9867 . So at the end it's a good news, the test caught the issue.
          Hide
          Hudson added a comment -

          FAILURE: Integrated in hbase-0.96 #179 (See https://builds.apache.org/job/hbase-0.96/179/)
          HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96 by HBASE-9867 (nkeywal: rev 1538676)

          • /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
          Show
          Hudson added a comment - FAILURE: Integrated in hbase-0.96 #179 (See https://builds.apache.org/job/hbase-0.96/179/ ) HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96 by HBASE-9867 (nkeywal: rev 1538676) /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in hbase-0.96-hadoop2 #113 (See https://builds.apache.org/job/hbase-0.96-hadoop2/113/)
          HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96 by HBASE-9867 (nkeywal: rev 1538676)

          • /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
          Show
          Hudson added a comment - FAILURE: Integrated in hbase-0.96-hadoop2 #113 (See https://builds.apache.org/job/hbase-0.96-hadoop2/113/ ) HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96 by HBASE-9867 (nkeywal: rev 1538676) /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
          Hide
          stack added a comment -

          Released in 0.96.1. Issue closed.

          Show
          stack added a comment - Released in 0.96.1. Issue closed.

            People

            • Assignee:
              Nicolas Liochon
              Reporter:
              stack
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development