HBase
  1. HBase
  2. HBASE-4802

Disable show table metrics in bulk loader

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      During bulk load, the Configuration object may be set to null. This caused an NPE in per-CF metrics because it consults the Configuration to determine whether to show the Table name. Need to add simple change to allow the conf to be null & not specify table name in that instance.

      1. HBASE-4802.patch
        2 kB
        Nicolas Spiegelberg

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security-on-Hadoop-23 #11 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/11/)
          HBASE-7644 Port HBASE-4802 'Disable show table metrics in bulk loader' to 0.94 (Revision 1437091)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security-on-Hadoop-23 #11 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/11/ ) HBASE-7644 Port HBASE-4802 'Disable show table metrics in bulk loader' to 0.94 (Revision 1437091) Result = FAILURE tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security #96 (See https://builds.apache.org/job/HBase-0.94-security/96/)
          HBASE-7644 Port HBASE-4802 'Disable show table metrics in bulk loader' to 0.94 (Revision 1437091)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security #96 (See https://builds.apache.org/job/HBase-0.94-security/96/ ) HBASE-7644 Port HBASE-4802 'Disable show table metrics in bulk loader' to 0.94 (Revision 1437091) Result = FAILURE tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94 #751 (See https://builds.apache.org/job/HBase-0.94/751/)
          HBASE-7644 Port HBASE-4802 'Disable show table metrics in bulk loader' to 0.94 (Revision 1437091)

          Result = SUCCESS
          tedyu :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
          Show
          Hudson added a comment - Integrated in HBase-0.94 #751 (See https://builds.apache.org/job/HBase-0.94/751/ ) HBASE-7644 Port HBASE-4802 'Disable show table metrics in bulk loader' to 0.94 (Revision 1437091) Result = SUCCESS tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
          Hide
          Liang Xie added a comment -

          Ted Yu, we hit just the same issue on 0.94.3 today as well, it really does work after applying the original patch

          Show
          Liang Xie added a comment - Ted Yu , we hit just the same issue on 0.94.3 today as well, it really does work after applying the original patch
          Hide
          Ted Yu added a comment -

          Using 0.94.2, we observed:

          13/01/21 20:31:22 ERROR mapreduce.LoadIncrementalHFiles: Unexpected execution exception during splitting
          java.util.concurrent.ExecutionException: java.lang.IllegalStateException: The value of the hbase.metrics.showTableName conf option has not been specified in SchemaMetrics
          	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
          	at java.util.concurrent.FutureTask.get(FutureTask.java:83)
          	at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplitPhase(LoadIncrementalHFiles.java:333)
          	at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:232)
          	at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:699)
          	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
          	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
          	at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:704)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
          	at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:51)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
          

          We (Nick Dimiduk, Enis and I) think the fix in this JIRA should be backported to 0.94

          Show
          Ted Yu added a comment - Using 0.94.2, we observed: 13/01/21 20:31:22 ERROR mapreduce.LoadIncrementalHFiles: Unexpected execution exception during splitting java.util.concurrent.ExecutionException: java.lang.IllegalStateException: The value of the hbase.metrics.showTableName conf option has not been specified in SchemaMetrics at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) at java.util.concurrent.FutureTask.get(FutureTask.java:83) at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplitPhase(LoadIncrementalHFiles.java:333) at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:232) at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:699) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:704) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:51) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) We (Nick Dimiduk, Enis and I) think the fix in this JIRA should be backported to 0.94
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #196 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/196/)
          HBASE-4802 Disable show table metrics in bulk loader (Revision 1391206)

          Result = FAILURE
          stack :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #196 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/196/ ) HBASE-4802 Disable show table metrics in bulk loader (Revision 1391206) Result = FAILURE stack : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #3386 (See https://builds.apache.org/job/HBase-TRUNK/3386/)
          HBASE-4802 Disable show table metrics in bulk loader (Revision 1391206)

          Result = FAILURE
          stack :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #3386 (See https://builds.apache.org/job/HBase-TRUNK/3386/ ) HBASE-4802 Disable show table metrics in bulk loader (Revision 1391206) Result = FAILURE stack : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
          Hide
          stack added a comment -

          Committed to trunk. Thanks for patch Liyin.

          Show
          stack added a comment - Committed to trunk. Thanks for patch Liyin.
          Hide
          Lars Hofhansl added a comment -

          Moving out of 0.94. Pull back if you disagree.

          Show
          Lars Hofhansl added a comment - Moving out of 0.94. Pull back if you disagree.
          Hide
          Lars Hofhansl added a comment -

          Should we commit this?

          Show
          Lars Hofhansl added a comment - Should we commit this?
          Hide
          Nicolas Spiegelberg added a comment -

          correct. I am adding the null check 1 level deeper in the stack under SchemaMetrics.configureGlobally(). This will ensure that setUseTableName(false) is called.

          Show
          Nicolas Spiegelberg added a comment - correct. I am adding the null check 1 level deeper in the stack under SchemaMetrics.configureGlobally(). This will ensure that setUseTableName(false) is called.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12503953/HBASE-4802.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javadoc. The javadoc tool appears to have generated -163 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 51 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/268//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/268//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/268//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12503953/HBASE-4802.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -163 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 51 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/268//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/268//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/268//console This message is automatically generated.
          Hide
          stack added a comment -

          Is this right Nicolas? You add a null check but at same time are removing a null check.

          Show
          stack added a comment - Is this right Nicolas? You add a null check but at same time are removing a null check.
          Hide
          Nicolas Spiegelberg added a comment -

          Part of 89-fb to 92 port. See r1182037

          Show
          Nicolas Spiegelberg added a comment - Part of 89-fb to 92 port. See r1182037

            People

            • Assignee:
              Liyin Tang
              Reporter:
              Nicolas Spiegelberg
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development