HBase
  1. HBase
  2. HBASE-11378

TableMapReduceUtil overwrites user supplied options for multiple tables/scaners job

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.0, 0.98.4
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In TableMapReduceUtil#initTableMapperJob, we have

      HBaseConfiguration.addHbaseResources(job.getConfiguration());

      It should use merge instead. Otherwise, user supplied options are overwritten.

      1. hbase-11378.patch
        0.8 kB
        Jimmy Xiang

        Activity

        Hide
        Enis Soztutar added a comment -

        Closing this issue after 0.99.0 release.

        Show
        Enis Soztutar added a comment - Closing this issue after 0.99.0 release.
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #328 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/328/)
        HBASE-11378 TableMapReduceUtil overwrites user supplied options for multiple tables/scaners job (jxiang: rev 23cd02a21cae7b98b12178cd04fb2e88aa56b4a2)

        • hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #328 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/328/ ) HBASE-11378 TableMapReduceUtil overwrites user supplied options for multiple tables/scaners job (jxiang: rev 23cd02a21cae7b98b12178cd04fb2e88aa56b4a2) hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
        Hide
        Nick Dimiduk added a comment -

        Nice one Jimmy!

        Show
        Nick Dimiduk added a comment - Nice one Jimmy!
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.98 #347 (See https://builds.apache.org/job/HBase-0.98/347/)
        HBASE-11378 TableMapReduceUtil overwrites user supplied options for multiple tables/scaners job (jxiang: rev 23cd02a21cae7b98b12178cd04fb2e88aa56b4a2)

        • hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.98 #347 (See https://builds.apache.org/job/HBase-0.98/347/ ) HBASE-11378 TableMapReduceUtil overwrites user supplied options for multiple tables/scaners job (jxiang: rev 23cd02a21cae7b98b12178cd04fb2e88aa56b4a2) hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK #5219 (See https://builds.apache.org/job/HBase-TRUNK/5219/)
        HBASE-11378 TableMapReduceUtil overwrites user supplied options for multiple tables/scaners job (jxiang: rev 45bc13d87a08dd56c01a20e3b574f85882500810)

        • hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #5219 (See https://builds.apache.org/job/HBase-TRUNK/5219/ ) HBASE-11378 TableMapReduceUtil overwrites user supplied options for multiple tables/scaners job (jxiang: rev 45bc13d87a08dd56c01a20e3b574f85882500810) hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
        Hide
        Jimmy Xiang added a comment -

        Integrated into trunk and 0.98. Thanks. The zombie test could be TestHRegionBusyWait since I didn't see it in the test report. However, this test should not be affected. It seems to be fine locally.

        Show
        Jimmy Xiang added a comment - Integrated into trunk and 0.98. Thanks. The zombie test could be TestHRegionBusyWait since I didn't see it in the test report. However, this test should not be affected. It seems to be fine locally.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12651437/hbase-11378.patch
        against trunk revision .
        ATTACHMENT ID: 12651437

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:

        -1 core zombie tests. There are 1 zombie test(s):

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12651437/hbase-11378.patch against trunk revision . ATTACHMENT ID: 12651437 +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: -1 core zombie tests . There are 1 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9795//console This message is automatically generated.
        Hide
        Jimmy Xiang added a comment -

        Some MR job on secure HBase failed. We suspected the configuration was not picked up. It turns out to be lost on the way.

        Show
        Jimmy Xiang added a comment - Some MR job on secure HBase failed. We suspected the configuration was not picked up. It turns out to be lost on the way.
        Hide
        Matteo Bertozzi added a comment -

        +1, how did you notice the problem? an "ignored" -D or something else?

        Show
        Matteo Bertozzi added a comment - +1, how did you notice the problem? an "ignored" -D or something else?

          People

          • Assignee:
            Jimmy Xiang
            Reporter:
            Jimmy Xiang
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development