Hive
  1. Hive
  2. HIVE-5515

Writing to an HBase table throws IllegalArgumentException, failing job submission

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.13.0
    • Component/s: HBase Handler
    • Labels:
    • Environment:

      Hadoop2, Hive 0.12.0, HBase-0.96RC

    • Tags:
      hbase-handler

      Description

      Inserting data into HBase table via hive query fails with the following message:

      $ hive -e "FROM pgc INSERT OVERWRITE TABLE pagecounts_hbase SELECT pgc.* WHERE rowkey LIKE 'en/q%' LIMIT 10;"
      ...
      Total MapReduce jobs = 1
      Launching Job 1 out of 1
      Number of reduce tasks determined at compile time: 1
      In order to change the average load for a reducer (in bytes):
        set hive.exec.reducers.bytes.per.reducer=<number>
      In order to limit the maximum number of reducers:
        set hive.exec.reducers.max=<number>
      In order to set a constant number of reducers:
        set mapred.reduce.tasks=<number>
      java.lang.IllegalArgumentException: Property value must not be null
              at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
              at org.apache.hadoop.conf.Configuration.set(Configuration.java:810)
              at org.apache.hadoop.conf.Configuration.set(Configuration.java:792)
              at org.apache.hadoop.hive.ql.exec.Utilities.copyTableJobPropertiesToConf(Utilities.java:2002)
              at org.apache.hadoop.hive.ql.exec.FileSinkOperator.checkOutputSpecs(FileSinkOperator.java:947)
              at org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl.checkOutputSpecs(HiveOutputFormatImpl.java:67)
              at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:458)
              at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)
              at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
              at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:415)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
              at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
              at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
              at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:415)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
              at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
              at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
              at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:425)
              at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136)
              at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:151)
              at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:65)
              at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1414)
              at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1192)
              at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1020)
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:888)
              at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
              at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
              at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
              at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
              at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:731)
              at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
              at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:601)
              at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
      Job Submission failed with exception 'java.lang.IllegalArgumentException(Property value must not be null)'
      FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
      
      1. HIVE-5515.1.patch
        2 kB
        Viraj Bhat
      2. HIVE-5515.2.patch
        2 kB
        Viraj Bhat
      3. HIVE-5515.patch
        3 kB
        Viraj Bhat

        Issue Links

          Activity

          Nick Dimiduk created issue -
          Hide
          Sushanth Sowmyan added a comment -

          This looks suspiciously similar to HIVE-5431, linking the two.

          Show
          Sushanth Sowmyan added a comment - This looks suspiciously similar to HIVE-5431 , linking the two.
          Sushanth Sowmyan made changes -
          Field Original Value New Value
          Link This issue relates to HIVE-5431 [ HIVE-5431 ]
          Hide
          Viraj Bhat added a comment -

          Sushanth did this Jira get fixed with HIVE-5431

          Show
          Viraj Bhat added a comment - Sushanth did this Jira get fixed with HIVE-5431
          Hide
          Viraj Bhat added a comment -

          The main reason for this error is that:
          hive.passthrough.storagehandler.of does not get set.

          Checking why this is the case. The unit tests pass without any problems!!

          Viraj

          Show
          Viraj Bhat added a comment - The main reason for this error is that: hive.passthrough.storagehandler.of does not get set. Checking why this is the case. The unit tests pass without any problems!! Viraj
          Hide
          Sushanth Sowmyan added a comment -

          Yeah, the unit test passes with a similar scenario, but this fails when run on a standalone command. It is the same key here too.

          Show
          Sushanth Sowmyan added a comment - Yeah, the unit test passes with a similar scenario, but this fails when run on a standalone command. It is the same key here too.
          Hide
          Sushanth Sowmyan added a comment -

          And no, HIVE-5431 did not fix this problem - that fixed it only on the read side, where it made sense that the property had not been set, and therefore, should not have been copied. In this case, it makes sense that it should be copied, but does not make sense that it has not yet been set.

          Show
          Sushanth Sowmyan added a comment - And no, HIVE-5431 did not fix this problem - that fixed it only on the read side, where it made sense that the property had not been set, and therefore, should not have been copied. In this case, it makes sense that it should be copied, but does not make sense that it has not yet been set.
          Hide
          Viraj Bhat added a comment -

          Sushanth,the main reason this is failing is because:
          getOutputFormatSubstitute() function is not called inside the Table.java at the point where the storagehandlerflag is true. Hence the HiveFileFormatUtils.setRealOutputFormatClassName() is not triggered. I am wondering how this is possible when running in real cluster mode and not in the unit testcase mode.

          Show
          Viraj Bhat added a comment - Sushanth,the main reason this is failing is because: getOutputFormatSubstitute() function is not called inside the Table.java at the point where the storagehandlerflag is true. Hence the HiveFileFormatUtils.setRealOutputFormatClassName() is not triggered. I am wondering how this is possible when running in real cluster mode and not in the unit testcase mode.
          Viraj Bhat made changes -
          Assignee Viraj Bhat [ viraj ]
          Hide
          Viraj Bhat added a comment -

          Hi Sushanth,
          I think this is a major issue that if tables created using the HBaseStorageHandler in Hive 12 they cannot be populated using the command line. I found out the root cause of the problem. When the unit tests are run the tTable.getSd().getOutputFormat() in Table.java returns null and the real output format is populated. When a table is already created and we are using an "INSERT OVERWRITE" the tTable.getSd().getOutputFormat() returns the HivePassThroughOutputFormat and the real output format is not populated hence the null pointer exception.

          Also took this opportunity to make the call in checkOutputSpecs in FileSinkOperator.java from getConf().getTableInfo() to conf.getTableInfo()
          Attaching patch and will let the CI build pick this up.

          Regards
          Viraj

          Show
          Viraj Bhat added a comment - Hi Sushanth, I think this is a major issue that if tables created using the HBaseStorageHandler in Hive 12 they cannot be populated using the command line. I found out the root cause of the problem. When the unit tests are run the tTable.getSd().getOutputFormat() in Table.java returns null and the real output format is populated. When a table is already created and we are using an "INSERT OVERWRITE" the tTable.getSd().getOutputFormat() returns the HivePassThroughOutputFormat and the real output format is not populated hence the null pointer exception. Also took this opportunity to make the call in checkOutputSpecs in FileSinkOperator.java from getConf().getTableInfo() to conf.getTableInfo() Attaching patch and will let the CI build pick this up. Regards Viraj
          Hide
          Viraj Bhat added a comment -

          Correct INSERT OVERWRITE for tables created using the HBaseStorageHandler

          Show
          Viraj Bhat added a comment - Correct INSERT OVERWRITE for tables created using the HBaseStorageHandler
          Viraj Bhat made changes -
          Attachment HIVE-5515.patch [ 12608629 ]
          Viraj Bhat made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Labels hbase
          Fix Version/s 0.12.0 [ 12324312 ]
          Tags hbase-handler
          Hide
          Viraj Bhat added a comment -

          Hi Nick,
          Can you check if this patch solves the null pointer exception on your cluster.
          Viraj

          Show
          Viraj Bhat added a comment - Hi Nick, Can you check if this patch solves the null pointer exception on your cluster. Viraj
          Hide
          Nick Dimiduk added a comment -

          Thanks for the patch Viraj. I'll give this a spin tomorrow, assuming I can work out how to build it

          FYI, I saw the same error using both INSERT INTO and INSERT OVERWRITE.

          Show
          Nick Dimiduk added a comment - Thanks for the patch Viraj. I'll give this a spin tomorrow, assuming I can work out how to build it FYI, I saw the same error using both INSERT INTO and INSERT OVERWRITE.
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12608629/HIVE-5515.patch

          SUCCESS: +1 4411 tests passed

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1140/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1140/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12608629/HIVE-5515.patch SUCCESS: +1 4411 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1140/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1140/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
          Hide
          Viraj Bhat added a comment -

          Hi Thejas, Sushanth,
          Can we review/ commit this patch if possible, we have tested this internally on the clusters, it works. Without this we cannot populate the table created using the HBaseStorageHandler in Hive.
          Regards
          Viraj

          Show
          Viraj Bhat added a comment - Hi Thejas, Sushanth, Can we review/ commit this patch if possible, we have tested this internally on the clusters, it works. Without this we cannot populate the table created using the HBaseStorageHandler in Hive. Regards Viraj
          Hide
          Nick Dimiduk added a comment -

          My hardware was gobbled up recently so I haven't tested the patch yet. Hopefully next week.

          Show
          Nick Dimiduk added a comment - My hardware was gobbled up recently so I haven't tested the patch yet. Hopefully next week.
          Navis made changes -
          Fix Version/s 0.13.0 [ 12324986 ]
          Fix Version/s 0.12.0 [ 12324312 ]
          Hide
          Viraj Bhat added a comment -

          Hi Nick,
          Our QE tested this on a Hadoop cluster. It now populates data
          Viraj

          Show
          Viraj Bhat added a comment - Hi Nick, Our QE tested this on a Hadoop cluster. It now populates data Viraj
          Hide
          Nick Dimiduk added a comment -

          Hi Viraj Bhat.

          I'm working with Sushanth Sowmyan to get a working build that I can test. In the mean time, I think the patch looks good. Can you also fix the unit test so that it would catch this issue in the future?

          Thanks,
          Nick

          Show
          Nick Dimiduk added a comment - Hi Viraj Bhat . I'm working with Sushanth Sowmyan to get a working build that I can test. In the mean time, I think the patch looks good. Can you also fix the unit test so that it would catch this issue in the future? Thanks, Nick
          Hide
          Viraj Bhat added a comment -

          Hi Nick,
          Currently none of the testcases can reproduce this behavior. My suspicion is we need to have an actual Hive set up where the metstore is populated with metadata.
          Viraj

          Show
          Viraj Bhat added a comment - Hi Nick, Currently none of the testcases can reproduce this behavior. My suspicion is we need to have an actual Hive set up where the metstore is populated with metadata. Viraj
          Hide
          Sushanth Sowmyan added a comment -

          Hi Viraj,

          With your changes, my experiments in my vm seem to succeed. However, as with HIVE-5431, we really do need to have a unit test for this. I pushed for that one making it through without a test at that time because it was difficult to re-create as a test, and we could clearly demonstrate an obvious problem and prove we fixed it. Now that we aren't looking at 0.13 release deadlines yet, I think we should spend some time trying to create a test for this.

          At the base, it looks like we can create a test for this by writing a junit unit test that manages mulitple hive client instances, each of which resets the conf at the every access.

          For HIVE-5431, I was considering something like writing out a hbase table directly using hbase/hfiles and then trying to read it as a hbase table from hive. Also possible is a change to the QTestUtil framework so as to have a param that allows us to reset confs between calls.

          Show
          Sushanth Sowmyan added a comment - Hi Viraj, With your changes, my experiments in my vm seem to succeed. However, as with HIVE-5431 , we really do need to have a unit test for this. I pushed for that one making it through without a test at that time because it was difficult to re-create as a test, and we could clearly demonstrate an obvious problem and prove we fixed it. Now that we aren't looking at 0.13 release deadlines yet, I think we should spend some time trying to create a test for this. At the base, it looks like we can create a test for this by writing a junit unit test that manages mulitple hive client instances, each of which resets the conf at the every access. For HIVE-5431 , I was considering something like writing out a hbase table directly using hbase/hfiles and then trying to read it as a hbase table from hive. Also possible is a change to the QTestUtil framework so as to have a param that allows us to reset confs between calls.
          Hide
          Nick Dimiduk added a comment -

          Thanks again for the patch, Viraj Bhat.

          I was able to test your patch on my cluster – problem solved!

          For historical purposes, this is the complete command I ran on my HDP-2.0 cluster. Continuing from my example above:

          HADOOP_CLASSPATH=/etc/hbase/conf:/usr/lib/hbase/lib/hbase-server-0.96.0.2.0.6.0-76-hadoop2.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hbase/lib/hbase-hadoop-compat-0.96.0.2.0.6.0-76-hadoop2.jar:/usr/lib/hbase/lib/hbase-common-0.96.0.2.0.6.0-76-hadoop2.jar:/usr/lib/hbase/lib/htrace-core-2.01.jar:/usr/lib/hbase/lib/hbase-protocol-0.96.0.2.0.6.0-76-hadoop2.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hbase/lib/hbase-client-0.96.0.2.0.6.0-76-hadoop2.jar:/usr/lib/zookeeper/zookeeper-3.4.5.2.0.6.0-76.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar hive -e "SET hive.aux.jars.path = file:///etc/hbase/conf/hbase-site.xml,file:///usr/lib/hive/lib/hive-hbase-handler-0.12.0.2.0.6.0-76.jar,file:///usr/lib/hbase/lib/hbase-server-0.96.0.2.0.6.0-76-hadoop2.jar,file:///usr/lib/hadoop/lib/guava-11.0.2.jar,file:///usr/lib/hbase/lib/hbase-hadoop-compat-0.96.0.2.0.6.0-76-hadoop2.jar,file:///usr/lib/hbase/lib/hbase-common-0.96.0.2.0.6.0-76-hadoop2.jar,file:///usr/lib/hbase/lib/htrace-core-2.01.jar,file:///usr/lib/hbase/lib/hbase-protocol-0.96.0.2.0.6.0-76-hadoop2.jar,file:///usr/lib/hadoop/lib/netty-3.6.2.Final.jar,file:///usr/lib/hbase/lib/hbase-client-0.96.0.2.0.6.0-76-hadoop2.jar,file:///usr/lib/zookeeper/zookeeper-3.4.5.2.0.6.0-76.jar,file:///usr/lib/hadoop/lib/protobuf-java-2.5.0.jar; FROM pgc INSERT INTO TABLE pagecounts_hbase SELECT pgc.* WHERE rowkey LIKE 'en/q%' LIMIT 10;"
          
          Show
          Nick Dimiduk added a comment - Thanks again for the patch, Viraj Bhat . I was able to test your patch on my cluster – problem solved! For historical purposes, this is the complete command I ran on my HDP-2.0 cluster. Continuing from my example above: HADOOP_CLASSPATH=/etc/hbase/conf:/usr/lib/hbase/lib/hbase-server-0.96.0.2.0.6.0-76-hadoop2.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hbase/lib/hbase-hadoop-compat-0.96.0.2.0.6.0-76-hadoop2.jar:/usr/lib/hbase/lib/hbase-common-0.96.0.2.0.6.0-76-hadoop2.jar:/usr/lib/hbase/lib/htrace-core-2.01.jar:/usr/lib/hbase/lib/hbase-protocol-0.96.0.2.0.6.0-76-hadoop2.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hbase/lib/hbase-client-0.96.0.2.0.6.0-76-hadoop2.jar:/usr/lib/zookeeper/zookeeper-3.4.5.2.0.6.0-76.jar:/usr/lib/hadoop/lib/protobuf-java-2.5.0.jar hive -e "SET hive.aux.jars.path = file:///etc/hbase/conf/hbase-site.xml,file:///usr/lib/hive/lib/hive-hbase-handler-0.12.0.2.0.6.0-76.jar,file:///usr/lib/hbase/lib/hbase-server-0.96.0.2.0.6.0-76-hadoop2.jar,file:///usr/lib/hadoop/lib/guava-11.0.2.jar,file:///usr/lib/hbase/lib/hbase-hadoop-compat-0.96.0.2.0.6.0-76-hadoop2.jar,file:///usr/lib/hbase/lib/hbase-common-0.96.0.2.0.6.0-76-hadoop2.jar,file:///usr/lib/hbase/lib/htrace-core-2.01.jar,file:///usr/lib/hbase/lib/hbase-protocol-0.96.0.2.0.6.0-76-hadoop2.jar,file:///usr/lib/hadoop/lib/netty-3.6.2.Final.jar,file:///usr/lib/hbase/lib/hbase-client-0.96.0.2.0.6.0-76-hadoop2.jar,file:///usr/lib/zookeeper/zookeeper-3.4.5.2.0.6.0-76.jar,file:///usr/lib/hadoop/lib/protobuf-java-2.5.0.jar; FROM pgc INSERT INTO TABLE pagecounts_hbase SELECT pgc.* WHERE rowkey LIKE 'en/q%' LIMIT 10;"
          Hide
          Sushanth Sowmyan added a comment -

          Hi Viraj,

          I've done some more tests, and am +1 with the solution. A couple of nitpicks though, apart from my request for tests.

          a) Please do not use conf directly, stick with getConf()
          b) Please try to limit the length of individual lines, having lines with 220 chars make for more unreadable code - I believe hive actually has a stylecheck rule for length, but I forget what's the limit (checkstyle.xml says 2000, but that's ridiculous) - I would try to keep it within about 80 if possible,

          Show
          Sushanth Sowmyan added a comment - Hi Viraj, I've done some more tests, and am +1 with the solution. A couple of nitpicks though, apart from my request for tests. a) Please do not use conf directly, stick with getConf() b) Please try to limit the length of individual lines, having lines with 220 chars make for more unreadable code - I believe hive actually has a stylecheck rule for length, but I forget what's the limit (checkstyle.xml says 2000, but that's ridiculous) - I would try to keep it within about 80 if possible,
          Hide
          Viraj Bhat added a comment -

          Hi Sushanth,
          Thanks for your comments about the unit test case. I think the current test cases do not exercise the path where they read the metadata from the metastore. Also about fixing the patch. Let me use "conf" and also limit the individual lines to 80. I will repost it as soon as possible.
          Viraj

          Show
          Viraj Bhat added a comment - Hi Sushanth, Thanks for your comments about the unit test case. I think the current test cases do not exercise the path where they read the metadata from the metastore. Also about fixing the patch. Let me use "conf" and also limit the individual lines to 80. I will repost it as soon as possible. Viraj
          Viraj Bhat made changes -
          Attachment HIVE-5515.1.patch [ 12615700 ]
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12615700/HIVE-5515.1.patch

          ERROR: -1 due to 1 failed/errored test(s), 4684 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
          

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/445/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/445/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests failed with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12615700

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12615700/HIVE-5515.1.patch ERROR: -1 due to 1 failed/errored test(s), 4684 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/445/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/445/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12615700
          Hide
          Nick Dimiduk added a comment -

          Ping Viraj Bhat. Any word on the updated patch?

          Show
          Nick Dimiduk added a comment - Ping Viraj Bhat . Any word on the updated patch?
          Swarnim Kulkarni made changes -
          Link This issue duplicates HIVE-5680 [ HIVE-5680 ]
          Swarnim Kulkarni made changes -
          Link This issue is duplicated by HIVE-5680 [ HIVE-5680 ]
          Swarnim Kulkarni made changes -
          Link This issue duplicates HIVE-5680 [ HIVE-5680 ]
          Hide
          Viraj Bhat added a comment -

          Hi Nick,
          The updated patch was posted on HIVE-5515.1.patch was posted on 25/Nov/13 14:40 where I made the changes Sushanth suggested. The 1 test case which failed might be not related. If possible I can re-run it. Please test this patch and it should work.
          Regards
          Viraj

          Show
          Viraj Bhat added a comment - Hi Nick, The updated patch was posted on HIVE-5515 .1.patch was posted on 25/Nov/13 14:40 where I made the changes Sushanth suggested. The 1 test case which failed might be not related. If possible I can re-run it. Please test this patch and it should work. Regards Viraj
          Hide
          Viraj Bhat added a comment -

          Reattaching patch for running CI builds

          Show
          Viraj Bhat added a comment - Reattaching patch for running CI builds
          Viraj Bhat made changes -
          Attachment HIVE-5515.2.patch [ 12622363 ]
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12622363/HIVE-5515.2.patch

          SUCCESS: +1 4917 tests passed

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/850/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/850/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12622363

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12622363/HIVE-5515.2.patch SUCCESS: +1 4917 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/850/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/850/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12622363
          Hide
          Viraj Bhat added a comment -

          Sushanth, Thejas, Ashutosh and Nick, could you please take a look and commit if possible.
          Viraj

          Show
          Viraj Bhat added a comment - Sushanth, Thejas, Ashutosh and Nick, could you please take a look and commit if possible. Viraj
          Hide
          Sushanth Sowmyan added a comment -

          I'm good with this latest patch, and as to my request for tests, since this is pretty hard to write a test for with our current test framework, but shows up in the e2e tests on actual clusters, I'm okay with committing this, and having a new jira created for a follow up task for that.

          Show
          Sushanth Sowmyan added a comment - I'm good with this latest patch, and as to my request for tests, since this is pretty hard to write a test for with our current test framework, but shows up in the e2e tests on actual clusters, I'm okay with committing this, and having a new jira created for a follow up task for that.
          Hide
          Sushanth Sowmyan added a comment -

          Ashutosh Chauhan, I'm +1 on this patch. Let's get it in and close this jira.

          Viraj Bhat, could you create another jira for a test case that would illustrate this issue?

          Show
          Sushanth Sowmyan added a comment - Ashutosh Chauhan , I'm +1 on this patch. Let's get it in and close this jira. Viraj Bhat , could you create another jira for a test case that would illustrate this issue?
          Hide
          Viraj Bhat added a comment -

          Sushanth, thanks for your help. Let me create another Jira and link it to HIVE-5260

          Show
          Viraj Bhat added a comment - Sushanth, thanks for your help. Let me create another Jira and link it to HIVE-5260
          Hide
          Ashutosh Chauhan added a comment -

          +1

          Show
          Ashutosh Chauhan added a comment - +1
          Hide
          Viraj Bhat added a comment -

          Ashutosh please commit when you get time.
          Viraj

          Show
          Viraj Bhat added a comment - Ashutosh please commit when you get time. Viraj
          Hide
          Viraj Bhat added a comment -

          Created HIVE-6195 to address the Sushanth concern.

          Show
          Viraj Bhat added a comment - Created HIVE-6195 to address the Sushanth concern.
          Hide
          Ashutosh Chauhan added a comment -

          Committed to trunk. Thanks, Viraj & Sushanth!

          Show
          Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Viraj & Sushanth!
          Ashutosh Chauhan made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]

            People

            • Assignee:
              Viraj Bhat
              Reporter:
              Nick Dimiduk
            • Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development