Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.9.0
    • Component/s: Query Processor
    • Labels:
      None
    • Environment:

      Reproduced only on hadoop-0.20.2-CDH3u1, work fine on hadoop-0.20.2

    • Hadoop Flags:
      Reviewed

      Description

      Trying table sampling on any non-empty table throws NPE. This does not occur by test on mini-MR.

      select count(*) from emp tablesample (0.1 percent);     
      Total MapReduce jobs = 1
      Launching Job 1 out of 1
      Number of reduce tasks determined at compile time: 1
      In order to change the average load for a reducer (in bytes):
        set hive.exec.reducers.bytes.per.reducer=<number>
      In order to limit the maximum number of reducers:
        set hive.exec.reducers.max=<number>
      In order to set a constant number of reducers:
        set mapred.reduce.tasks=<number>
      java.lang.NullPointerException
      	at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.sampleSplits(CombineHiveInputFormat.java:450)
      	at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:403)
      	at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:971)
      	at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:963)
      	at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
      	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880)
      	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:396)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
      	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
      	at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807)
      	at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:432)
      	at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
      	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
      	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
      	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
      	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
      	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
      	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
      	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
      	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
      	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:597)
      	at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
      Job Submission failed with exception 'java.lang.NullPointerException(null)'
      FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
      
      

        Issue Links

          Activity

          Hide
          Phabricator added a comment -

          navis requested code review of "HIVE-2778 [jira] Fail on table sampling".
          Reviewers: JIRA

          HIVE-2778 fix NPE on table sampling

          Trying table sampling on any non-empty table throws NPE. This does not occur by test on mini-MR. <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>select count from emp tablesample (0.1 percent); Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks determined at compile time: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=<number> In order to limit the maximum number of reducers: set hive.exec.reducers.max=<number> In order to set a constant number of reducers: set mapred.reduce.tasks=<number> java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.sampleSplits(CombineHiveInputFormat.java:450) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:403) at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:971) at org.apache.hadoop.mapred.
          JobClient.writeSplits(JobClient.java:963) at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833) at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:432) at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332) at org.apache.hadoop.hive.ql.D
          river.execute(Driver.java:1123) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:186) Job Submission failed with exception 'java.lang.NullPointerException(null)' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask </pre> </div></div>

          TEST PLAN
          EMPTY

          REVISION DETAIL
          https://reviews.facebook.net/D1593

          AFFECTED FILES
          ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/3375/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - navis requested code review of " HIVE-2778 [jira] Fail on table sampling". Reviewers: JIRA HIVE-2778 fix NPE on table sampling Trying table sampling on any non-empty table throws NPE. This does not occur by test on mini-MR. <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>select count from emp tablesample (0.1 percent); Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks determined at compile time: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=<number> In order to limit the maximum number of reducers: set hive.exec.reducers.max=<number> In order to set a constant number of reducers: set mapred.reduce.tasks=<number> java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.sampleSplits(CombineHiveInputFormat.java:450) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:403) at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:971) at org.apache.hadoop.mapred. JobClient.writeSplits(JobClient.java:963) at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833) at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:432) at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332) at org.apache.hadoop.hive.ql.D river.execute(Driver.java:1123) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:186) Job Submission failed with exception 'java.lang.NullPointerException(null)' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask </pre> </div></div> TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1593 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/3375/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
          Hide
          Phabricator added a comment -

          navis updated the revision "HIVE-2778 [jira] Fail on table sampling".
          Reviewers: JIRA

          Walk around for hadoop-0.20.2-cdh3u1

          REVISION DETAIL
          https://reviews.facebook.net/D1593

          AFFECTED FILES
          ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java

          Show
          Phabricator added a comment - navis updated the revision " HIVE-2778 [jira] Fail on table sampling". Reviewers: JIRA Walk around for hadoop-0.20.2-cdh3u1 REVISION DETAIL https://reviews.facebook.net/D1593 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java
          Hide
          Phabricator added a comment -

          cwsteinbach has accepted the revision "HIVE-2778 [jira] Fail on table sampling".

          +1. Will commit if tests pass.

          REVISION DETAIL
          https://reviews.facebook.net/D1593

          BRANCH
          DPAL-724

          Show
          Phabricator added a comment - cwsteinbach has accepted the revision " HIVE-2778 [jira] Fail on table sampling". +1. Will commit if tests pass. REVISION DETAIL https://reviews.facebook.net/D1593 BRANCH DPAL-724
          Hide
          Carl Steinbach added a comment -

          @Navis: Can you please grant license rights to the ASF? Patch looks good otherwise. Thanks.

          Show
          Carl Steinbach added a comment - @Navis: Can you please grant license rights to the ASF? Patch looks good otherwise. Thanks.
          Hide
          Navis added a comment -

          Granted license to ASF.. Couldn't it be automated by phabricator?

          Show
          Navis added a comment - Granted license to ASF.. Couldn't it be automated by phabricator?
          Hide
          Carl Steinbach added a comment -

          Committed to trunk. Thanks Navis!

          Show
          Carl Steinbach added a comment - Committed to trunk. Thanks Navis!
          Hide
          Phabricator added a comment -

          navis has committed the revision "HIVE-2778 [jira] Fail on table sampling".

          Change committed by cws.

          REVISION DETAIL
          https://reviews.facebook.net/D1593

          COMMIT
          https://reviews.facebook.net/rHIVE1301310

          Show
          Phabricator added a comment - navis has committed the revision " HIVE-2778 [jira] Fail on table sampling". Change committed by cws. REVISION DETAIL https://reviews.facebook.net/D1593 COMMIT https://reviews.facebook.net/rHIVE1301310
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1313 (See https://builds.apache.org/job/Hive-trunk-h0.21/1313/)
          HIVE-2778 [jira] Fail on table sampling
          (Navis Ryu via Carl Steinbach)

          Summary:
          HIVE-2778 fix NPE on table sampling

          Trying table sampling on any non-empty table throws NPE. This does not occur by
          test on mini-MR. <div class="preformatted panel" style="border-width:
          1px;"><div class="preformattedContent panelContent"> <pre>select count from
          emp tablesample (0.1 percent); Total MapReduce jobs = 1 Launching Job 1 out
          of 1 Number of reduce tasks determined at compile time: 1 In order to change the
          average load for a reducer (in bytes): set
          hive.exec.reducers.bytes.per.reducer=<number> In order to limit the maximum
          number of reducers: set hive.exec.reducers.max=<number> In order to set a
          constant number of reducers: set mapred.reduce.tasks=<number>
          java.lang.NullPointerException at
          org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.sampleSplits(CombineHiveInputFormat.java:450)
          at
          org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:403)
          at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:971) at
          org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:963) at
          org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170) at
          org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880) at
          org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833) at
          java.security.AccessController.doPrivileged(Native Method) at
          javax.security.auth.Subject.doAs(Subject.java:396) at
          org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
          at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833) at
          org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807) at
          org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:432) at
          org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136) at
          org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134) at
          org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at
          org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332) at
          org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123) at
          org.apache.hadoop.hive.ql.Driver.run(Driver.java:931) at
          org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at
          org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at
          org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at
          org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at
          org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at
          sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
          sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at
          sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597) at
          org.apache.hadoop.util.RunJar.main(RunJar.java:186) Job Submission failed with
          exception 'java.lang.NullPointerException(null)' FAILED: Execution Error, return
          code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask </pre> </div></div>

          Test Plan: EMPTY

          Reviewers: JIRA, cwsteinbach

          Reviewed By: cwsteinbach

          Differential Revision: https://reviews.facebook.net/D1593 (Revision 1301310)

          Result = FAILURE
          cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301310
          Files :

          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1313 (See https://builds.apache.org/job/Hive-trunk-h0.21/1313/ ) HIVE-2778 [jira] Fail on table sampling (Navis Ryu via Carl Steinbach) Summary: HIVE-2778 fix NPE on table sampling Trying table sampling on any non-empty table throws NPE. This does not occur by test on mini-MR. <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>select count from emp tablesample (0.1 percent); Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks determined at compile time: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=<number> In order to limit the maximum number of reducers: set hive.exec.reducers.max=<number> In order to set a constant number of reducers: set mapred.reduce.tasks=<number> java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.sampleSplits(CombineHiveInputFormat.java:450) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:403) at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:971) at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:963) at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833) at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:432) at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:186) Job Submission failed with exception 'java.lang.NullPointerException(null)' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask </pre> </div></div> Test Plan: EMPTY Reviewers: JIRA, cwsteinbach Reviewed By: cwsteinbach Differential Revision: https://reviews.facebook.net/D1593 (Revision 1301310) Result = FAILURE cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301310 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java
          Hide
          Ashutosh Chauhan added a comment -

          This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.

          Show
          Ashutosh Chauhan added a comment - This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
          HIVE-2778 [jira] Fail on table sampling
          (Navis Ryu via Carl Steinbach)

          Summary:
          HIVE-2778 fix NPE on table sampling

          Trying table sampling on any non-empty table throws NPE. This does not occur by
          test on mini-MR. <div class="preformatted panel" style="border-width:
          1px;"><div class="preformattedContent panelContent"> <pre>select count from
          emp tablesample (0.1 percent); Total MapReduce jobs = 1 Launching Job 1 out
          of 1 Number of reduce tasks determined at compile time: 1 In order to change the
          average load for a reducer (in bytes): set
          hive.exec.reducers.bytes.per.reducer=<number> In order to limit the maximum
          number of reducers: set hive.exec.reducers.max=<number> In order to set a
          constant number of reducers: set mapred.reduce.tasks=<number>
          java.lang.NullPointerException at
          org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.sampleSplits(CombineHiveInputFormat.java:450)
          at
          org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:403)
          at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:971) at
          org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:963) at
          org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170) at
          org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880) at
          org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833) at
          java.security.AccessController.doPrivileged(Native Method) at
          javax.security.auth.Subject.doAs(Subject.java:396) at
          org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
          at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833) at
          org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807) at
          org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:432) at
          org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136) at
          org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134) at
          org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at
          org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332) at
          org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123) at
          org.apache.hadoop.hive.ql.Driver.run(Driver.java:931) at
          org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at
          org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at
          org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at
          org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at
          org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at
          sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
          sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at
          sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597) at
          org.apache.hadoop.util.RunJar.main(RunJar.java:186) Job Submission failed with
          exception 'java.lang.NullPointerException(null)' FAILED: Execution Error, return
          code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask </pre> </div></div>

          Test Plan: EMPTY

          Reviewers: JIRA, cwsteinbach

          Reviewed By: cwsteinbach

          Differential Revision: https://reviews.facebook.net/D1593 (Revision 1301310)

          Result = ABORTED
          cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301310
          Files :

          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-2778 [jira] Fail on table sampling (Navis Ryu via Carl Steinbach) Summary: HIVE-2778 fix NPE on table sampling Trying table sampling on any non-empty table throws NPE. This does not occur by test on mini-MR. <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>select count from emp tablesample (0.1 percent); Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks determined at compile time: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=<number> In order to limit the maximum number of reducers: set hive.exec.reducers.max=<number> In order to set a constant number of reducers: set mapred.reduce.tasks=<number> java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.sampleSplits(CombineHiveInputFormat.java:450) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:403) at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:971) at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:963) at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833) at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:432) at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:186) Job Submission failed with exception 'java.lang.NullPointerException(null)' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask </pre> </div></div> Test Plan: EMPTY Reviewers: JIRA, cwsteinbach Reviewed By: cwsteinbach Differential Revision: https://reviews.facebook.net/D1593 (Revision 1301310) Result = ABORTED cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301310 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java

            People

            • Assignee:
              Navis
              Reporter:
              Navis
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development