Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5256

CombineInputFormat isn't thread safe affecting HiveServer

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.2.1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      This was originally fixed as part of MAPREDUCE-5038, but that got reverted now. Which uncovers this issue, breaking HiveServer. Originally reported by Thejas M Nair.

      1. MAPREDUCE-5256-20130517.txt
        3 kB
        Vinod Kumar Vavilapalli

        Issue Links

          Activity

          Matt Foley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Matt Foley made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Matt Foley added a comment - - edited

          +1. Committed to branch-1.2. Seems to be superfluous in branch-1. Thanks, Vinod!

          Show
          Matt Foley added a comment - - edited +1. Committed to branch-1.2. Seems to be superfluous in branch-1. Thanks, Vinod!
          Matt Foley made changes -
          Fix Version/s 1.2.1 [ 12324149 ]
          Hide
          Thejas M Nair added a comment -

          Thanks for the patch, after applying it, we don't see the issue.

          Show
          Thejas M Nair added a comment - Thanks for the patch, after applying it, we don't see the issue.
          Vinod Kumar Vavilapalli made changes -
          Link This issue is related to MAPREDUCE-5038 [ MAPREDUCE-5038 ]
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-5256-20130517.txt [ 12583700 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Here's a patch to fix this. Difficult to write test. Not useful much too.

          I'll request Thejas to verify this and report back.

          Show
          Vinod Kumar Vavilapalli added a comment - Here's a patch to fix this. Difficult to write test. Not useful much too. I'll request Thejas to verify this and report back.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Here's the exception trace from HiveServer, as Thejas reported to me:

          java.lang.NullPointerException
          at java.util.ArrayList.addAll(ArrayList.java:472)
          at org.apache.hadoop.mapred.lib.CombineFileInputFormat.getHosts(CombineFileInputFormat.java:568)
          at org.apache.hadoop.mapred.lib.CombineFileInputFormat.getMoreSplits(CombineFileInputFormat.java:410)
          at org.apache.hadoop.mapred.lib.CombineFileInputFormat.getSplits(CombineFileInputFormat.java:217)
          at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:409)
          at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:375)
          at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.processPaths(CombineHiveInputFormat.java:419)
          at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:390)
          at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:1051)
          at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1043)
          at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
          at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:959)
          at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)
          at java.security.AccessController.doPrivileged(Native Method)
          at javax.security.auth.Subject.doAs(Subject.java:396)
          at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1178)
          at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
          at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886)
          at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:447)
          at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
          at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
          at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
          at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1347)
          at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1130)
          at org.apache.hadoop.hive.ql.Driver.run(Driver.java:940)
          at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
          at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:644)
          at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:628)
          at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
          at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
          at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:662)
          Job Submission failed with exception 'java.lang.NullPointerException(null)'
          FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
          Hive history file=/var/log/hadoop/hive/tmp/root/hive_job_log_root_201305161238_1398906826.txt
          Hive history file=/var/log/hadoop/hive/tmp/root/hive_job_log_root_201305161238_2072032671.txt
          
          Show
          Vinod Kumar Vavilapalli added a comment - Here's the exception trace from HiveServer, as Thejas reported to me: java.lang.NullPointerException at java.util.ArrayList.addAll(ArrayList.java:472) at org.apache.hadoop.mapred.lib.CombineFileInputFormat.getHosts(CombineFileInputFormat.java:568) at org.apache.hadoop.mapred.lib.CombineFileInputFormat.getMoreSplits(CombineFileInputFormat.java:410) at org.apache.hadoop.mapred.lib.CombineFileInputFormat.getSplits(CombineFileInputFormat.java:217) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:409) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:375) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.processPaths(CombineHiveInputFormat.java:419) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:390) at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:1051) at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1043) at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:959) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1178) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912) at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:447) at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1347) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1130) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:940) at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:644) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:628) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang. Thread .run( Thread .java:662) Job Submission failed with exception 'java.lang.NullPointerException( null )' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask Hive history file=/ var /log/hadoop/hive/tmp/root/hive_job_log_root_201305161238_1398906826.txt Hive history file=/ var /log/hadoop/hive/tmp/root/hive_job_log_root_201305161238_2072032671.txt
          Vinod Kumar Vavilapalli made changes -
          Field Original Value New Value
          Project Hadoop Common [ 12310240 ] Hadoop Map/Reduce [ 12310941 ]
          Key HADOOP-9575 MAPREDUCE-5256
          Affects Version/s 1.2.0 [ 12321661 ]
          Affects Version/s 1.2.0 [ 12321659 ]
          Target Version/s 1.2.1 [ 12324147 ] 1.2.1 [ 12324149 ]
          Vinod Kumar Vavilapalli created issue -

            People

            • Assignee:
              Vinod Kumar Vavilapalli
              Reporter:
              Vinod Kumar Vavilapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development