Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5256

CombineInputFormat isn't thread safe affecting HiveServer

        Details

        • Type: Bug
        • Status: Closed
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: 1.2.0
        • Fix Version/s: 1.2.1
        • Component/s: None
        • Labels:
          None
        • Target Version/s:

          Description

          This was originally fixed as part of MAPREDUCE-5038, but that got reverted now. Which uncovers this issue, breaking HiveServer. Originally reported by Thejas M Nair.

          1. Text File
            MAPREDUCE-5256-20130517.txt
            3 kB
            Vinod Kumar Vavilapalli

            Issue Links

            is related to

            Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-5038 old API CombineFileInputFormat missing fixes that are in new API

            • Major - Major loss of function.
            • Reopened

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              vinodkv Vinod Kumar Vavilapalli added a comment -

              Here's the exception trace from HiveServer, as Thejas reported to me:

              java.lang.NullPointerException
at java.util.ArrayList.addAll(ArrayList.java:472)
at org.apache.hadoop.mapred.lib.CombineFileInputFormat.getHosts(CombineFileInputFormat.java:568)
at org.apache.hadoop.mapred.lib.CombineFileInputFormat.getMoreSplits(CombineFileInputFormat.java:410)
at org.apache.hadoop.mapred.lib.CombineFileInputFormat.getSplits(CombineFileInputFormat.java:217)
at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:409)
at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:375)
at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.processPaths(CombineHiveInputFormat.java:419)
at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:390)
at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:1051)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1043)
at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:959)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1178)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886)
at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:447)
at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1347)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1130)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:940)
at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:644)
at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:628)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Job Submission failed with exception 'java.lang.NullPointerException(null)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
Hive history file=/var/log/hadoop/hive/tmp/root/hive_job_log_root_201305161238_1398906826.txt
Hive history file=/var/log/hadoop/hive/tmp/root/hive_job_log_root_201305161238_2072032671.txt
              Show
              vinodkv Vinod Kumar Vavilapalli added a comment - Here's the exception trace from HiveServer, as Thejas reported to me: java.lang.NullPointerException at java.util.ArrayList.addAll(ArrayList.java:472) at org.apache.hadoop.mapred.lib.CombineFileInputFormat.getHosts(CombineFileInputFormat.java:568) at org.apache.hadoop.mapred.lib.CombineFileInputFormat.getMoreSplits(CombineFileInputFormat.java:410) at org.apache.hadoop.mapred.lib.CombineFileInputFormat.getSplits(CombineFileInputFormat.java:217) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:409) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:375) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.processPaths(CombineHiveInputFormat.java:419) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:390) at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:1051) at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1043) at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:959) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1178) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912) at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:447) at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1347) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1130) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:940) at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:644) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:628) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang. Thread .run( Thread .java:662) Job Submission failed with exception 'java.lang.NullPointerException( null )' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask Hive history file=/ var /log/hadoop/hive/tmp/root/hive_job_log_root_201305161238_1398906826.txt Hive history file=/ var /log/hadoop/hive/tmp/root/hive_job_log_root_201305161238_2072032671.txt
              Hide
              Permalink
              vinodkv Vinod Kumar Vavilapalli added a comment -

              Here's a patch to fix this. Difficult to write test. Not useful much too.

              I'll request Thejas to verify this and report back.

              Show
              vinodkv Vinod Kumar Vavilapalli added a comment - Here's a patch to fix this. Difficult to write test. Not useful much too. I'll request Thejas to verify this and report back.
              Hide
              Permalink
              thejas Thejas M Nair added a comment -

              Thanks for the patch, after applying it, we don't see the issue.

              Show
              thejas Thejas M Nair added a comment - Thanks for the patch, after applying it, we don't see the issue.
              Hide
              Permalink
              mattf Matt Foley added a comment - - edited

              +1. Committed to branch-1.2. Seems to be superfluous in branch-1. Thanks, Vinod!

              Show
              mattf Matt Foley added a comment - - edited +1. Committed to branch-1.2. Seems to be superfluous in branch-1. Thanks, Vinod!

                People

                • Assignee:
                  vinodkv Vinod Kumar Vavilapalli
                  Reporter:
                  vinodkv Vinod Kumar Vavilapalli
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development