Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-10677

hive.exec.parallel=true has problem when it is used for analyze table column stats

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.1.0, 1.2.0
    • 1.2.1
    • None
    • None

    Description

      To reproduce it, in q tests.

      hive> set hive.exec.parallel;
      hive.exec.parallel=true
      hive> analyze table src compute statistics for columns;
      FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.ColumnStatsTask
      java.lang.RuntimeException: Error caching map.xml: java.io.IOException: java.lang.InterruptedException
      	at org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:747)
      	at org.apache.hadoop.hive.ql.exec.Utilities.setMapWork(Utilities.java:682)
      	at org.apache.hadoop.hive.ql.exec.Utilities.setMapRedWork(Utilities.java:674)
      	at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:375)
      	at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
      	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
      	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
      	at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:75)
      Caused by: java.io.IOException: java.lang.InterruptedException
      	at org.apache.hadoop.util.Shell.runCommand(Shell.java:541)
      	at org.apache.hadoop.util.Shell.run(Shell.java:455)
      	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
      	at org.apache.hadoop.util.Shell.execCommand(Shell.java:791)
      	at org.apache.hadoop.util.Shell.execCommand(Shell.java:774)
      	at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:646)
      	at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:472)
      	at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:460)
      	at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:426)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:906)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:887)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:784)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:773)
      	at org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:715)
      	... 7 more
      hive> Job Submission failed with exception 'java.lang.RuntimeException(Error caching map.xml: java.io.IOException: java.lang.InterruptedException)'
      

      Attachments

        1. HIVE-10677.01.patch
          4 kB
          Pengcheng Xiong
        2. HIVE-10677.02.patch
          11 kB
          Pengcheng Xiong

        Issue Links

          Activity

            People

              pxiong Pengcheng Xiong
              pxiong Pengcheng Xiong
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: