[HIVE-10677] hive.exec.parallel=true has problem when it is used for analyze table column stats

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.12.0, 0.13.0, 0.14.0, 1.0.0, 1.1.0, 1.2.0
• Fix Version/s: 1.2.1
• Assignee: Pengcheng Xiong Assign to me
• Reporter: Pengcheng Xiong
• Created: 12/May/15 00:13
• Updated: 22/Jun/15 07:39
• Resolved: 23/May/15 19:36

Description

To reproduce it, in q tests.

hive> set hive.exec.parallel;
hive.exec.parallel=true
hive> analyze table src compute statistics for columns;
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.ColumnStatsTask
java.lang.RuntimeException: Error caching map.xml: java.io.IOException: java.lang.InterruptedException
	at org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:747)
	at org.apache.hadoop.hive.ql.exec.Utilities.setMapWork(Utilities.java:682)
	at org.apache.hadoop.hive.ql.exec.Utilities.setMapRedWork(Utilities.java:674)
	at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:375)
	at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:75)
Caused by: java.io.IOException: java.lang.InterruptedException
	at org.apache.hadoop.util.Shell.runCommand(Shell.java:541)
	at org.apache.hadoop.util.Shell.run(Shell.java:455)
	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
	at org.apache.hadoop.util.Shell.execCommand(Shell.java:791)
	at org.apache.hadoop.util.Shell.execCommand(Shell.java:774)
	at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:646)
	at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:472)
	at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:460)
	at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:426)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:906)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:887)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:784)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:773)
	at org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:715)
	... 7 more
hive> Job Submission failed with exception 'java.lang.RuntimeException(Error caching map.xml: java.io.IOException: java.lang.InterruptedException)'

Issue Links

• is duplicated by: HIVE-11069 ColumnStatsTask doesn't work with hive.exec.parallel
• is related to: HIVE-10404 hive.exec.parallel=true causes "out of sequence response" and SocketTimeoutException: Read timed out