[HIVE-1277] Select query with specific projection(s) fails if the local file system directory for ${hive.user.scratchdir} does not exist.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.6.0
• Component/s: Query Processor

Environment

Mac OSX 10.6.2, Java SE 6 (1.6.0_17)

• Hadoop Flags: Reviewed
• Assignee: Arvind Prabhakar
• Reporter: Arvind Prabhakar
• Created: 24/Mar/10 20:17
• Updated: 17/Dec/11 00:03
• Resolved: 26/Mar/10 05:10

Description

When a simple select query with specified projections is processed, the MapRedTask attempts to serialize the execution plan to a scratch directory on the local file system specified by hive.user.scratchdir configuration property. The default value of this property points to /tmp/hive-${user.name}. On some *nix systems, the /tmp directory gets cleaned up on every reboot thus causing the query execution to fail.

Exception stacktrace

hive> select foo from dummy;
Total MapReduce jobs = 1
Launching Job 1 out of 1
java.io.IOException: No such file or directory
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.checkAndCreate(File.java:1704)
at java.io.File.createTempFile(File.java:1792)
at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:89)
...