[HIVE-17620] Use the default MR scratch directory (HDFS) in the only case when hive.blobstore.optimizations.enabled=true AND isFinalJob=true

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0, 2.3.0, 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Hive
• Assignee: Gergely Hajós
• Reporter: Gergely Hajós
• Created: 27/Sep/17 16:55
• Updated: 22/May/18 23:13
• Resolved: 04/Dec/17 08:47

Description

Introduced in HIVE-15121. Context::getTempDirForPath tries to use temporary MR directory instead of blobstore directory in three cases:

if (!isFinalJob && BlobStorageUtils.areOptimizationsEnabled(conf)) {

while the only valid case for using a temporary MR dir is when optimization is enabled and the job is not final:

if (BlobStorageUtils.areOptimizationsEnabled(conf) && !isFinalJob) {