Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
None
-
None
-
None
-
None
Description
Even though bucket is empty, zero length file is being created in tez mode.
steps to reproduce the issue:
hive> set hive.execution.engine;
hive.execution.engine=tez
hive> drop table if exists emptybucket_orc;
OK
Time taken: 5.416 seconds
hive> create table emptybucket_orc(age int) clustered by (age) sorted by (age) into 99 buckets stored as orc;
OK
Time taken: 0.493 seconds
hive> insert into table emptybucket_orc select distinct(age) from studenttab10k limit 0;
Query ID = hrt_qa_20160523231955_8b981be7-68c4-4416-8a48-5f8c7ff551c3
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1464045121842_0002)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... llap SUCCEEDED 1 1 0 0 0 0
Reducer 2 ...... llap SUCCEEDED 1 1 0 0 0 0
Reducer 3 ...... llap SUCCEEDED 1 1 0 0 0 0
Reducer 4 ...... llap SUCCEEDED 99 99 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 04/04 [==========================>>] 100% ELAPSED TIME: 11.00 s
----------------------------------------------------------------------------------------------
Loading data to table default.emptybucket_orc
OK
Time taken: 16.907 seconds
hive> dfs -ls /apps/hive/warehouse/emptybucket_orc;
Found 99 items
-rwxrwxrwx 3 hrt_qa hdfs 0 2016-05-23 23:20 /apps/hive/warehouse/emptybucket_orc/000000_0
-rwxrwxrwx 3 hrt_qa hdfs 0 2016-05-23 23:20 /apps/hive/warehouse/emptybucket_orc/000001_0
......
Expected behavior:
In tez mode, zero length file shouldn't get created on hdfs if bucket is empty