[HIVE-2125] alter table concatenate fails and deletes data

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.8.0
• Component/s: Query Processor
• Hadoop Flags: Reviewed
• Assignee: He Yongqiang
• Reporter: Joydeep Sen Sarma
• Created: 22/Apr/11 17:36
• Updated: 16/Dec/11 23:56
• Resolved: 28/Apr/11 17:53

Description

the number of reducers is not set by this command (unlike other hive queries). since mapred.reduce.tasks=-1 (to let hive infer this automatically) - jobtracker fails the job (number of reducers cannot be negative)

hive> alter table ad_imps_2 partition(ds='2009-06-16') concatenate;
alter table ad_imps_2 partition(ds='2009-06-16') concatenate;
Starting Job = job_201103101203_453180, Tracking URL = http://curium.data.facebook.com:50030/jobdetails.jsp?jobid=job_201103101203_453180
Kill Command = /mnt/vol/hive/sites/curium/hadoop/bin/../bin/hadoop job -Dmapred.job.tracker=curium.data.facebook.com:50029 -kill job_201103101203_453180
Hadoop job information for null: number of mappers: 0; number of reducers: 0
2011-04-22 10:21:24,046 null map = 100%, reduce = 100%
Ended Job = job_201103101203_453180 with errors
Moved to trash: /user/facebook/warehouse/ad_imps_2/_backup.ds=2009-06-16
after the job fails - the partition is deleted

thankfully it's still in trash