Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-2125

alter table concatenate fails and deletes data

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 0.8.0
    • Query Processor
    • None
    • Reviewed

    Description

      the number of reducers is not set by this command (unlike other hive queries). since mapred.reduce.tasks=-1 (to let hive infer this automatically) - jobtracker fails the job (number of reducers cannot be negative)

      hive> alter table ad_imps_2 partition(ds='2009-06-16') concatenate;
      alter table ad_imps_2 partition(ds='2009-06-16') concatenate;
      Starting Job = job_201103101203_453180, Tracking URL = http://curium.data.facebook.com:50030/jobdetails.jsp?jobid=job_201103101203_453180
      Kill Command = /mnt/vol/hive/sites/curium/hadoop/bin/../bin/hadoop job -Dmapred.job.tracker=curium.data.facebook.com:50029 -kill job_201103101203_453180
      Hadoop job information for null: number of mappers: 0; number of reducers: 0
      2011-04-22 10:21:24,046 null map = 100%, reduce = 100%
      Ended Job = job_201103101203_453180 with errors
      Moved to trash: /user/facebook/warehouse/ad_imps_2/_backup.ds=2009-06-16
      after the job fails - the partition is deleted

      thankfully it's still in trash

      Attachments

        1. HIVE-2125.2.patch
          32 kB
          He Yongqiang
        2. HIVE-2125.1.patch
          31 kB
          He Yongqiang

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            he yongqiang He Yongqiang Assign to me
            jsensarma Joydeep Sen Sarma
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment