Hive
  1. Hive
  2. HIVE-1767

Merge files does not work with dynamic partition

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    1. HIVE-1767.1.patch
      68 kB
      He Yongqiang

      Activity

      Transition Time In Source Status Execution Times Last Executer Last Execution Date
      Open Open Patch Available Patch Available
      53m 44s 1 He Yongqiang 04/Nov/10 21:21
      Patch Available Patch Available Open Open
      21m 45s 1 Namit Jain 04/Nov/10 21:43
      Open Open Resolved Resolved
      19h 29m 1 Namit Jain 05/Nov/10 17:12
      Resolved Resolved Closed Closed
      406d 6h 48m 1 Carl Steinbach 17/Dec/11 00:01
      Carl Steinbach made changes -
      Status Resolved [ 5 ] Closed [ 6 ]
      Namit Jain made changes -
      Status Open [ 1 ] Resolved [ 5 ]
      Hadoop Flags [Reviewed]
      Fix Version/s 0.7.0 [ 12315150 ]
      Resolution Fixed [ 1 ]
      Hide
      Namit Jain added a comment -

      Committed. Thanks Yongqiang

      Show
      Namit Jain added a comment - Committed. Thanks Yongqiang
      Hide
      Namit Jain added a comment -

      +1

      running tests

      Show
      Namit Jain added a comment - +1 running tests
      He Yongqiang made changes -
      Attachment HIVE-1767.1.patch [ 12458849 ]
      Namit Jain made changes -
      Status Patch Available [ 10002 ] Open [ 1 ]
      Hide
      Namit Jain added a comment -

      load data local inpath '/Users/heyongqiang/Documents/workspace/Hive-Index/data/files/srcbucket20.txt' INTO TABLE srcpart_merge_dp partition(ds='2008-04-08', hr=11);
      load data local inpath '/Users/heyongqiang/Documents/workspace/Hive-Index/data/files/srcbucket21.txt' INTO TABLE srcpart_merge_dp partition(ds='2008-04-08', hr=11);
      load data local inpath '/Users/heyongqiang/Documents/workspace/Hive-Index/data/files/srcbucket22.txt' INTO TABLE srcpart_merge_dp partition(ds='2008-04-08', hr=11);
      load data local inpath '/Users/heyongqiang/Documents/workspace/Hive-Index/data/files/srcbucket23.txt' INTO TABLE srcpart_merge_dp partition(ds='2008-04-08', hr=11);

      The test needs to be updated

      Show
      Namit Jain added a comment - load data local inpath '/Users/heyongqiang/Documents/workspace/Hive-Index/data/files/srcbucket20.txt' INTO TABLE srcpart_merge_dp partition(ds='2008-04-08', hr=11); load data local inpath '/Users/heyongqiang/Documents/workspace/Hive-Index/data/files/srcbucket21.txt' INTO TABLE srcpart_merge_dp partition(ds='2008-04-08', hr=11); load data local inpath '/Users/heyongqiang/Documents/workspace/Hive-Index/data/files/srcbucket22.txt' INTO TABLE srcpart_merge_dp partition(ds='2008-04-08', hr=11); load data local inpath '/Users/heyongqiang/Documents/workspace/Hive-Index/data/files/srcbucket23.txt' INTO TABLE srcpart_merge_dp partition(ds='2008-04-08', hr=11); The test needs to be updated
      He Yongqiang made changes -
      Attachment HIVE-1767.1.patch [ 12458845 ]
      He Yongqiang made changes -
      Status Open [ 1 ] Patch Available [ 10002 ]
      Hide
      He Yongqiang added a comment -

      When there is a dynamic partition, the merge task never gets launched. I was trying to figure out what's happening there.

      It turns out a bug in ConditionalResolverMergeFiles that leads to the size information counted is not correct for dp queries.

      Attached a patch for it.

      Show
      He Yongqiang added a comment - When there is a dynamic partition, the merge task never gets launched. I was trying to figure out what's happening there. It turns out a bug in ConditionalResolverMergeFiles that leads to the size information counted is not correct for dp queries. Attached a patch for it.
      He Yongqiang made changes -
      Field Original Value New Value
      Attachment HIVE-1767.1.patch [ 12458845 ]
      Hide
      Ning Zhang added a comment -

      Can you elaborate the bug in the description?

      Show
      Ning Zhang added a comment - Can you elaborate the bug in the description?
      He Yongqiang created issue -

        People

        • Assignee:
          He Yongqiang
          Reporter:
          He Yongqiang
        • Votes:
          0 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development