Details

      Description

      Currently group by on struct and union types are not supported. This issue will enable support for those.

      1. HIVE-2517_3.patch
        26 kB
        Ashutosh Chauhan
      2. hive-2517_2.patch
        25 kB
        Ashutosh Chauhan
      3. ASF.LICENSE.NOT.GRANTED--HIVE-2517.D2151.1.patch
        24 kB
        Phabricator
      4. hive-2517_1.patch
        26 kB
        Ashutosh Chauhan
      5. hive-2517.patch
        50 kB
        Ashutosh Chauhan

        Issue Links

          Activity

          Hide
          Ashutosh Chauhan added a comment -

          Patch which enables this support. Test cases included.

          Show
          Ashutosh Chauhan added a comment - Patch which enables this support. Test cases included.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2459/
          -----------------------------------------------------------

          Review request for hive, John Sichi, Yongqiang He, Ning Zhang, and namit jain.

          Summary
          -------

          Support for grouping on union and struct type.

          This addresses bug HIVE-2517.
          https://issues.apache.org/jira/browse/HIVE-2517

          Diffs


          trunk/ql/src/test/queries/clientpositive/groupby_complex_types.q 1186610
          trunk/ql/src/test/results/clientpositive/groupby_complex_types.q.out 1186610
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1186610
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1186610
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUnion.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1186610
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/objectinspector/LazyBinaryObjectInspectorFactory.java 1186610
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/objectinspector/LazyBinaryUnionObjectInspector.java PRE-CREATION
          trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1186610

          Diff: https://reviews.apache.org/r/2459/diff

          Testing
          -------

          Added *.q tests

          Thanks,

          Ashutosh

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2459/ ----------------------------------------------------------- Review request for hive, John Sichi, Yongqiang He, Ning Zhang, and namit jain. Summary ------- Support for grouping on union and struct type. This addresses bug HIVE-2517 . https://issues.apache.org/jira/browse/HIVE-2517 Diffs trunk/ql/src/test/queries/clientpositive/groupby_complex_types.q 1186610 trunk/ql/src/test/results/clientpositive/groupby_complex_types.q.out 1186610 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 1186610 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 1186610 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUnion.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 1186610 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/objectinspector/LazyBinaryObjectInspectorFactory.java 1186610 trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/objectinspector/LazyBinaryUnionObjectInspector.java PRE-CREATION trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 1186610 Diff: https://reviews.apache.org/r/2459/diff Testing ------- Added *.q tests Thanks, Ashutosh
          Hide
          Ashutosh Chauhan added a comment -

          I think it doesn't make sense to do a group by on union data type since what data type it contains varies on row by row basis. This is same as I have pointed on HIVE-2508 So, I have stripped off bits on union type from this patch. I have left hash code still in there because its useful for GenericUDFHash().

          Show
          Ashutosh Chauhan added a comment - I think it doesn't make sense to do a group by on union data type since what data type it contains varies on row by row basis. This is same as I have pointed on HIVE-2508 So, I have stripped off bits on union type from this patch. I have left hash code still in there because its useful for GenericUDFHash().
          Hide
          Ashutosh Chauhan added a comment -

          Any one care to take a look at this one?

          Show
          Ashutosh Chauhan added a comment - Any one care to take a look at this one?
          Hide
          Carl Steinbach added a comment -

          @Ashutosh: Patch needs to be rebased on trunk. Also, can you please submit a review request on phabricator? Thanks.

          Show
          Carl Steinbach added a comment - @Ashutosh: Patch needs to be rebased on trunk. Also, can you please submit a review request on phabricator? Thanks.
          Hide
          Phabricator added a comment -

          ashutoshc requested code review of "HIVE-2517 [jira] Support group by on union and struct type".
          Reviewers: JIRA

          https://issues.apache.org/jira/browse/HIVE-2517

          Rebased to trunk.

          Currently group by on struct and union types are not supported. This issue will enable support for those.

          TEST PLAN
          EMPTY

          REVISION DETAIL
          https://reviews.facebook.net/D2151

          AFFECTED FILES
          serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
          ql/src/test/results/clientpositive/groupby_complex_types.q.out
          ql/src/test/queries/clientpositive/groupby_complex_types.q

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/4713/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - ashutoshc requested code review of " HIVE-2517 [jira] Support group by on union and struct type". Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2517 Rebased to trunk. Currently group by on struct and union types are not supported. This issue will enable support for those. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2151 AFFECTED FILES serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java ql/src/test/results/clientpositive/groupby_complex_types.q.out ql/src/test/queries/clientpositive/groupby_complex_types.q MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/4713/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
          Hide
          Ashutosh Chauhan added a comment -

          Patch with license.

          Show
          Ashutosh Chauhan added a comment - Patch with license.
          Hide
          Ashutosh Chauhan added a comment -

          Unlinking from 0.9

          Show
          Ashutosh Chauhan added a comment - Unlinking from 0.9
          Hide
          Philip Tromans added a comment -

          Hi,

          Can we bump this please? I just came up against this issue, wrote the patch, was about to submit it, only to find it already here...

          Cheers.

          Show
          Philip Tromans added a comment - Hi, Can we bump this please? I just came up against this issue, wrote the patch, was about to submit it, only to find it already here... Cheers.
          Hide
          Ashutosh Chauhan added a comment -

          @Philip : Certainly. Since I wrote the patch I cant check this in. If some other committer volunteers to help, then we can get this in.

          Show
          Ashutosh Chauhan added a comment - @Philip : Certainly. Since I wrote the patch I cant check this in. If some other committer volunteers to help, then we can get this in.
          Hide
          Navis added a comment -

          Ashutosh Chauhan Could you update this? Let's get it in.

          Show
          Navis added a comment - Ashutosh Chauhan Could you update this? Let's get it in.
          Hide
          Ashutosh Chauhan added a comment -

          Sure. I will update this soon.

          Show
          Ashutosh Chauhan added a comment - Sure. I will update this soon.
          Hide
          Ashutosh Chauhan added a comment -

          Sorry took a while. Patch rebased on latest trunk. Navis Can you take a look?

          Show
          Ashutosh Chauhan added a comment - Sorry took a while. Patch rebased on latest trunk. Navis Can you take a look?
          Hide
          Ashutosh Chauhan added a comment -

          All tests passed.

           Tests Failures Errors Success rate   Time
           2642  0        0	 100.00%      33330.960
          
          Show
          Ashutosh Chauhan added a comment - All tests passed. Tests Failures Errors Success rate Time 2642 0 0 100.00% 33330.960
          Hide
          Navis added a comment -

          Committed to trunk. Thanks, Ashutosh!

          Show
          Navis added a comment - Committed to trunk. Thanks, Ashutosh!
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #2174 (See https://builds.apache.org/job/Hive-trunk-h0.21/2174/)
          HIVE-2517 : Support group by on struct type (Ashutosh Chauhan via Navis) (Revision 1498755)

          Result = FAILURE
          navis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1498755
          Files :

          • /hive/trunk/ql/src/test/queries/clientpositive/groupby_complex_types.q
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_complex_types.q.out
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #2174 (See https://builds.apache.org/job/Hive-trunk-h0.21/2174/ ) HIVE-2517 : Support group by on struct type (Ashutosh Chauhan via Navis) (Revision 1498755) Result = FAILURE navis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1498755 Files : /hive/trunk/ql/src/test/queries/clientpositive/groupby_complex_types.q /hive/trunk/ql/src/test/results/clientpositive/groupby_complex_types.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #268 (See https://builds.apache.org/job/Hive-trunk-hadoop2/268/)
          HIVE-2517 : Support group by on struct type (Ashutosh Chauhan via Navis) (Revision 1498755)

          Result = FAILURE
          navis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1498755
          Files :

          • /hive/trunk/ql/src/test/queries/clientpositive/groupby_complex_types.q
          • /hive/trunk/ql/src/test/results/clientpositive/groupby_complex_types.q.out
          • /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #268 (See https://builds.apache.org/job/Hive-trunk-hadoop2/268/ ) HIVE-2517 : Support group by on struct type (Ashutosh Chauhan via Navis) (Revision 1498755) Result = FAILURE navis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1498755 Files : /hive/trunk/ql/src/test/queries/clientpositive/groupby_complex_types.q /hive/trunk/ql/src/test/results/clientpositive/groupby_complex_types.q.out /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
          Hide
          Ashutosh Chauhan added a comment -

          This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          Show
          Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

            People

            • Assignee:
              Ashutosh Chauhan
              Reporter:
              Ashutosh Chauhan
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development