Hive
  1. Hive
  2. HIVE-3125

sort_array doesn't work with LazyPrimitive

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.10.0
    • Component/s: UDF
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The sort_array function doesn't work against data that's actually come out of a table. The test suite only covers constants given in the query.

      If you try and use sort_array on an array from a table, then you get a ClassCastException that you can't convert LazyX to Comparable.

      1. HIVE-3125.2.patch.txt
        7 kB
        Philip Tromans
      2. HIVE-3125.1.patch.txt
        5 kB
        Philip Tromans

        Activity

        Hide
        Philip Tromans added a comment -

        I'm not entirely sure if this is the right way to fix this. The right thing to do may well be to make LazyInteger Comparable, but I thought it'd be best for me to stay out of Hive's serde stuff. I'm still a little new round here...

        This fixes it by using the ObjectInspectorUtils compare() method.

        Show
        Philip Tromans added a comment - I'm not entirely sure if this is the right way to fix this. The right thing to do may well be to make LazyInteger Comparable, but I thought it'd be best for me to stay out of Hive's serde stuff. I'm still a little new round here... This fixes it by using the ObjectInspectorUtils compare() method.
        Hide
        Edward Capriolo added a comment -

        Nice catch. A feature that sorts out literals only is not useful outside of the unit tests. I will look at this.

        Show
        Edward Capriolo added a comment - Nice catch. A feature that sorts out literals only is not useful outside of the unit tests. I will look at this.
        Hide
        Carl Steinbach added a comment -

        @Philip: Good catch. Can you please submit a review request for this patch on reviews.apache.org? Thanks.

        Show
        Carl Steinbach added a comment - @Philip: Good catch. Can you please submit a review request for this patch on reviews.apache.org? Thanks.
        Hide
        Philip Tromans added a comment -

        @Carl: reviews.apache.org appears to be down at the moment. I'll keep checking and submit a request if it comes back up.

        Show
        Philip Tromans added a comment - @Carl: reviews.apache.org appears to be down at the moment. I'll keep checking and submit a request if it comes back up.
        Hide
        Philip Tromans added a comment -

        @Carl, @Edward: Review board seems to be back now. I've submitted a review request: https://reviews.apache.org/r/5384/

        Show
        Philip Tromans added a comment - @Carl, @Edward: Review board seems to be back now. I've submitted a review request: https://reviews.apache.org/r/5384/
        Hide
        Carl Steinbach added a comment -

        @Philip: Patch looks good overall. I added a couple small requests to reviewboard. Thanks.

        Show
        Carl Steinbach added a comment - @Philip: Patch looks good overall. I added a couple small requests to reviewboard. Thanks.
        Hide
        Philip Tromans added a comment -

        @Carl, @Edward: I've updated the patch on reviewboard.

        Show
        Philip Tromans added a comment - @Carl, @Edward: I've updated the patch on reviewboard.
        Hide
        Edward Capriolo added a comment -

        +1 will commit if tests pass

        Show
        Edward Capriolo added a comment - +1 will commit if tests pass
        Hide
        Edward Capriolo added a comment -

        Committed. Thanks Philip.

        Show
        Edward Capriolo added a comment - Committed. Thanks Philip.
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
        HIVE-3125 sort_array does not work with LazyPrimitive Philip Tromans (via egc) (Revision 1353203)

        Result = ABORTED
        ecapriolo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1353203
        Files :

        • /hive/trunk/data/files/primitive_type_arrays.txt
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArray.java
        • /hive/trunk/ql/src/test/queries/clientpositive/udf_sort_array.q
        • /hive/trunk/ql/src/test/results/clientpositive/udf_sort_array.q.out
        Show
        Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-3125 sort_array does not work with LazyPrimitive Philip Tromans (via egc) (Revision 1353203) Result = ABORTED ecapriolo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1353203 Files : /hive/trunk/data/files/primitive_type_arrays.txt /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSortArray.java /hive/trunk/ql/src/test/queries/clientpositive/udf_sort_array.q /hive/trunk/ql/src/test/results/clientpositive/udf_sort_array.q.out
        Hide
        Ashutosh Chauhan added a comment -

        This issue is fixed and released as part of 0.10.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

        Show
        Ashutosh Chauhan added a comment - This issue is fixed and released as part of 0.10.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

          People

          • Assignee:
            Philip Tromans
            Reporter:
            Philip Tromans
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development