Hive
  1. Hive
  2. HIVE-2755

union follwowed by union_subq does not work if the subquery union has reducers

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Activity

      Hide
      Hudson added a comment -

      Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
      HIVE-2755 union follwowed by union_subq does not work if the subquery union
      has reducers (He Yongqiang via namit) (Revision 1237253)

      Result = ABORTED
      namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237253
      Files :

      • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java
      • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRUnion1.java
      • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcContext.java
      • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java
      • /hive/trunk/ql/src/test/queries/clientpositive/union28.q
      • /hive/trunk/ql/src/test/queries/clientpositive/union29.q
      • /hive/trunk/ql/src/test/queries/clientpositive/union30.q
      • /hive/trunk/ql/src/test/results/clientpositive/union28.q.out
      • /hive/trunk/ql/src/test/results/clientpositive/union29.q.out
      • /hive/trunk/ql/src/test/results/clientpositive/union30.q.out
      Show
      Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-2755 union follwowed by union_subq does not work if the subquery union has reducers (He Yongqiang via namit) (Revision 1237253) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237253 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRUnion1.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcContext.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java /hive/trunk/ql/src/test/queries/clientpositive/union28.q /hive/trunk/ql/src/test/queries/clientpositive/union29.q /hive/trunk/ql/src/test/queries/clientpositive/union30.q /hive/trunk/ql/src/test/results/clientpositive/union28.q.out /hive/trunk/ql/src/test/results/clientpositive/union29.q.out /hive/trunk/ql/src/test/results/clientpositive/union30.q.out
      Hide
      Ashutosh Chauhan added a comment -

      This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.

      Show
      Ashutosh Chauhan added a comment - This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.
      Hide
      Hudson added a comment -

      Integrated in Hive-trunk-h0.21 #1227 (See https://builds.apache.org/job/Hive-trunk-h0.21/1227/)
      HIVE-2755 union follwowed by union_subq does not work if the subquery union
      has reducers (He Yongqiang via namit)

      namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237253
      Files :

      • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java
      • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRUnion1.java
      • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcContext.java
      • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java
      • /hive/trunk/ql/src/test/queries/clientpositive/union28.q
      • /hive/trunk/ql/src/test/queries/clientpositive/union29.q
      • /hive/trunk/ql/src/test/queries/clientpositive/union30.q
      • /hive/trunk/ql/src/test/results/clientpositive/union28.q.out
      • /hive/trunk/ql/src/test/results/clientpositive/union29.q.out
      • /hive/trunk/ql/src/test/results/clientpositive/union30.q.out
      Show
      Hudson added a comment - Integrated in Hive-trunk-h0.21 #1227 (See https://builds.apache.org/job/Hive-trunk-h0.21/1227/ ) HIVE-2755 union follwowed by union_subq does not work if the subquery union has reducers (He Yongqiang via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237253 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRUnion1.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcContext.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java /hive/trunk/ql/src/test/queries/clientpositive/union28.q /hive/trunk/ql/src/test/queries/clientpositive/union29.q /hive/trunk/ql/src/test/queries/clientpositive/union30.q /hive/trunk/ql/src/test/results/clientpositive/union28.q.out /hive/trunk/ql/src/test/results/clientpositive/union29.q.out /hive/trunk/ql/src/test/results/clientpositive/union30.q.out
      Hide
      Namit Jain added a comment -

      Committed. Thanks Yongqiang

      Show
      Namit Jain added a comment - Committed. Thanks Yongqiang
      Hide
      Phabricator added a comment -

      heyongqiang updated the revision "HIVE-2755 [jira] union follwowed by union_subq does not work if the subquery union has reducers".
      Reviewers: JIRA

      fix one failing test create_view.q

      REVISION DETAIL
      https://reviews.facebook.net/D1479

      AFFECTED FILES
      ql/src/test/results/clientpositive/union28.q.out
      ql/src/test/results/clientpositive/union29.q.out
      ql/src/test/results/clientpositive/union30.q.out
      ql/src/test/queries/clientpositive/union30.q
      ql/src/test/queries/clientpositive/union28.q
      ql/src/test/queries/clientpositive/union29.q
      ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRUnion1.java
      ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java
      ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcContext.java
      ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java

      Show
      Phabricator added a comment - heyongqiang updated the revision " HIVE-2755 [jira] union follwowed by union_subq does not work if the subquery union has reducers". Reviewers: JIRA fix one failing test create_view.q REVISION DETAIL https://reviews.facebook.net/D1479 AFFECTED FILES ql/src/test/results/clientpositive/union28.q.out ql/src/test/results/clientpositive/union29.q.out ql/src/test/results/clientpositive/union30.q.out ql/src/test/queries/clientpositive/union30.q ql/src/test/queries/clientpositive/union28.q ql/src/test/queries/clientpositive/union29.q ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRUnion1.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcContext.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java
      Hide
      Phabricator added a comment -

      heyongqiang updated the revision "HIVE-2755 [jira] union follwowed by union_subq does not work if the subquery union has reducers".
      Reviewers: JIRA

      address namit's comments

      REVISION DETAIL
      https://reviews.facebook.net/D1479

      AFFECTED FILES
      ql/src/test/results/clientpositive/union28.q.out
      ql/src/test/results/clientpositive/union29.q.out
      ql/src/test/results/clientpositive/union30.q.out
      ql/src/test/queries/clientpositive/union30.q
      ql/src/test/queries/clientpositive/union28.q
      ql/src/test/queries/clientpositive/union29.q
      ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRUnion1.java
      ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java
      ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcContext.java
      ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java

      Show
      Phabricator added a comment - heyongqiang updated the revision " HIVE-2755 [jira] union follwowed by union_subq does not work if the subquery union has reducers". Reviewers: JIRA address namit's comments REVISION DETAIL https://reviews.facebook.net/D1479 AFFECTED FILES ql/src/test/results/clientpositive/union28.q.out ql/src/test/results/clientpositive/union29.q.out ql/src/test/results/clientpositive/union30.q.out ql/src/test/queries/clientpositive/union30.q ql/src/test/queries/clientpositive/union28.q ql/src/test/queries/clientpositive/union29.q ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRUnion1.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcContext.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java
      Hide
      Phabricator added a comment -

      njain has commented on the revision "HIVE-2755 [jira] union follwowed by union_subq does not work if the subquery union has reducers".

      Also can you add a new test to verify this ?

      select ... T
      union all
      (select .. T1
      union all
      select .. T2)s;

      Basically, a map-only union followed by another union

      REVISION DETAIL
      https://reviews.facebook.net/D1479

      Show
      Phabricator added a comment - njain has commented on the revision " HIVE-2755 [jira] union follwowed by union_subq does not work if the subquery union has reducers". Also can you add a new test to verify this ? select ... T union all (select .. T1 union all select .. T2)s; Basically, a map-only union followed by another union REVISION DETAIL https://reviews.facebook.net/D1479
      Hide
      Phabricator added a comment -

      njain has commented on the revision "HIVE-2755 [jira] union follwowed by union_subq does not work if the subquery union has reducers".

      INLINE COMMENTS
      ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java:149 There seems to be a bug here.
      Shouldn't you be looking at stack.get(start)

      REVISION DETAIL
      https://reviews.facebook.net/D1479

      Show
      Phabricator added a comment - njain has commented on the revision " HIVE-2755 [jira] union follwowed by union_subq does not work if the subquery union has reducers". INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java:149 There seems to be a bug here. Shouldn't you be looking at stack.get(start) REVISION DETAIL https://reviews.facebook.net/D1479
      Hide
      Phabricator added a comment -

      heyongqiang requested code review of "HIVE-2755 [jira] union follwowed by union_subq does not work if the subquery union has reducers".
      Reviewers: JIRA

      https://issues.apache.org/jira/browse/HIVE-2755

      TEST PLAN
      EMPTY

      REVISION DETAIL
      https://reviews.facebook.net/D1479

      AFFECTED FILES
      ql/src/test/results/clientpositive/union28.q.out
      ql/src/test/queries/clientpositive/union28.q
      ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java

      MANAGE HERALD DIFFERENTIAL RULES
      https://reviews.facebook.net/herald/view/differential/

      WHY DID I GET THIS EMAIL?
      https://reviews.facebook.net/herald/transcript/3069/

      Tip: use the X-Herald-Rules header to filter Herald messages in your client.

      Show
      Phabricator added a comment - heyongqiang requested code review of " HIVE-2755 [jira] union follwowed by union_subq does not work if the subquery union has reducers". Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2755 TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1479 AFFECTED FILES ql/src/test/results/clientpositive/union28.q.out ql/src/test/queries/clientpositive/union28.q ql/src/java/org/apache/hadoop/hive/ql/optimizer/unionproc/UnionProcFactory.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/3069/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
      Show
      He Yongqiang added a comment - https://reviews.facebook.net/D1479

        People

        • Assignee:
          He Yongqiang
          Reporter:
          He Yongqiang
        • Votes:
          0 Vote for this issue
          Watchers:
          1 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development