Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9249

java.lang.ClassCastException: org.apache.hadoop.hive.serde2.io.HiveVarcharWritable cannot be cast to org.apache.hadoop.hive.common.type.HiveVarchar when joining tables

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.14.0
    • Fix Version/s: 1.0.0
    • Component/s: Vectorization
    • Labels:
      None
    • Release Note:
      Hide
      HIVE-9249: Vectorization: Join involving CHAR/VARCHAR fails during execution. (Matt McCline, reviewed by Gopal V)
      Show
      HIVE-9249 : Vectorization: Join involving CHAR/VARCHAR fails during execution. (Matt McCline, reviewed by Gopal V)

      Description

      VectorColumnAssignFactory doesn't handle HiveCharWritable / HiveVarcharWritable objects.

      Either:
      HiveVarcharWritable cannot be cast to ... HiveVarchar
      or
      HiveCharWritable cannot be cast to ... HiveChar

      Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.io.HiveVarcharWritable cannot be cast to org.apache.hadoop.hive.common.type.HiveVarchar
      	at org.apache.hadoop.hive.ql.exec.vector.VectorColumnAssignFactory$17.assignObjectValue(VectorColumnAssignFactory.java:417)
      	at org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.internalForward(VectorMapJoinOperator.java:196)
      	at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genAllOneUniqueJoinObject(CommonJoinOperator.java:670)
      	at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:748)
      	at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:299)
      	... 24 more
      
      1. HIVE-9249.01.patch
        87 kB
        Matt McCline
      2. HIVE-9249.hive-14.patch
        88 kB
        Gopal V

        Issue Links

          Activity

          Hide
          gopalv Gopal V added a comment -

          LGTM - +1, tests pending.

          Show
          gopalv Gopal V added a comment - LGTM - +1, tests pending.
          Hide
          hiveqa Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12689968/HIVE-9249.01.patch

          ERROR: -1 due to 1 failed/errored test(s), 6727 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2246/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2246/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2246/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12689968 - PreCommit-HIVE-TRUNK-Build

          Show
          hiveqa Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12689968/HIVE-9249.01.patch ERROR: -1 due to 1 failed/errored test(s), 6727 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2246/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2246/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2246/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12689968 - PreCommit-HIVE-TRUNK-Build
          Hide
          mmccline Matt McCline added a comment -

          Test failure is unrelated.

          Show
          mmccline Matt McCline added a comment - Test failure is unrelated.
          Hide
          mmccline Matt McCline added a comment -

          Vikram Dixit K can you approve for 0.14.1 – Thank You.

          Show
          mmccline Matt McCline added a comment - Vikram Dixit K can you approve for 0.14.1 – Thank You.
          Hide
          vikram.dixit Vikram Dixit K added a comment -

          +1 for 0.14

          Show
          vikram.dixit Vikram Dixit K added a comment - +1 for 0.14
          Hide
          mmccline Matt McCline added a comment -

          Refresh patch.

          Show
          mmccline Matt McCline added a comment - Refresh patch.
          Hide
          mmccline Matt McCline added a comment -

          No changes, ready to go in.

          Show
          mmccline Matt McCline added a comment - No changes, ready to go in.
          Hide
          gopalv Gopal V added a comment -

          Matt McCline: Added to my builds for the day.

          Will commit to 0.15 really soon. Will commit to branch, once the newly added tests are run once.

          Show
          gopalv Gopal V added a comment - Matt McCline : Added to my builds for the day. Will commit to 0.15 really soon. Will commit to branch, once the newly added tests are run once.
          Hide
          gopalv Gopal V added a comment -

          Committed to 0.15.0.

          Not committed to 0.14.1 because of plan formatting changes in the golden files.

          clientpositive/vector_varchar_mapjoin1.q.out
          148,150d147
          <                 condition expressions:
          <                   0 {c1}
          <                   1 {c1} {c2}
          167,169d163
          <                 condition expressions:
          <                   0 {c1} {c2}
          <                   1 {c1} {c2}
          244,246d237
          <                 condition expressions:
          <                   0 {c1}
          <                   1 {c1} {c2}
          263,265d253
          <                 condition expressions:
          <                   0 {c1} {c2}
          <                   1 {c1} {c2}
          342,344d329
          <                 condition expressions:
          <                   0 {c1} {c2}
          <                   1 {c1}
          361,363d345
          <                 condition expressions:
          <                   0 {c1} {c2}
          <                   1 {c1} {c2}
          

          Will generate a new patch, in a few mins.

          Show
          gopalv Gopal V added a comment - Committed to 0.15.0. Not committed to 0.14.1 because of plan formatting changes in the golden files. clientpositive/vector_varchar_mapjoin1.q.out 148,150d147 < condition expressions: < 0 {c1} < 1 {c1} {c2} 167,169d163 < condition expressions: < 0 {c1} {c2} < 1 {c1} {c2} 244,246d237 < condition expressions: < 0 {c1} < 1 {c1} {c2} 263,265d253 < condition expressions: < 0 {c1} {c2} < 1 {c1} {c2} 342,344d329 < condition expressions: < 0 {c1} {c2} < 1 {c1} 361,363d345 < condition expressions: < 0 {c1} {c2} < 1 {c1} {c2} Will generate a new patch, in a few mins.
          Hide
          gopalv Gopal V added a comment -

          Matt McCline: I have regenerated the golden files for the branch. Can you verify the mapjoin.out files with a look-see?

          Show
          gopalv Gopal V added a comment - Matt McCline : I have regenerated the golden files for the branch. Can you verify the mapjoin.out files with a look-see?
          Hide
          mmccline Matt McCline added a comment -

          Yes, Gopal V the hive-14 patch looks good. Thanks.

          Show
          mmccline Matt McCline added a comment - Yes, Gopal V the hive-14 patch looks good. Thanks.
          Hide
          gopalv Gopal V added a comment -

          Committed to branch and trunk, thanks Matt McCline.

          Show
          gopalv Gopal V added a comment - Committed to branch and trunk, thanks Matt McCline .
          Hide
          gopalv Gopal V added a comment -

          All tests on 0.14 branch branch passes - https://builds.apache.org/job/Hive-0.14/818/.

          Show
          gopalv Gopal V added a comment - All tests on 0.14 branch branch passes - https://builds.apache.org/job/Hive-0.14/818/ .
          Hide
          thejas Thejas M Nair added a comment -

          Updating release version for jiras resolved in 1.0.0 .

          Show
          thejas Thejas M Nair added a comment - Updating release version for jiras resolved in 1.0.0 .
          Hide
          thejas Thejas M Nair added a comment -

          This issue has been fixed in Apache Hive 1.0.0. If there is any issue with the fix, please open a new jira to address it.

          Show
          thejas Thejas M Nair added a comment - This issue has been fixed in Apache Hive 1.0.0. If there is any issue with the fix, please open a new jira to address it.

            People

            • Assignee:
              mmccline Matt McCline
              Reporter:
              mmccline Matt McCline
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development