Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-32624

Replace getClass.getName with getClass.getCanonicalName in CodegenContext.addReferenceObj

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.1, 3.1.0
    • Component/s: SQL
    • Labels:
      None

      Description

      scala> Array[Byte](1, 2).getClass.getName
      res13: String = [B
      
      scala> Array[Byte](1, 2).getClass.getCanonicalName
      res14: String = byte[]
      

      [B is not a correct java type. We should use byte[]. Otherwise will hit compile issue:

      20:49:54.885 ERROR org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator: 
      ...
      /* 029 */     if (!isNull_2) {
      /* 030 */       value_1 = org.apache.spark.sql.catalyst.util.TypeUtils.compareBinary(value_2, (([B) references[0] /* min */)) >= 0 && org.apache.spark.sql.catalyst.util.TypeUtils.compareBinary(value_2, (([B) references[1] /* max */)) <= 0 ).mightContainBinary(value_2);
      ...
      
      20:49:54.886 WARN org.apache.spark.sql.catalyst.expressions.Predicate: Expr codegen error and falling back to interpreter mode
      java.util.concurrent.ExecutionException: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 30, Column 81: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 30, Column 81: Unexpected token "[" in primary
      	at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:306)
      

        Attachments

          Activity

            People

            • Assignee:
              yumwang Yuming Wang
              Reporter:
              yumwang Yuming Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: