Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-32624

Replace getClass.getName with getClass.getCanonicalName in CodegenContext.addReferenceObj

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.1, 3.1.0
    • SQL
    • None

    Description

      scala> Array[Byte](1, 2).getClass.getName
      res13: String = [B
      
      scala> Array[Byte](1, 2).getClass.getCanonicalName
      res14: String = byte[]
      

      [B is not a correct java type. We should use byte[]. Otherwise will hit compile issue:

      20:49:54.885 ERROR org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator: 
      ...
      /* 029 */     if (!isNull_2) {
      /* 030 */       value_1 = org.apache.spark.sql.catalyst.util.TypeUtils.compareBinary(value_2, (([B) references[0] /* min */)) >= 0 && org.apache.spark.sql.catalyst.util.TypeUtils.compareBinary(value_2, (([B) references[1] /* max */)) <= 0 ).mightContainBinary(value_2);
      ...
      
      20:49:54.886 WARN org.apache.spark.sql.catalyst.expressions.Predicate: Expr codegen error and falling back to interpreter mode
      java.util.concurrent.ExecutionException: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 30, Column 81: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 30, Column 81: Unexpected token "[" in primary
      	at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:306)
      

      Attachments

        Activity

          People

            yumwang Yuming Wang
            yumwang Yuming Wang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: