Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18835

Do not expose shaded types in JavaTypeInference API

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.1.0
    • 2.1.0
    • SQL
    • None

    Description

      Currently, inferDataType(TypeToken) is called from a different maven module, and because we shade Guava, that sometimes leads to errors (e.g. when running tests using maven):

      udf3Test(test.org.apache.spark.sql.JavaUDFSuite)  Time elapsed: 0.084 sec  <<< ERROR!
      java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.JavaTypeInference$.inferDataType(Lcom/google/common/reflect/TypeToken;)Lscala/Tuple2;
              at test.org.apache.spark.sql.JavaUDFSuite.udf3Test(JavaUDFSuite.java:107)
      
      
      Results :
      
      Tests in error: 
        JavaUDFSuite.udf3Test:107 ยป NoSuchMethod org.apache.spark.sql.catalyst.JavaTyp...
      

      Instead, we shouldn't expose Guava types in these APIs.

      Attachments

        Activity

          People

            vanzin Marcelo Masiero Vanzin
            vanzin Marcelo Masiero Vanzin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: