Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20926

Exposure to Guava libraries by directly accessing tableRelationCache in SessionCatalog caused failures

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.2.0, 2.3.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      Because of shading that we did for guava libraries, we see test failures whenever those components directly access tableRelationCache in SessionCatalog.
      This can happen in any component that shaded guava library. Failures looks like this:

      java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.catalog.SessionCatalog.tableRelationCache()Lcom/google/common/cache/Cache;
      01:25:14   at org.apache.spark.sql.hive.test.TestHiveSparkSession.reset(TestHive.scala:492)
      01:25:14   at org.apache.spark.sql.hive.test.TestHiveContext.reset(TestHive.scala:138)
      01:25:14   at org.apache.spark.sql.hive.test.TestHiveSingleton$class.afterAll(TestHiveSingleton.scala:32)
      01:25:14   at org.apache.spark.sql.hive.StatisticsSuite.afterAll(StatisticsSuite.scala:34)
      01:25:14   at org.scalatest.BeforeAndAfterAll$class.afterAll(BeforeAndAfterAll.scala:213)
      01:25:14   at org.apache.spark.SparkFunSuite.afterAll(SparkFunSuite.scala:31)
      01:25:14   at org.scalatest.BeforeAndAfterAll$$anonfun$run$1.apply(BeforeAndAfterAll.scala:280)
      01:25:14   at org.scalatest.BeforeAndAfterAll$$anonfun$run$1.apply(BeforeAndAfterAll.scala:278)
      01:25:14   at org.scalatest.CompositeStatus.whenCompleted(Status.scala:377)
      01:25:14   at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:278)
      

        Attachments

          Activity

            People

            • Assignee:
              rezasafi Reza Safi
              Reporter:
              rezasafi Reza Safi
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: