Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-15455

For IsolatedClientLoader, we need to provide a conf to disable sharing Hadoop classes

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 2.0.0
    • None
    • SQL
    • None

    Description

      Right now, we always share Hadoop classes between Spark side and the metastore client side (HiveClientImpl). However, it is possible that the Hadoop used by the metastore client is in a different version of Hadoop. Thus, in this case, we cannot share Hadoop classes. Once we disable sharing Hadoop classes, we cannot pass a Hadoop Configuration to HiveClientImpl because Configuration will be loaded by different classloaders.

      Attachments

        Activity

          People

            yhuai Yin Huai
            yhuai Yin Huai
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: