[ZEPPELIN-1909] spark session with hive support can't be created correctly

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.7.0
• Fix Version/s: 0.7.0
• Component/s: Interpreters
• Assignee: Unassigned
• Reporter: lichenglin
• Created: 06/Jan/17 01:15
• Updated: 10/Jan/17 07:35
• Resolved: 10/Jan/17 07:35

Description

    if (useHiveContext()) {
      if (hiveClassesArePresent()) {
        Utils.invokeMethod(builder, "enableHiveSupport");
        sparkSession = Utils.invokeMethod(builder, "getOrCreate");
        logger.info("Created Spark session with Hive support");
      } else {
        Utils.invokeMethod(builder, "config",
            new Class[]{ String.class, String.class},
            new Object[]{ "spark.sql.catalogImplementation", "in-memory"});
        sparkSession = Utils.invokeMethod(builder, "getOrCreate");
        logger.info("Created Spark session with Hive support");
      }
    } 

 /**
   * See org.apache.spark.sql.SparkSession.hiveClassesArePresent
   * @return
   */
  private boolean hiveClassesArePresent() {
    try {
      this.getClass().forName("org.apache.spark.sql.hive.HiveSessionState");
      this.getClass().forName("org.apache.spark.sql.hive.HiveSharedState");
      this.getClass().forName("org.apache.hadoop.hive.conf.HiveConf");
      return true;
    } catch (ClassNotFoundException | NoClassDefFoundError e) {
      return false;
    }
  }

since "org.apache.spark.sql.hive.HiveSharedState" has been removed from spark2.1.x

So hiveClassesArePresent will always return false
and spark session with hive support will always be created with
"spark.sql.catalogImplementation=in-memory"

Maybe just remove

      this.getClass().forName("org.apache.spark.sql.hive.HiveSharedState");

is OK?

Issue Links

• blocks: ZEPPELIN-1795 Release 0.7.0
• links to: GitHub Pull Request #1856