Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.5.0
    • Component/s: None
    • Labels:
      None

      Description

      Add support for Hive and HiveQL to the zeppelin framework.

      Users can use the %hive directive to submit HiveQL statements.

        Issue Links

          Activity

          Hide
          venkateshrin Ram Venkatesh added a comment - - edited

          Work in progress patch attached to illustrate the design, it uses the HiveJDBC driver to submit statements to HiveServer2.

          The following items need to be cleaned up:

          1. Support for other versions of Hive
          2. Query progress reporting
          Show
          venkateshrin Ram Venkatesh added a comment - - edited Work in progress patch attached to illustrate the design, it uses the HiveJDBC driver to submit statements to HiveServer2. The following items need to be cleaned up: Support for other versions of Hive Query progress reporting
          Hide
          jongyoul Jongyoul Lee added a comment -

          Ram Venkatesh

          Hi, could you make a PR for you patch? Zeppelin uses github's PR as a review system.

          Show
          jongyoul Jongyoul Lee added a comment - Ram Venkatesh Hi, could you make a PR for you patch? Zeppelin uses github's PR as a review system.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user RamVenkatesh opened a pull request:

          https://github.com/apache/incubator-zeppelin/pull/12

          ZEPPELIN-11 work in progress patch for hive interpreter

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/RamVenkatesh/incubator-zeppelin ZEPPELIN-11

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/incubator-zeppelin/pull/12.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #12


          commit 3ab8c3c8fdd899166f9b13a8397118fbc2e92cc6
          Author: Ram Venkatesh <rvenkatesh@hortonworks.com>
          Date: 2015-03-30T03:58:40Z

          ZEPPELIN-11 work in progress patch for hive interpreter


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user RamVenkatesh opened a pull request: https://github.com/apache/incubator-zeppelin/pull/12 ZEPPELIN-11 work in progress patch for hive interpreter You can merge this pull request into a Git repository by running: $ git pull https://github.com/RamVenkatesh/incubator-zeppelin ZEPPELIN-11 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-zeppelin/pull/12.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #12 commit 3ab8c3c8fdd899166f9b13a8397118fbc2e92cc6 Author: Ram Venkatesh <rvenkatesh@hortonworks.com> Date: 2015-03-30T03:58:40Z ZEPPELIN-11 work in progress patch for hive interpreter
          Hide
          venkateshrin Ram Venkatesh added a comment -

          Jongyoul Lee thank you for the comment, just uploaded the PR.

          Show
          venkateshrin Ram Venkatesh added a comment - Jongyoul Lee thank you for the comment, just uploaded the PR.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user felixcheung commented on the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87557814

          Is there a reason for a JDBC-to-Hive interpreter?
          Zeppelin already has Spark's HiveContext enabled by default.
          Please see additional discussion at https://github.com/NFLabs/zeppelin/pull/400

          Show
          githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87557814 Is there a reason for a JDBC-to-Hive interpreter? Zeppelin already has Spark's HiveContext enabled by default. Please see additional discussion at https://github.com/NFLabs/zeppelin/pull/400
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user RamVenkatesh commented on the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87560827

          The Jdbc-to-Hive interpreter enables users to send queries to a Hive server to be executed directly instead of through Spark's HiveContext. My understanding is Spark's HiveContext executes a subset of HiveQL within Spark using the Spark Sql support.

          Show
          githubbot ASF GitHub Bot added a comment - Github user RamVenkatesh commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87560827 The Jdbc-to-Hive interpreter enables users to send queries to a Hive server to be executed directly instead of through Spark's HiveContext. My understanding is Spark's HiveContext executes a subset of HiveQL within Spark using the Spark Sql support.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Leemoonsoo commented on the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87870293

          Thanks for the contribution.

          Here's some feedbacks,

          a. Could you take care of indentation of pom.xml?
          b. Version of hive-jdbc, hadoop-common dependency are hardcoded. Can we make them configurable?
          Note that spark interpreter module already uses hive.version and hadoop.version maven property for his dependency. Hive interpreter module might need new variable name.
          c. Can we have some unittest?

          Show
          githubbot ASF GitHub Bot added a comment - Github user Leemoonsoo commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87870293 Thanks for the contribution. Here's some feedbacks, a. Could you take care of indentation of pom.xml? b. Version of hive-jdbc, hadoop-common dependency are hardcoded. Can we make them configurable? Note that spark interpreter module already uses hive.version and hadoop.version maven property for his dependency. Hive interpreter module might need new variable name. c. Can we have some unittest?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jongyoul commented on the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87875307

          @RamVenkatesh Do you think you support hive itself, not hiveserver2?

          Show
          githubbot ASF GitHub Bot added a comment - Github user jongyoul commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87875307 @RamVenkatesh Do you think you support hive itself, not hiveserver2?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user RamVenkatesh commented on the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87939060

          @jongyoul I considered HiveClient and hiveserver. HiveClient will have client-side runtime issues, hiveserver has been deprecated for more than a year. It looks like Hiveserver2 is the recommended way to submit queries to Hive.

          Show
          githubbot ASF GitHub Bot added a comment - Github user RamVenkatesh commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87939060 @jongyoul I considered HiveClient and hiveserver. HiveClient will have client-side runtime issues, hiveserver has been deprecated for more than a year. It looks like Hiveserver2 is the recommended way to submit queries to Hive.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user RamVenkatesh commented on the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87939888

          @Leemoonsoo thank you for the review. I have addressed your comments in the latest patch.

          Show
          githubbot ASF GitHub Bot added a comment - Github user RamVenkatesh commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87939888 @Leemoonsoo thank you for the review. I have addressed your comments in the latest patch.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user felixcheung commented on a diff in the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#discussion_r27453903

          — Diff: hive/src/main/java/org/apache/zeppelin/hive/HiveInterpreter.java —
          @@ -0,0 +1,203 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +package org.apache.zeppelin.hive;
          +
          +import java.sql.*;
          +import java.util.List;
          +import java.util.Properties;
          +
          +import com.nflabs.zeppelin.interpreter.*;
          +import org.apache.commons.lang.StringUtils;
          +import org.slf4j.Logger;
          +import org.slf4j.LoggerFactory;
          +
          +import com.nflabs.zeppelin.interpreter.InterpreterResult.Code;
          +import com.nflabs.zeppelin.scheduler.Scheduler;
          +import com.nflabs.zeppelin.scheduler.SchedulerFactory;
          +
          +/**
          + * Hive interpreter for Zeppelin.
          + */
          +public class HiveInterpreter extends Interpreter {
          + Logger logger = LoggerFactory.getLogger(HiveInterpreter.class);
          + int commandTimeOut = 600000;
          +
          + static final String HIVESERVER_URL = "hive.hiveserver2.url";
          + static final String HIVESERVER_USER = "hive.hiveserver2.user";
          + static final String HIVESERVER_PASSWORD = "hive.hiveserver2.password";
          +
          + static {
          + Interpreter.register(
          + "hive",
          + "hive",
          + HiveInterpreter.class.getName(),
          + new InterpreterPropertyBuilder()
          + .add(HIVESERVER_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.")
          — End diff –

          Is there a convention for the name of the property? It seems most commonly they are in the form of my.property.name

          Show
          githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on a diff in the pull request: https://github.com/apache/incubator-zeppelin/pull/12#discussion_r27453903 — Diff: hive/src/main/java/org/apache/zeppelin/hive/HiveInterpreter.java — @@ -0,0 +1,203 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.zeppelin.hive; + +import java.sql.*; +import java.util.List; +import java.util.Properties; + +import com.nflabs.zeppelin.interpreter.*; +import org.apache.commons.lang.StringUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.nflabs.zeppelin.interpreter.InterpreterResult.Code; +import com.nflabs.zeppelin.scheduler.Scheduler; +import com.nflabs.zeppelin.scheduler.SchedulerFactory; + +/** + * Hive interpreter for Zeppelin. + */ +public class HiveInterpreter extends Interpreter { + Logger logger = LoggerFactory.getLogger(HiveInterpreter.class); + int commandTimeOut = 600000; + + static final String HIVESERVER_URL = "hive.hiveserver2.url"; + static final String HIVESERVER_USER = "hive.hiveserver2.user"; + static final String HIVESERVER_PASSWORD = "hive.hiveserver2.password"; + + static { + Interpreter.register( + "hive", + "hive", + HiveInterpreter.class.getName(), + new InterpreterPropertyBuilder() + .add(HIVESERVER_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.") — End diff – Is there a convention for the name of the property? It seems most commonly they are in the form of my.property.name
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user felixcheung commented on a diff in the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#discussion_r27453966

          — Diff: hive/src/main/java/org/apache/zeppelin/hive/HiveInterpreter.java —
          @@ -0,0 +1,203 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +package org.apache.zeppelin.hive;
          +
          +import java.sql.*;
          +import java.util.List;
          +import java.util.Properties;
          +
          +import com.nflabs.zeppelin.interpreter.*;
          +import org.apache.commons.lang.StringUtils;
          +import org.slf4j.Logger;
          +import org.slf4j.LoggerFactory;
          +
          +import com.nflabs.zeppelin.interpreter.InterpreterResult.Code;
          +import com.nflabs.zeppelin.scheduler.Scheduler;
          +import com.nflabs.zeppelin.scheduler.SchedulerFactory;
          +
          +/**
          + * Hive interpreter for Zeppelin.
          + */
          +public class HiveInterpreter extends Interpreter {
          + Logger logger = LoggerFactory.getLogger(HiveInterpreter.class);
          + int commandTimeOut = 600000;
          +
          + static final String HIVESERVER_URL = "hive.hiveserver2.url";
          + static final String HIVESERVER_USER = "hive.hiveserver2.user";
          + static final String HIVESERVER_PASSWORD = "hive.hiveserver2.password";
          +
          + static

          { + Interpreter.register( + "hive", + "hive", + HiveInterpreter.class.getName(), + new InterpreterPropertyBuilder() + .add(HIVESERVER_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.") + .add(HIVESERVER_USER, "hive", "The hive user") + .add(HIVESERVER_PASSWORD, "", "The password for the hive user").build()); + }

          +
          + public HiveInterpreter(Properties property)

          { + super(property); + }

          +
          + Connection jdbcConnection;
          + Exception exceptionOnConnect;
          +
          + //Test only method
          + public Connection getJdbcConnection()
          + throws SQLException

          { + String url = getProperty(HIVESERVER_URL); + String user = getProperty(HIVESERVER_USER); + String password = getProperty(HIVESERVER_PASSWORD); + + return DriverManager.getConnection(url, user, password); + }

          +
          + @Override
          + public void open() {
          + logger.info("Jdbc open connection called!");
          + try

          { + String driverName = "org.apache.hive.jdbc.HiveDriver"; + Class.forName(driverName); + }

          catch (ClassNotFoundException e)

          { + logger.error("Can not open connection", e); + exceptionOnConnect = e; + return; + }

          + try

          { + jdbcConnection = getJdbcConnection(); + exceptionOnConnect = null; + logger.info("Successfully created Jdbc connection"); + }

          + catch (SQLException e)

          { + logger.error("Cannot open connection", e); + exceptionOnConnect = e; + }

          + }
          +
          + @Override
          + public void close() {
          + try {
          + if (jdbcConnection != null)

          { + jdbcConnection.close(); + }

          + }
          + catch (SQLException e)

          { + logger.error("Cannot close connection", e); + }

          + finally

          { + jdbcConnection = null; + exceptionOnConnect = null; + }

          + }
          +
          + Statement currentStatement;
          + private InterpreterResult executeSql(String sql) {
          + try {
          + if (exceptionOnConnect != null)

          { + return new InterpreterResult(Code.ERROR, exceptionOnConnect.getMessage()); + }

          + currentStatement = jdbcConnection.createStatement();
          + StringBuilder msg = null;
          + if (StringUtils.containsIgnoreCase(sql, "EXPLAIN ")) {
          — End diff –

          should this consider the case where EXPLAIN is in a comment or as a name of a column?

          Show
          githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on a diff in the pull request: https://github.com/apache/incubator-zeppelin/pull/12#discussion_r27453966 — Diff: hive/src/main/java/org/apache/zeppelin/hive/HiveInterpreter.java — @@ -0,0 +1,203 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.zeppelin.hive; + +import java.sql.*; +import java.util.List; +import java.util.Properties; + +import com.nflabs.zeppelin.interpreter.*; +import org.apache.commons.lang.StringUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.nflabs.zeppelin.interpreter.InterpreterResult.Code; +import com.nflabs.zeppelin.scheduler.Scheduler; +import com.nflabs.zeppelin.scheduler.SchedulerFactory; + +/** + * Hive interpreter for Zeppelin. + */ +public class HiveInterpreter extends Interpreter { + Logger logger = LoggerFactory.getLogger(HiveInterpreter.class); + int commandTimeOut = 600000; + + static final String HIVESERVER_URL = "hive.hiveserver2.url"; + static final String HIVESERVER_USER = "hive.hiveserver2.user"; + static final String HIVESERVER_PASSWORD = "hive.hiveserver2.password"; + + static { + Interpreter.register( + "hive", + "hive", + HiveInterpreter.class.getName(), + new InterpreterPropertyBuilder() + .add(HIVESERVER_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.") + .add(HIVESERVER_USER, "hive", "The hive user") + .add(HIVESERVER_PASSWORD, "", "The password for the hive user").build()); + } + + public HiveInterpreter(Properties property) { + super(property); + } + + Connection jdbcConnection; + Exception exceptionOnConnect; + + //Test only method + public Connection getJdbcConnection() + throws SQLException { + String url = getProperty(HIVESERVER_URL); + String user = getProperty(HIVESERVER_USER); + String password = getProperty(HIVESERVER_PASSWORD); + + return DriverManager.getConnection(url, user, password); + } + + @Override + public void open() { + logger.info("Jdbc open connection called!"); + try { + String driverName = "org.apache.hive.jdbc.HiveDriver"; + Class.forName(driverName); + } catch (ClassNotFoundException e) { + logger.error("Can not open connection", e); + exceptionOnConnect = e; + return; + } + try { + jdbcConnection = getJdbcConnection(); + exceptionOnConnect = null; + logger.info("Successfully created Jdbc connection"); + } + catch (SQLException e) { + logger.error("Cannot open connection", e); + exceptionOnConnect = e; + } + } + + @Override + public void close() { + try { + if (jdbcConnection != null) { + jdbcConnection.close(); + } + } + catch (SQLException e) { + logger.error("Cannot close connection", e); + } + finally { + jdbcConnection = null; + exceptionOnConnect = null; + } + } + + Statement currentStatement; + private InterpreterResult executeSql(String sql) { + try { + if (exceptionOnConnect != null) { + return new InterpreterResult(Code.ERROR, exceptionOnConnect.getMessage()); + } + currentStatement = jdbcConnection.createStatement(); + StringBuilder msg = null; + if (StringUtils.containsIgnoreCase(sql, "EXPLAIN ")) { — End diff – should this consider the case where EXPLAIN is in a comment or as a name of a column?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user RamVenkatesh commented on a diff in the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#discussion_r27454028

          — Diff: hive/src/main/java/org/apache/zeppelin/hive/HiveInterpreter.java —
          @@ -0,0 +1,203 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +package org.apache.zeppelin.hive;
          +
          +import java.sql.*;
          +import java.util.List;
          +import java.util.Properties;
          +
          +import com.nflabs.zeppelin.interpreter.*;
          +import org.apache.commons.lang.StringUtils;
          +import org.slf4j.Logger;
          +import org.slf4j.LoggerFactory;
          +
          +import com.nflabs.zeppelin.interpreter.InterpreterResult.Code;
          +import com.nflabs.zeppelin.scheduler.Scheduler;
          +import com.nflabs.zeppelin.scheduler.SchedulerFactory;
          +
          +/**
          + * Hive interpreter for Zeppelin.
          + */
          +public class HiveInterpreter extends Interpreter {
          + Logger logger = LoggerFactory.getLogger(HiveInterpreter.class);
          + int commandTimeOut = 600000;
          +
          + static final String HIVESERVER_URL = "hive.hiveserver2.url";
          + static final String HIVESERVER_USER = "hive.hiveserver2.user";
          + static final String HIVESERVER_PASSWORD = "hive.hiveserver2.password";
          +
          + static {
          + Interpreter.register(
          + "hive",
          + "hive",
          + HiveInterpreter.class.getName(),
          + new InterpreterPropertyBuilder()
          + .add(HIVESERVER_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.")
          — End diff –

          Am not sure, decided to use the . for separation and readability. Could also do hive.hiveserver2-url. What do others think?

          Show
          githubbot ASF GitHub Bot added a comment - Github user RamVenkatesh commented on a diff in the pull request: https://github.com/apache/incubator-zeppelin/pull/12#discussion_r27454028 — Diff: hive/src/main/java/org/apache/zeppelin/hive/HiveInterpreter.java — @@ -0,0 +1,203 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.zeppelin.hive; + +import java.sql.*; +import java.util.List; +import java.util.Properties; + +import com.nflabs.zeppelin.interpreter.*; +import org.apache.commons.lang.StringUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.nflabs.zeppelin.interpreter.InterpreterResult.Code; +import com.nflabs.zeppelin.scheduler.Scheduler; +import com.nflabs.zeppelin.scheduler.SchedulerFactory; + +/** + * Hive interpreter for Zeppelin. + */ +public class HiveInterpreter extends Interpreter { + Logger logger = LoggerFactory.getLogger(HiveInterpreter.class); + int commandTimeOut = 600000; + + static final String HIVESERVER_URL = "hive.hiveserver2.url"; + static final String HIVESERVER_USER = "hive.hiveserver2.user"; + static final String HIVESERVER_PASSWORD = "hive.hiveserver2.password"; + + static { + Interpreter.register( + "hive", + "hive", + HiveInterpreter.class.getName(), + new InterpreterPropertyBuilder() + .add(HIVESERVER_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.") — End diff – Am not sure, decided to use the . for separation and readability. Could also do hive.hiveserver2-url. What do others think?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user RamVenkatesh commented on a diff in the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#discussion_r27454153

          — Diff: hive/src/main/java/org/apache/zeppelin/hive/HiveInterpreter.java —
          @@ -0,0 +1,203 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +package org.apache.zeppelin.hive;
          +
          +import java.sql.*;
          +import java.util.List;
          +import java.util.Properties;
          +
          +import com.nflabs.zeppelin.interpreter.*;
          +import org.apache.commons.lang.StringUtils;
          +import org.slf4j.Logger;
          +import org.slf4j.LoggerFactory;
          +
          +import com.nflabs.zeppelin.interpreter.InterpreterResult.Code;
          +import com.nflabs.zeppelin.scheduler.Scheduler;
          +import com.nflabs.zeppelin.scheduler.SchedulerFactory;
          +
          +/**
          + * Hive interpreter for Zeppelin.
          + */
          +public class HiveInterpreter extends Interpreter {
          + Logger logger = LoggerFactory.getLogger(HiveInterpreter.class);
          + int commandTimeOut = 600000;
          +
          + static final String HIVESERVER_URL = "hive.hiveserver2.url";
          + static final String HIVESERVER_USER = "hive.hiveserver2.user";
          + static final String HIVESERVER_PASSWORD = "hive.hiveserver2.password";
          +
          + static

          { + Interpreter.register( + "hive", + "hive", + HiveInterpreter.class.getName(), + new InterpreterPropertyBuilder() + .add(HIVESERVER_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.") + .add(HIVESERVER_USER, "hive", "The hive user") + .add(HIVESERVER_PASSWORD, "", "The password for the hive user").build()); + }

          +
          + public HiveInterpreter(Properties property)

          { + super(property); + }

          +
          + Connection jdbcConnection;
          + Exception exceptionOnConnect;
          +
          + //Test only method
          + public Connection getJdbcConnection()
          + throws SQLException

          { + String url = getProperty(HIVESERVER_URL); + String user = getProperty(HIVESERVER_USER); + String password = getProperty(HIVESERVER_PASSWORD); + + return DriverManager.getConnection(url, user, password); + }

          +
          + @Override
          + public void open() {
          + logger.info("Jdbc open connection called!");
          + try

          { + String driverName = "org.apache.hive.jdbc.HiveDriver"; + Class.forName(driverName); + }

          catch (ClassNotFoundException e)

          { + logger.error("Can not open connection", e); + exceptionOnConnect = e; + return; + }

          + try

          { + jdbcConnection = getJdbcConnection(); + exceptionOnConnect = null; + logger.info("Successfully created Jdbc connection"); + }

          + catch (SQLException e)

          { + logger.error("Cannot open connection", e); + exceptionOnConnect = e; + }

          + }
          +
          + @Override
          + public void close() {
          + try {
          + if (jdbcConnection != null)

          { + jdbcConnection.close(); + }

          + }
          + catch (SQLException e)

          { + logger.error("Cannot close connection", e); + }

          + finally

          { + jdbcConnection = null; + exceptionOnConnect = null; + }

          + }
          +
          + Statement currentStatement;
          + private InterpreterResult executeSql(String sql) {
          + try {
          + if (exceptionOnConnect != null)

          { + return new InterpreterResult(Code.ERROR, exceptionOnConnect.getMessage()); + }

          + currentStatement = jdbcConnection.createStatement();
          + StringBuilder msg = null;
          + if (StringUtils.containsIgnoreCase(sql, "EXPLAIN ")) {
          — End diff –

          Good question. I could not come up with a way to do this that did not involve parsing - could have gone with starts with, but that will not cover the case of blank lines at the front, comments, or hive options. It might be better to have an Explain button in the UI in addition to Play for this type of note and handle the Explain queries that way. I will open a JIRA to track it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user RamVenkatesh commented on a diff in the pull request: https://github.com/apache/incubator-zeppelin/pull/12#discussion_r27454153 — Diff: hive/src/main/java/org/apache/zeppelin/hive/HiveInterpreter.java — @@ -0,0 +1,203 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.zeppelin.hive; + +import java.sql.*; +import java.util.List; +import java.util.Properties; + +import com.nflabs.zeppelin.interpreter.*; +import org.apache.commons.lang.StringUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.nflabs.zeppelin.interpreter.InterpreterResult.Code; +import com.nflabs.zeppelin.scheduler.Scheduler; +import com.nflabs.zeppelin.scheduler.SchedulerFactory; + +/** + * Hive interpreter for Zeppelin. + */ +public class HiveInterpreter extends Interpreter { + Logger logger = LoggerFactory.getLogger(HiveInterpreter.class); + int commandTimeOut = 600000; + + static final String HIVESERVER_URL = "hive.hiveserver2.url"; + static final String HIVESERVER_USER = "hive.hiveserver2.user"; + static final String HIVESERVER_PASSWORD = "hive.hiveserver2.password"; + + static { + Interpreter.register( + "hive", + "hive", + HiveInterpreter.class.getName(), + new InterpreterPropertyBuilder() + .add(HIVESERVER_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.") + .add(HIVESERVER_USER, "hive", "The hive user") + .add(HIVESERVER_PASSWORD, "", "The password for the hive user").build()); + } + + public HiveInterpreter(Properties property) { + super(property); + } + + Connection jdbcConnection; + Exception exceptionOnConnect; + + //Test only method + public Connection getJdbcConnection() + throws SQLException { + String url = getProperty(HIVESERVER_URL); + String user = getProperty(HIVESERVER_USER); + String password = getProperty(HIVESERVER_PASSWORD); + + return DriverManager.getConnection(url, user, password); + } + + @Override + public void open() { + logger.info("Jdbc open connection called!"); + try { + String driverName = "org.apache.hive.jdbc.HiveDriver"; + Class.forName(driverName); + } catch (ClassNotFoundException e) { + logger.error("Can not open connection", e); + exceptionOnConnect = e; + return; + } + try { + jdbcConnection = getJdbcConnection(); + exceptionOnConnect = null; + logger.info("Successfully created Jdbc connection"); + } + catch (SQLException e) { + logger.error("Cannot open connection", e); + exceptionOnConnect = e; + } + } + + @Override + public void close() { + try { + if (jdbcConnection != null) { + jdbcConnection.close(); + } + } + catch (SQLException e) { + logger.error("Cannot close connection", e); + } + finally { + jdbcConnection = null; + exceptionOnConnect = null; + } + } + + Statement currentStatement; + private InterpreterResult executeSql(String sql) { + try { + if (exceptionOnConnect != null) { + return new InterpreterResult(Code.ERROR, exceptionOnConnect.getMessage()); + } + currentStatement = jdbcConnection.createStatement(); + StringBuilder msg = null; + if (StringUtils.containsIgnoreCase(sql, "EXPLAIN ")) { — End diff – Good question. I could not come up with a way to do this that did not involve parsing - could have gone with starts with, but that will not cover the case of blank lines at the front, comments, or hive options. It might be better to have an Explain button in the UI in addition to Play for this type of note and handle the Explain queries that way. I will open a JIRA to track it.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user felixcheung commented on a diff in the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#discussion_r27454836

          — Diff: hive/src/main/java/org/apache/zeppelin/hive/HiveInterpreter.java —
          @@ -0,0 +1,203 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +package org.apache.zeppelin.hive;
          +
          +import java.sql.*;
          +import java.util.List;
          +import java.util.Properties;
          +
          +import com.nflabs.zeppelin.interpreter.*;
          +import org.apache.commons.lang.StringUtils;
          +import org.slf4j.Logger;
          +import org.slf4j.LoggerFactory;
          +
          +import com.nflabs.zeppelin.interpreter.InterpreterResult.Code;
          +import com.nflabs.zeppelin.scheduler.Scheduler;
          +import com.nflabs.zeppelin.scheduler.SchedulerFactory;
          +
          +/**
          + * Hive interpreter for Zeppelin.
          + */
          +public class HiveInterpreter extends Interpreter {
          + Logger logger = LoggerFactory.getLogger(HiveInterpreter.class);
          + int commandTimeOut = 600000;
          +
          + static final String HIVESERVER_URL = "hive.hiveserver2.url";
          + static final String HIVESERVER_USER = "hive.hiveserver2.user";
          + static final String HIVESERVER_PASSWORD = "hive.hiveserver2.password";
          +
          + static

          { + Interpreter.register( + "hive", + "hive", + HiveInterpreter.class.getName(), + new InterpreterPropertyBuilder() + .add(HIVESERVER_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.") + .add(HIVESERVER_USER, "hive", "The hive user") + .add(HIVESERVER_PASSWORD, "", "The password for the hive user").build()); + }

          +
          + public HiveInterpreter(Properties property)

          { + super(property); + }

          +
          + Connection jdbcConnection;
          + Exception exceptionOnConnect;
          +
          + //Test only method
          + public Connection getJdbcConnection()
          + throws SQLException

          { + String url = getProperty(HIVESERVER_URL); + String user = getProperty(HIVESERVER_USER); + String password = getProperty(HIVESERVER_PASSWORD); + + return DriverManager.getConnection(url, user, password); + }

          +
          + @Override
          + public void open() {
          + logger.info("Jdbc open connection called!");
          + try

          { + String driverName = "org.apache.hive.jdbc.HiveDriver"; + Class.forName(driverName); + }

          catch (ClassNotFoundException e)

          { + logger.error("Can not open connection", e); + exceptionOnConnect = e; + return; + }

          + try

          { + jdbcConnection = getJdbcConnection(); + exceptionOnConnect = null; + logger.info("Successfully created Jdbc connection"); + }

          + catch (SQLException e)

          { + logger.error("Cannot open connection", e); + exceptionOnConnect = e; + }

          + }
          +
          + @Override
          + public void close() {
          + try {
          + if (jdbcConnection != null)

          { + jdbcConnection.close(); + }

          + }
          + catch (SQLException e)

          { + logger.error("Cannot close connection", e); + }

          + finally

          { + jdbcConnection = null; + exceptionOnConnect = null; + }

          + }
          +
          + Statement currentStatement;
          + private InterpreterResult executeSql(String sql) {
          + try {
          + if (exceptionOnConnect != null)

          { + return new InterpreterResult(Code.ERROR, exceptionOnConnect.getMessage()); + }

          + currentStatement = jdbcConnection.createStatement();
          + StringBuilder msg = null;
          + if (StringUtils.containsIgnoreCase(sql, "EXPLAIN ")) {
          — End diff –

          Is there a way to interpret the result (`msg`) instead?

          Show
          githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on a diff in the pull request: https://github.com/apache/incubator-zeppelin/pull/12#discussion_r27454836 — Diff: hive/src/main/java/org/apache/zeppelin/hive/HiveInterpreter.java — @@ -0,0 +1,203 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.zeppelin.hive; + +import java.sql.*; +import java.util.List; +import java.util.Properties; + +import com.nflabs.zeppelin.interpreter.*; +import org.apache.commons.lang.StringUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.nflabs.zeppelin.interpreter.InterpreterResult.Code; +import com.nflabs.zeppelin.scheduler.Scheduler; +import com.nflabs.zeppelin.scheduler.SchedulerFactory; + +/** + * Hive interpreter for Zeppelin. + */ +public class HiveInterpreter extends Interpreter { + Logger logger = LoggerFactory.getLogger(HiveInterpreter.class); + int commandTimeOut = 600000; + + static final String HIVESERVER_URL = "hive.hiveserver2.url"; + static final String HIVESERVER_USER = "hive.hiveserver2.user"; + static final String HIVESERVER_PASSWORD = "hive.hiveserver2.password"; + + static { + Interpreter.register( + "hive", + "hive", + HiveInterpreter.class.getName(), + new InterpreterPropertyBuilder() + .add(HIVESERVER_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.") + .add(HIVESERVER_USER, "hive", "The hive user") + .add(HIVESERVER_PASSWORD, "", "The password for the hive user").build()); + } + + public HiveInterpreter(Properties property) { + super(property); + } + + Connection jdbcConnection; + Exception exceptionOnConnect; + + //Test only method + public Connection getJdbcConnection() + throws SQLException { + String url = getProperty(HIVESERVER_URL); + String user = getProperty(HIVESERVER_USER); + String password = getProperty(HIVESERVER_PASSWORD); + + return DriverManager.getConnection(url, user, password); + } + + @Override + public void open() { + logger.info("Jdbc open connection called!"); + try { + String driverName = "org.apache.hive.jdbc.HiveDriver"; + Class.forName(driverName); + } catch (ClassNotFoundException e) { + logger.error("Can not open connection", e); + exceptionOnConnect = e; + return; + } + try { + jdbcConnection = getJdbcConnection(); + exceptionOnConnect = null; + logger.info("Successfully created Jdbc connection"); + } + catch (SQLException e) { + logger.error("Cannot open connection", e); + exceptionOnConnect = e; + } + } + + @Override + public void close() { + try { + if (jdbcConnection != null) { + jdbcConnection.close(); + } + } + catch (SQLException e) { + logger.error("Cannot close connection", e); + } + finally { + jdbcConnection = null; + exceptionOnConnect = null; + } + } + + Statement currentStatement; + private InterpreterResult executeSql(String sql) { + try { + if (exceptionOnConnect != null) { + return new InterpreterResult(Code.ERROR, exceptionOnConnect.getMessage()); + } + currentStatement = jdbcConnection.createStatement(); + StringBuilder msg = null; + if (StringUtils.containsIgnoreCase(sql, "EXPLAIN ")) { — End diff – Is there a way to interpret the result (`msg`) instead?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Leemoonsoo commented on the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87954867

          @RamVenkatesh
          Thanks for updating patch.

          I'm not sure if any complex step required before call `new HiveServer2()` for testing. Please check here https://github.com/apache/hive/blob/trunk/service/src/test/org/apache/hive/service/cli/thrift/ThriftCLIServiceTest.java#L64. If it is not too difficult, how about start hiveserver2 in testcase?

          Show
          githubbot ASF GitHub Bot added a comment - Github user Leemoonsoo commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87954867 @RamVenkatesh Thanks for updating patch. I'm not sure if any complex step required before call `new HiveServer2()` for testing. Please check here https://github.com/apache/hive/blob/trunk/service/src/test/org/apache/hive/service/cli/thrift/ThriftCLIServiceTest.java#L64 . If it is not too difficult, how about start hiveserver2 in testcase?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user RamVenkatesh commented on a diff in the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#discussion_r27455206

          — Diff: hive/src/main/java/org/apache/zeppelin/hive/HiveInterpreter.java —
          @@ -0,0 +1,203 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +package org.apache.zeppelin.hive;
          +
          +import java.sql.*;
          +import java.util.List;
          +import java.util.Properties;
          +
          +import com.nflabs.zeppelin.interpreter.*;
          +import org.apache.commons.lang.StringUtils;
          +import org.slf4j.Logger;
          +import org.slf4j.LoggerFactory;
          +
          +import com.nflabs.zeppelin.interpreter.InterpreterResult.Code;
          +import com.nflabs.zeppelin.scheduler.Scheduler;
          +import com.nflabs.zeppelin.scheduler.SchedulerFactory;
          +
          +/**
          + * Hive interpreter for Zeppelin.
          + */
          +public class HiveInterpreter extends Interpreter {
          + Logger logger = LoggerFactory.getLogger(HiveInterpreter.class);
          + int commandTimeOut = 600000;
          +
          + static final String HIVESERVER_URL = "hive.hiveserver2.url";
          + static final String HIVESERVER_USER = "hive.hiveserver2.user";
          + static final String HIVESERVER_PASSWORD = "hive.hiveserver2.password";
          +
          + static

          { + Interpreter.register( + "hive", + "hive", + HiveInterpreter.class.getName(), + new InterpreterPropertyBuilder() + .add(HIVESERVER_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.") + .add(HIVESERVER_USER, "hive", "The hive user") + .add(HIVESERVER_PASSWORD, "", "The password for the hive user").build()); + }

          +
          + public HiveInterpreter(Properties property)

          { + super(property); + }

          +
          + Connection jdbcConnection;
          + Exception exceptionOnConnect;
          +
          + //Test only method
          + public Connection getJdbcConnection()
          + throws SQLException

          { + String url = getProperty(HIVESERVER_URL); + String user = getProperty(HIVESERVER_USER); + String password = getProperty(HIVESERVER_PASSWORD); + + return DriverManager.getConnection(url, user, password); + }

          +
          + @Override
          + public void open() {
          + logger.info("Jdbc open connection called!");
          + try

          { + String driverName = "org.apache.hive.jdbc.HiveDriver"; + Class.forName(driverName); + }

          catch (ClassNotFoundException e)

          { + logger.error("Can not open connection", e); + exceptionOnConnect = e; + return; + }

          + try

          { + jdbcConnection = getJdbcConnection(); + exceptionOnConnect = null; + logger.info("Successfully created Jdbc connection"); + }

          + catch (SQLException e)

          { + logger.error("Cannot open connection", e); + exceptionOnConnect = e; + }

          + }
          +
          + @Override
          + public void close() {
          + try {
          + if (jdbcConnection != null)

          { + jdbcConnection.close(); + }

          + }
          + catch (SQLException e)

          { + logger.error("Cannot close connection", e); + }

          + finally

          { + jdbcConnection = null; + exceptionOnConnect = null; + }

          + }
          +
          + Statement currentStatement;
          + private InterpreterResult executeSql(String sql) {
          + try {
          + if (exceptionOnConnect != null)

          { + return new InterpreterResult(Code.ERROR, exceptionOnConnect.getMessage()); + }

          + currentStatement = jdbcConnection.createStatement();
          + StringBuilder msg = null;
          + if (StringUtils.containsIgnoreCase(sql, "EXPLAIN ")) {
          — End diff –

          The result is a rowset with one column, indistinguishable from a query that returns a single column.

          Show
          githubbot ASF GitHub Bot added a comment - Github user RamVenkatesh commented on a diff in the pull request: https://github.com/apache/incubator-zeppelin/pull/12#discussion_r27455206 — Diff: hive/src/main/java/org/apache/zeppelin/hive/HiveInterpreter.java — @@ -0,0 +1,203 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.zeppelin.hive; + +import java.sql.*; +import java.util.List; +import java.util.Properties; + +import com.nflabs.zeppelin.interpreter.*; +import org.apache.commons.lang.StringUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.nflabs.zeppelin.interpreter.InterpreterResult.Code; +import com.nflabs.zeppelin.scheduler.Scheduler; +import com.nflabs.zeppelin.scheduler.SchedulerFactory; + +/** + * Hive interpreter for Zeppelin. + */ +public class HiveInterpreter extends Interpreter { + Logger logger = LoggerFactory.getLogger(HiveInterpreter.class); + int commandTimeOut = 600000; + + static final String HIVESERVER_URL = "hive.hiveserver2.url"; + static final String HIVESERVER_USER = "hive.hiveserver2.user"; + static final String HIVESERVER_PASSWORD = "hive.hiveserver2.password"; + + static { + Interpreter.register( + "hive", + "hive", + HiveInterpreter.class.getName(), + new InterpreterPropertyBuilder() + .add(HIVESERVER_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.") + .add(HIVESERVER_USER, "hive", "The hive user") + .add(HIVESERVER_PASSWORD, "", "The password for the hive user").build()); + } + + public HiveInterpreter(Properties property) { + super(property); + } + + Connection jdbcConnection; + Exception exceptionOnConnect; + + //Test only method + public Connection getJdbcConnection() + throws SQLException { + String url = getProperty(HIVESERVER_URL); + String user = getProperty(HIVESERVER_USER); + String password = getProperty(HIVESERVER_PASSWORD); + + return DriverManager.getConnection(url, user, password); + } + + @Override + public void open() { + logger.info("Jdbc open connection called!"); + try { + String driverName = "org.apache.hive.jdbc.HiveDriver"; + Class.forName(driverName); + } catch (ClassNotFoundException e) { + logger.error("Can not open connection", e); + exceptionOnConnect = e; + return; + } + try { + jdbcConnection = getJdbcConnection(); + exceptionOnConnect = null; + logger.info("Successfully created Jdbc connection"); + } + catch (SQLException e) { + logger.error("Cannot open connection", e); + exceptionOnConnect = e; + } + } + + @Override + public void close() { + try { + if (jdbcConnection != null) { + jdbcConnection.close(); + } + } + catch (SQLException e) { + logger.error("Cannot close connection", e); + } + finally { + jdbcConnection = null; + exceptionOnConnect = null; + } + } + + Statement currentStatement; + private InterpreterResult executeSql(String sql) { + try { + if (exceptionOnConnect != null) { + return new InterpreterResult(Code.ERROR, exceptionOnConnect.getMessage()); + } + currentStatement = jdbcConnection.createStatement(); + StringBuilder msg = null; + if (StringUtils.containsIgnoreCase(sql, "EXPLAIN ")) { — End diff – The result is a rowset with one column, indistinguishable from a query that returns a single column.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user RamVenkatesh commented on the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87956864

          I intentionally wanted to keep the unit tests simple with no dependencies on hiveserver2, hive, hadoop etc. Adding them will make the test case more real, but also increase burden on anyone who wants to run unit tests. This way we get some coverage of the interpreter while mocking tiny bits of the Jdbc driver. We can add the mocks that we need as we add more functionality (such as progress).

          Show
          githubbot ASF GitHub Bot added a comment - Github user RamVenkatesh commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-87956864 I intentionally wanted to keep the unit tests simple with no dependencies on hiveserver2, hive, hadoop etc. Adding them will make the test case more real, but also increase burden on anyone who wants to run unit tests. This way we get some coverage of the interpreter while mocking tiny bits of the Jdbc driver. We can add the mocks that we need as we add more functionality (such as progress).
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Leemoonsoo commented on the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-89779571

          Tested and Looks good to me.

          Show
          githubbot ASF GitHub Bot added a comment - Github user Leemoonsoo commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-89779571 Tested and Looks good to me.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user RamVenkatesh commented on the pull request:

          https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-89787557

          @Leemoonsoo Thank you for the review and testing, appreciate it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user RamVenkatesh commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/12#issuecomment-89787557 @Leemoonsoo Thank you for the review and testing, appreciate it.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/incubator-zeppelin/pull/12

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/incubator-zeppelin/pull/12
          Hide
          moon Lee moon soo added a comment -
          Show
          moon Lee moon soo added a comment - Issue resolved by pull request 12 https://github.com/apache/incubator-zeppelin/pulls/12

            People

            • Assignee:
              venkateshrin Ram Venkatesh
              Reporter:
              venkateshrin Ram Venkatesh
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development