Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-4071

ConvertJSONToSQL does not support Hive

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • Extensions
    • None

    Description

      Currently, the ConvertJSONToSQL processor does not support Hive as the target database, either using a HiveConnectionPool or a DBCPConnectionPool configured with a Hive driver. At the very least:

      1) A SQLException occurs when determining the auto-increment value. This is due to a Hive bug (HIVE-13528 https://issues.apache.org/jira/browse/HIVE-13528) where the column is not named according to the spec (http://docs.oracle.com/javase/8/docs/api/java/sql/DatabaseMetaData.html#getColumns-java.lang.String-java.lang.String-java.lang.String-java.lang.String-).

      2) Column names are returned with the table name prepended (using a dot separator)

      3) There may be other JDBC API calls that are not supported by the Hive JDBC driver.

      #1 could be solved by checking for "IS_AUTOINCREMENT" then failing over to "IS_AUTO_INCREMENT". #2 could be solved with a lastIndexOf("."), as is done in org.apache.nifi.util.hive.HiveJdbcCommon#createSchema(java.sql.ResultSet, java.lang.String). #3 would take some investigation into the other JDBC API calls being made.

      Attachments

        Activity

          People

            Unassigned Unassigned
            mattyb149 Matt Burgess
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: