Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15204

Hive-Hbase integration thorws "java.lang.ClassNotFoundException: NULL::character varying" (Postgres)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.1.0
    • Fix Version/s: None
    • Component/s: HBase Handler
    • Labels:
    • Environment:

      apache-hive-2.1.0-bin
      hbase-1.1.1

    • Flags:
      Important

      Description

      When doing hive to hbase integration, we have observed that current Apache Hive 2.x is not able to recognise 'NULL::character varying' (Variant data type of NULL in prostgres) properly and throws the java.lang.ClassNotFoundException exception.

      Exception:
      ERROR ql.Driver: FAILED: RuntimeException java.lang.ClassNotFoundException: NULL::character varying
      java.lang.RuntimeException: java.lang.ClassNotFoundException: NULL::character varying
      ....
      Caused by: java.lang.ClassNotFoundException: NULL::character varying
      at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

      Reason:

      org.apache.hadoop.hive.ql.metadata.Table.java

      final public Class<? extends InputFormat> getInputFormatClass() {
      if (inputFormatClass == null) {
      try {
      String className = tTable.getSd().getInputFormat();

      if (className == null) { /If the className is one of the postgres variant of NULL i.e. 'NULL::character varying' control is going to else block and throwing error./

      if (getStorageHandler() == null)

      { return null; }

      inputFormatClass = getStorageHandler().getInputFormatClass();
      } else

      { inputFormatClass = (Class<? extends InputFormat>) Class.forName(className, true, Utilities.getSessionSpecifiedClassLoader()); }

      } catch (ClassNotFoundException e)

      { throw new RuntimeException(e); }

      }
      return inputFormatClass;
      }

      Steps to reproduce:
      Hive 2.x (e.g. apache-hive-2.1.0-bin) and HBase (e.g. hbase-1.1.1)

      1. Install and configure Hive, if it is not already installed.
      2. Install and configure HBase, if it is not already installed.
      3. Configure the hive-site.xml File (as per recommended steps)
      4. Provide necessary jars to Hive (as per recommended steps)
      4. Create table in HBase as shown below -

      create 'hivehbase', 'ratings'
      put 'hivehbase', 'row1', 'ratings:userid', 'user1'
      put 'hivehbase', 'row1', 'ratings:bookid', 'book1'
      put 'hivehbase', 'row1', 'ratings:rating', '1'

      put 'hivehbase', 'row2', 'ratings:userid', 'user2'
      put 'hivehbase', 'row2', 'ratings:bookid', 'book1'
      put 'hivehbase', 'row2', 'ratings:rating', '3'

      put 'hivehbase', 'row3', 'ratings:userid', 'user2'
      put 'hivehbase', 'row3', 'ratings:bookid', 'book2'
      put 'hivehbase', 'row3', 'ratings:rating', '3'

      put 'hivehbase', 'row4', 'ratings:userid', 'user2'
      put 'hivehbase', 'row4', 'ratings:bookid', 'book4'
      put 'hivehbase', 'row4', 'ratings:rating', '1'

      5. Create external table as shown below
      CREATE EXTERNAL TABLE hbasehive_table
      (key string, userid string,bookid string,rating int)
      STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
      WITH SERDEPROPERTIES
      ("hbase.columns.mapping" = ":key,ratings:userid,ratings:bookid,ratings:rating")
      TBLPROPERTIES ("hbase.table.name" = "hivehbase");

      6. select * from hbasehive_table;
      FAILED: RuntimeException java.lang.ClassNotFoundException: NULL::character varying

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                anshuman.dwivedi Anshuman
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: