Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-4795

Delete/Alter/Describe actions fail when SerDe is not on class path

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.14.0
    • Component/s: CLI
    • Labels:
    • Environment:

      CDH 4.2
      java version "1.6.0_24"

    • Tags:
      serde

      Description

      Use case:

      Old table was created using a SerDe and system has gone through some re-configuration resulting in that serde no longer being used and hence removed. Actions necessary to identify and fix the offending tables (e.g. describe, alter, delete) fail due to the SerDe no longer being there.

      Repro Steps:

      1. Create table that uses a SerDe loaded in externally (via "add jar ...")
      2. Remove the jar that the SerDe comes from
      3. Attempt to describe, alter (set serde ...), or delete the table created in step #1

      Expected Behavior:

      Regardless of the status of the SerDe behind the table, perform those actions. Loading the SerDe shouldn't be a requirement to do so.

      Actual Behavior:

      SerDeException thrown:

      org.apache.hadoop.hive.serde2.SerDeUtils.lookupDeserializer(SerDeUtils.java:86)
      org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:207)
      org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:266)
      org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:259)
      org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:585)
      org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:168)
      org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:967)
      org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.addInputsOutputsAlterTable(DDLSemanticAnalyzer.java:1109)
      org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerde(DDLSemanticAnalyzer.java:1037)
      org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:214)
      ...
      

      or

      java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe f.q.c.n.MySerDe does not exist)
          at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:268)
          at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:259)
          at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:585)
          at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:168)
          at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:967)
          at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:868)
          at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer$QualifiedNameUtil.getAttemptTableName(DDLSemanticAnalyzer.java:1420)
          at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer$QualifiedNameUtil.getTableName(DDLSemanticAnalyzer.java:1478)
          ....
      

      Workaround:

      Find SerDe that the problematic table was configured to use (which may not readily be possible), and manually load it (via 'add jar ...') in order to fix the problem.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                calebjones Caleb Jones
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: