Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12887

Handle ORC schema on read with fewer columns than file schema (after Schema Evolution changes)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.0, 2.0.1, 2.1.0
    • Component/s: ORC
    • Labels:
      None
    • Target Version/s:

      Description

      Exception caused by reading after column removal.

      Caused by: java.lang.IndexOutOfBoundsException: Index: 10, Size: 10
      	at java.util.ArrayList.rangeCheck(ArrayList.java:653)
      	at java.util.ArrayList.get(ArrayList.java:429)
      	at java.util.Collections$UnmodifiableList.get(Collections.java:1309)
      	at org.apache.hadoop.hive.ql.io.orc.OrcProto$Type.getSubtypes(OrcProto.java:12240)
      	at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory$StructTreeReader.<init>(TreeReaderFactory.java:2053)
      	at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory.createTreeReader(TreeReaderFactory.java:2481)
      	at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.<init>(RecordReaderImpl.java:216)
      	at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions(ReaderImpl.java:598)
      	at org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$ReaderPair.<init>(OrcRawRecordMerger.java:179)
      	at org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$OriginalReaderPair.<init>(OrcRawRecordMerger.java:222)
      	at org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.<init>(OrcRawRecordMerger.java:442)
      	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1285)
      	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1165)
      	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:249)
      

        Attachments

        1. HIVE-12887.01.patch
          8 kB
          Matt McCline
        2. HIVE-12887.02.patch
          16 kB
          Matt McCline

          Activity

            People

            • Assignee:
              mmccline Matt McCline
              Reporter:
              mmccline Matt McCline
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: