Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-11977

Hive should handle an external avro table with zero length files present

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.14.0, 1.0.0, 1.2.0, 1.1.0, 1.2.1
    • Fix Version/s: 2.0.0
    • Labels:
      None
    • Target Version/s:

      Description

      If a zero length file is in the top level directory housing an external avro table, all hive queries on the table fail.

      This issue is that org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader creates a new org.apache.avro.file.DataFileReader and DataFileReader throws an exception when trying to read an empty file (because the empty file lacks the magic number marking it as avro).

      AvroGenericRecordReader should detect an empty file and then behave reasonably.

      Caused by: java.io.IOException: Not a data file.
      at org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:102)
      at org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:97)
      at org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader.<init>(AvroGenericRecordReader.java:81)
      at org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat.getRecordReader(AvroContainerInputFormat.java:51)
      at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:246)
      ... 25 more

        Attachments

        1. HIVE-11977.2.patch
          4 kB
          Aaron Dossett
        2. HIVE-11977.patch
          2 kB
          Aaron Dossett

          Issue Links

            Activity

              People

              • Assignee:
                dossett@gmail.com Aaron Dossett
                Reporter:
                dossett@gmail.com Aaron Dossett
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: