Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7926 long-lived daemons for query fragment execution, I/O and caching
  3. HIVE-9978

LLAP: OrcColumnVectorProducer should handle reading isPresent columns only

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: llap
    • Fix Version/s: llap
    • Component/s: File Formats
    • Labels:
      None

      Description

      LlapInputFormat does not understand the difference between empty columns list and null columns list.

      The empty columns list indicates no columns read except the root struct isPresent column, while the null columns list indicates that all columns are being read.

      select count(1) from store_sales join date_dim on ss_sold_date_sk = d_date_sk where d_date = '1998-01-01';
      ...
      Caused by: java.lang.NullPointerException
              at org.apache.hadoop.hive.llap.io.decode.OrcColumnVectorProducer.createReadPipeline(OrcColumnVectorProducer.java:72)
              at org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.startRead(LlapInputFormat.java:181)
              at org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.next(LlapInputFormat.java:140)
              at org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.next(LlapInputFormat.java:99)
              at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:350)
              ... 22 more
      

        Attachments

        1. HIVE-9978.patch
          273 kB
          Sergey Shelukhin

          Issue Links

            Activity

              People

              • Assignee:
                sershe Sergey Shelukhin
                Reporter:
                gopalv Gopal Vijayaraghavan
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: