Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4108

Query on csv file w/ header fails with an exception when non existing column is requested

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.3.0
    • 1.4.0
    • Storage - Text & CSV
    • None

    Description

      Drill query on a csv file with header requesting column(s) that do not exists in header fails with an exception.

      Current behavior: once extractHeader is enabled, query columns must be columns from the header

      Expected behavior: non existing columns should appear with 'null' values like default drill behavior

      0: jdbc:drill:zk=local> select Category from dfs.`/tmp/cars.csvh` limit 10;
      java.lang.ArrayIndexOutOfBoundsException: -1
      	at org.apache.drill.exec.store.easy.text.compliant.FieldVarCharOutput.<init>(FieldVarCharOutput.java:104)
      	at org.apache.drill.exec.store.easy.text.compliant.CompliantTextRecordReader.setup(CompliantTextRecordReader.java:118)
      	at org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:108)
      	at org.apache.drill.exec.store.dfs.easy.EasyFormatPlugin.getReaderBatch(EasyFormatPlugin.java:198)
      	at org.apache.drill.exec.store.dfs.easy.EasyReaderBatchCreator.getBatch(EasyReaderBatchCreator.java:35)
      	at org.apache.drill.exec.store.dfs.easy.EasyReaderBatchCreator.getBatch(EasyReaderBatchCreator.java:28)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:151)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getRootExec(ImplCreator.java:105)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:79)
      	at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:230)
      	at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      Error: SYSTEM ERROR: ArrayIndexOutOfBoundsException: -1
      
      Fragment 0:0
      
      [Error Id: f272960e-fa2f-408e-918c-722190398cd3 on blackhole:31010] (state=,code=0)
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            apol Abhijit Pol
            abhipol Abhi Pol
            Khurram Faraaz Khurram Faraaz
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment