[DRILL-2291] CTAS broken when we have repeated lists

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Duplicate
• Fix Version/s: 0.9.0
• Component/s: Storage - Parquet
• Assignee: Abdel Hakim Deneche
• Reporter: Rahul Kumar Challapalli
• Created: 24/Feb/15 18:58
• Updated: 01/Apr/15 22:30
• Resolved: 01/Apr/15 22:29

Description

git.commit.id.abbrev=6676f2d

Data Set :

{
  "id" : 1,
  "lst_lst" : [[1,2],[3,4]]
}

The below CTAS statement fails (The underlying query succeeds)

create table nested_list as select d.lst_lst from `temp.json` d;
Query failed: RemoteRpcException: Failure while running fragment.[ e5f3cdf2-d507-412e-a11a-24c9d2362975 on qa-node190.qa.lab:31010 ]
[ e5f3cdf2-d507-412e-a11a-24c9d2362975 on qa-node190.qa.lab:31010 ]

Below is the log file :

2015-02-24 18:45:56,332 [2b13391c-2000-e539-4adc-a661458b24c1:frag:0:0] ERROR o.a.d.e.w.f.AbstractStatusReporter - Error 3d91c456-3a24-4a8d-84b3-57a287aa2c54: Failure while running fragment.
java.lang.NullPointerException: null
        at org.apache.drill.exec.store.parquet.ParquetRecordWriter.cleanup(ParquetRecordWriter.java:318) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.WriterRecordBatch.cleanup(WriterRecordBatch.java:187) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.cleanup(IteratorValidatorBatchIterator.java:148) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractSingleRecordBatch.cleanup(AbstractSingleRecordBatch.java:121) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.cleanup(IteratorValidatorBatchIterator.java:148) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.internalStop(ScreenCreator.java:178) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:101) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:57) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:116) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:303) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
2015-02-24 18:45:56,334 [2b13391c-2000-e539-4adc-a661458b24c1:frag:0:0] INFO  o.a.drill.exec.work.foreman.Foreman - State change requested.  RUNNING --> FAILED
org.apache.drill.exec.rpc.RemoteRpcException: Failure while running fragment.[ 3d91c456-3a24-4a8d-84b3-57a287aa2c54 on qa-node190.qa.lab:31010 ]
[ 3d91c456-3a24-4a8d-84b3-57a287aa2c54 on qa-node190.qa.lab:31010 ]

        at org.apache.drill.exec.work.foreman.QueryManager.statusUpdate(QueryManager.java:95) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.QueryManager$RootStatusReporter.statusChange(QueryManager.java:154) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:114) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:110) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.internalFail(FragmentExecutor.java:168) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:131) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:303) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71]
        at java.lang.Thread.run(Thread.java:745)

Issue Links

• duplicates: DRILL-2241 CTAS fails when writing a repeated list
• relates to: DRILL-2292 CTAS broken when we have repeated maps