[TAJO-1720] When converting type error in Parquet occurs, query execution doesn't stop.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 0.10.1
• Component/s: Storage
• Labels: parquet
• Assignee: Unassigned
• Reporter: Jongyoung Park
• Created: 29/Jul/15 07:30
• Updated: 15/Oct/15 04:35
• Resolved: 15/Oct/15 04:35

Description

I am testing Tajo with TPC-H dataset.
The scheme of Lineitem table is like following :

create external table lineitem_parquet (
    L_ORDERKEY bigint,
    L_PARTKEY bigint,
    L_SUPPKEY bigint,
    L_LINENUMBER bigint,
    L_QUANTITY double,
    L_EXTENDEDPRICE double,
    L_DISCOUNT double,
    L_TAX double,
    L_RETURNFLAG text,
    L_LINESTATUS text,
    L_SHIPDATE date,
    L_COMMITDATE date,
    L_RECEIPTDATE date,
    L_SHIPINSTRUCT text,
    L_SHIPMODE text,
    L_COMMENT text)
using parquet location 'hdfs://localhost:8020/tpch/parquet/lineitem';

After that, I did 'insert overwrite' into the table from lineitem as text file.
Because Tajo Parquet module doesn't support DATE type currently, it is clear that converting error occurs.

2015-07-29 16:18:13,826 ERROR org.apache.tajo.worker.TaskImpl: Cannot convert Tajo type: DATE
java.lang.RuntimeException: Cannot convert Tajo type: DATE
at org.apache.tajo.storage.parquet.TajoSchemaConverter.convertColumn(TajoSchemaConverter.java:191)
at org.apache.tajo.storage.parquet.TajoSchemaConverter.convert(TajoSchemaConverter.java:150)
at org.apache.tajo.storage.parquet.TajoWriteSupport.<init>(TajoWriteSupport.java:53)
at org.apache.tajo.storage.parquet.TajoParquetWriter.<init>(TajoParquetWriter.java:80)
at org.apache.tajo.storage.parquet.ParquetAppender.init(ParquetAppender.java:78)
at org.apache.tajo.engine.planner.physical.StoreTableExec.openNewFile(StoreTableExec.java:119)
at org.apache.tajo.engine.planner.physical.StoreTableExec.init(StoreTableExec.java:79)
at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:398)
at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-07-29 16:18:13,827 DEBUG org.apache.tajo.storage.text.DelimitedTextFile: DelimitedTextFileScanner processed record:0
2015-07-29 16:18:13,828 ERROR org.apache.tajo.worker.TaskContainer:
java.lang.NullPointerException
at org.apache.tajo.storage.parquet.ParquetAppender.getStats(ParquetAppender.java:147)
at org.apache.tajo.engine.planner.physical.StoreTableExec.close(StoreTableExec.java:156)
at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:411)
at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-07-29 16:18:13,828 INFO org.apache.tajo.worker.TaskAttemptContext: Query status of ta_1438154071434_0001_000001_000000_00 is changed to TA_FAILED
2015-07-29 16:18:13,830 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.tajo.master.event.TaskFatalErrorEvent.EventType: TA_FATAL_ERROR
2015-07-29 16:18:13,830 DEBUG org.apache.tajo.querymaster.DefaultTaskScheduler: Try to schedule task resources: 2

But query execution didn't stop and progress 0% was being displayed endlessly.
It looks it should display corresponding error and stop.