Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1720

When converting type error in Parquet occurs, query execution doesn't stop.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.10.1
    • Fix Version/s: None
    • Component/s: Storage
    • Labels:

      Description

      I am testing Tajo with TPC-H dataset.
      The scheme of Lineitem table is like following :

      create external table lineitem_parquet (
          L_ORDERKEY bigint,
          L_PARTKEY bigint,
          L_SUPPKEY bigint,
          L_LINENUMBER bigint,
          L_QUANTITY double,
          L_EXTENDEDPRICE double,
          L_DISCOUNT double,
          L_TAX double,
          L_RETURNFLAG text,
          L_LINESTATUS text,
          L_SHIPDATE date,
          L_COMMITDATE date,
          L_RECEIPTDATE date,
          L_SHIPINSTRUCT text,
          L_SHIPMODE text,
          L_COMMENT text)
      using parquet location 'hdfs://localhost:8020/tpch/parquet/lineitem';
      

      After that, I did 'insert overwrite' into the table from lineitem as text file.
      Because Tajo Parquet module doesn't support DATE type currently, it is clear that converting error occurs.

      2015-07-29 16:18:13,826 ERROR org.apache.tajo.worker.TaskImpl: Cannot convert Tajo type: DATE
      java.lang.RuntimeException: Cannot convert Tajo type: DATE
      at org.apache.tajo.storage.parquet.TajoSchemaConverter.convertColumn(TajoSchemaConverter.java:191)
      at org.apache.tajo.storage.parquet.TajoSchemaConverter.convert(TajoSchemaConverter.java:150)
      at org.apache.tajo.storage.parquet.TajoWriteSupport.<init>(TajoWriteSupport.java:53)
      at org.apache.tajo.storage.parquet.TajoParquetWriter.<init>(TajoParquetWriter.java:80)
      at org.apache.tajo.storage.parquet.ParquetAppender.init(ParquetAppender.java:78)
      at org.apache.tajo.engine.planner.physical.StoreTableExec.openNewFile(StoreTableExec.java:119)
      at org.apache.tajo.engine.planner.physical.StoreTableExec.init(StoreTableExec.java:79)
      at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:398)
      at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      2015-07-29 16:18:13,827 DEBUG org.apache.tajo.storage.text.DelimitedTextFile: DelimitedTextFileScanner processed record:0
      2015-07-29 16:18:13,828 ERROR org.apache.tajo.worker.TaskContainer:
      java.lang.NullPointerException
      at org.apache.tajo.storage.parquet.ParquetAppender.getStats(ParquetAppender.java:147)
      at org.apache.tajo.engine.planner.physical.StoreTableExec.close(StoreTableExec.java:156)
      at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:411)
      at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      2015-07-29 16:18:13,828 INFO org.apache.tajo.worker.TaskAttemptContext: Query status of ta_1438154071434_0001_000001_000000_00 is changed to TA_FAILED
      2015-07-29 16:18:13,830 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.tajo.master.event.TaskFatalErrorEvent.EventType: TA_FATAL_ERROR
      2015-07-29 16:18:13,830 DEBUG org.apache.tajo.querymaster.DefaultTaskScheduler: Try to schedule task resources: 2

      But query execution didn't stop and progress 0% was being displayed endlessly.
      It looks it should display corresponding error and stop.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              eminency Jongyoung Park
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: