Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1798

Dynamic partitioning occasionally fails.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0, 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      The following query runs continuously even though the progress is reached to 100%.

      tpch1> create table partitioned_lineitem (l_orderkey int8, l_partkey int8, l_suppkey int8, l_linenumber int8, l_quantity float8, l_extendedprice float8, l_discount float8, l_tax float8, l_linestatus text, l_shipdate text, l_commitdate text, l_receiptdate text, l_shipinstruct text, l_shipmode text, l_comment text) partition by column (l_returnflag text) as select l_orderkey, l_partkey, l_suppkey, l_linenumber , l_quantity , l_extendedprice , l_discount , l_tax , l_linestatus , l_shipdate , l_commitdate , l_receiptdate , l_shipinstruct , l_shipmode , l_comment, l_returnflag from lineitem;
      ...
      Progress: 100%, response time: 328.515 sec
      Progress: 100%, response time: 329.517 sec
      Progress: 100%, response time: 330.524 sec
      Progress: 100%, response time: 331.527 sec
      Progress: 100%, response time: 332.529 sec
      ...
      

      When I check the query status through the web UI, its all stages are already completed with 'SUCCEEDED' state. So, I killed the running query. However, surprisingly, I found that the result table of the above query is successfully registered in catalog as follows.

      tpch1> \d
      customer
      lineitem
      nation
      orders
      partitioned_lineitem
      partitioned_nation
      partitioned_region
      partsupp
      region
      supplier
      tpch1> \d partitioned_lineitem
      
      table name: tpch1.partitioned_lineitem
      table uri: hdfs://localhost:7020/tajo/warehouse/tpch1/partitioned_lineitem
      store type: TEXT
      number of rows: 6001215
      volume: 750.5 MB
      Options: 
      	'text.delimiter'='|'
      
      schema: 
      l_orderkey	INT8
      l_partkey	INT8
      l_suppkey	INT8
      l_linenumber	INT8
      l_quantity	FLOAT8
      l_extendedprice	FLOAT8
      l_discount	FLOAT8
      l_tax	FLOAT8
      l_linestatus	TEXT
      l_shipdate	TEXT
      l_commitdate	TEXT
      l_receiptdate	TEXT
      l_shipinstruct	TEXT
      l_shipmode	TEXT
      l_comment	TEXT
      
      Partitions: 
      type:COLUMN
      columns::l_returnflag (TEXT)
      
      tpch1> select count(*) from partitioned_lineitem;
      Progress: 6%, response time: 1.164 sec
      Progress: 6%, response time: 1.165 sec
      Progress: 6%, response time: 1.568 sec
      Progress: 100%, response time: 1.793 sec
      ?count
      -------------------------------
      6001215
      (1 rows, 1.793 sec, 8 B selected)
      

        Attachments

          Activity

            People

            • Assignee:
              blrunner Jaehwa Jung
              Reporter:
              jihoonson Jihoon Son
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: