XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: S3, Storage
    • Labels:
      None

      Description

      Query hangs while executing CTAS for partition table on aws as following:

      • Root directory exists on s3
      • Dataset : tpc-h 1G
      • Query
        CREATE TABLE LINEITEM_P (L_ORDERKEY bigint, L_PARTKEY bigint, L_SUPPKEY bigint, L_LINENUMBER bigint, L_QUANTITY double, L_EXTENDEDPRICE double, L_DISCOUNT double, L_TAX double, L_RETURNFLAG text, L_LINESTATUS text, L_COMMITDATE text, L_RECEIPTDATE text, L_SHIPINSTRUCT text, L_SHIPMODE text, L_COMMENT text) 
        USING TEXT WITH ('text.delimiter'='|')
        PARTITION BY COLUMN (L_SHIPDATE text) AS 
        SELECT L_ORDERKEY, L_PARTKEY, L_SUPPKEY, L_LINENUMBER, L_QUANTITY, L_EXTENDEDPRICE, L_DISCOUNT, L_TAX, L_RETURNFLAG, L_LINESTATUS, L_COMMITDATE, L_RECEIPTDATE, L_SHIPINSTRUCT, L_SHIPMODE, L_COMMENT, L_SHIPDATE FROM LINEITEM;
        
      • TSql logs
        Progress: 99%, response time: 1157.913 sec
        Progress: 99%, response time: 1158.914 sec
        Progress: 100%, response time: 1159.914 sec
        Progress: 100%, response time: 1160.915 sec
        Progress: 100%, response time: 1161.916 sec
        Progress: 100%, response time: 1162.917 sec
        Progress: 100%, response time: 1163.917 sec
        Progress: 100%, response time: 1164.918 sec
        Progress: 100%, response time: 1165.919 sec
        Progress: 100%, response time: 1166.919 sec
        Progress: 100%, response time: 1167.92 sec
        Progress: 100%, response time: 1168.921 sec
        Progress: 100%, response time: 1169.921 sec
        Progress: 100%, response time: 1170.922 sec
        Progress: 100%, response time: 1171.923 sec
        Progress: 100%, response time: 1172.924 sec
        Progress: 100%, response time: 1173.924 sec
        Progress: 100%, response time: 1174.925 sec
        
      • TajoWorker logs
        2016-01-08 07:50:56,158 INFO org.apache.tajo.querymaster.Stage: Stage completed - eb_1452238256580_0001_000002 (total=6, success=6, killed=0)
        2016-01-08 07:50:56,159 INFO org.apache.tajo.querymaster.Query: Processing q_1452238256580_0001 of type STAGE_COMPLETED
        2016-01-08 07:50:56,159 INFO org.apache.tajo.engine.planner.global.ParallelExecutionQueue: Next executable block eb_1452238256580_0001_000003
        2016-01-08 07:50:56,159 INFO org.apache.tajo.querymaster.Query: Complete Stage[eb_1452238256580_0001_000002], State: SUCCEEDED, 2/2. 
        2016-01-08 07:50:56,159 INFO org.apache.tajo.querymaster.Query: Processing q_1452238256580_0001 of type QUERY_COMPLETED
        2016-01-08 07:50:56,161 INFO org.apache.tajo.worker.ExecutionBlockContext: Worker's task counter - total:6, succeeded: 6, killed: 0, failed: 0
        2016-01-08 07:50:56,194 INFO org.apache.tajo.worker.TaskManager: Stopped execution block:eb_1452238256580_0001_000002
        

      I found that query hangs on FileTableSpace::commitTable. We need to implement commitTable in S3TableSpace by considering s3 features.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                blrunner JaeHwa Jung
                Reporter:
                blrunner JaeHwa Jung
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: