Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1959 Improve AWS S3 file system support
  3. TAJO-1905

Insert clause to partitioned table fails on S3

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: QueryMaster, S3
    • Labels:
      None

      Description

      Here is the error log

      2015-10-02 18:54:40,399 ERROR org.apache.hadoop.fs.s3a.S3AFileSystem: rename: src not found s3a://bucket/tpch-1g-p/lineitem/.staging/q_1443779192380_0001/RESULT/l_shipdate=1996-01-30
      2015-10-02 18:54:51,357 ERROR org.apache.hadoop.fs.s3a.S3AFileSystem: rename: src not found s3a://bucket/tpch-1g-p/lineitem/.staging/q_1443779192380_0001/RESULT/l_shipdate=1993-11-09
      2015-10-02 18:55:03,955 ERROR org.apache.tajo.querymaster.Query: No such file or directory: s3a://bucket/lineitem/l_shipdate=1994-02-02
      java.io.FileNotFoundException: No such file or directory: s3a://bucket/tpch-1g-p/lineitem/l_shipdate=1994-02-02
      at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:996)
      at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:77)
      at org.apache.hadoop.fs.FileSystem.getContentSummary(FileSystem.java:1467)
      at org.apache.tajo.querymaster.Query$QueryCompletedTransition.getPartitionsWithContentsSummary(Query.java:550)
      at org.apache.tajo.querymaster.Query$QueryCompletedTransition.finalizeQuery(Query.java:512)
      at org.apache.tajo.querymaster.Query$QueryCompletedTransition.transition(Query.java:446)
      at org.apache.tajo.querymaster.Query$QueryCompletedTransition.transition(Query.java:435)
      at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
      at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
      at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
      at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
      at org.apache.tajo.querymaster.Query.handle(Query.java:874)
      at org.apache.tajo.querymaster.Query.handle(Query.java:63)
      at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:175)
      at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:108)
      at java.lang.Thread.run(Thread.java:745)
      2015-10-02 18:55:03,958 INFO org.apache.tajo.querymaster.Query: q_1443779192380_0001 Query Transitioned from QUERY_RUNNING to QUERY_ERROR
      2015-10-02 18:55:03,958 INFO org.apache.tajo.querymaster.Query: Processing q_1443779192380_0001 of type DIAGNOSTIC_UPDATE
      2015-10-02 18:55:03,958 INFO org.apache.tajo.querymaster.QueryMasterTask: Query completion notified from q_1443779192380_0001 final state: QUERY_ERROR
      2015-10-02 18:55:03,960 INFO org.apache.tajo.querymaster.QueryMasterTask: Stopping QueryMasterTask:q_1443779192380_0001
      2015-10-02 18:55:03,960 INFO org.apache.tajo.querymaster.QueryMasterTask: Cleanup resources of all workers. Query: q_1443779192380_0001, workers: 1
      2015-10-02 18:55:03,962 INFO org.apache.tajo.querymaster.QueryMasterTask: Stopped QueryMasterTask:q_1443779192380_0001
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                blrunner JaeHwa Jung
                Reporter:
                jhkim Jinho Kim
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: