Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-387

Query is hanging when errors occurs in Query or SubQuery class

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: QueryMaster
    • Labels:
      None

      Description

      Currently, Query is hanging when errors occur in Query or SubQuery. This is a critical bug.

      Primarily, this patch fixes many bugs that cause infinite query hanging. In detail, this patch does as follows:

      • When there is no result file, Query::buildOrUpdateResultTableDesc causes NPE. This patch fixes this bug.
      • A table can be either a file or directory.
        • Before this patch, a table must be a directory.
      • Add ERROR state to Query
      • Allow Query to avoid hanging when errors occur
      • Enable Query to accumulate diagnostic messages.
      • Add ERROR state to SubQuery
      • Allow SubQuery to avoid hanging and retrieve resources when errors occur
      • Enable SubQuery to accumulate diagnostic messages.
      • Enable TajoCli to get proper error messages (e.g., Query Failed! or Internal Error!) when a query is failed or faces internal errors.
      1. TAJO-387.patch
        37 kB
        Hyunsik Choi

        Activity

        Hide
        jhkim Jinho Kim added a comment -

        Could you rebase the patch against the recent revision?

        Show
        jhkim Jinho Kim added a comment - Could you rebase the patch against the recent revision?
        Hide
        jhkim Jinho Kim added a comment -

        ++1

        Sorry, the patch worked well
        This is a very desired feature for me. Ship it!

        Show
        jhkim Jinho Kim added a comment - ++1 Sorry, the patch worked well This is a very desired feature for me. Ship it!
        Hide
        jihoonson Jihoon Son added a comment -

        Wait, I'm reviewing the patch.

        Show
        jihoonson Jihoon Son added a comment - Wait, I'm reviewing the patch.
        Hide
        jihoonson Jihoon Son added a comment -

        +1.
        This patch looks good to me.
        As Jinho said, however, the patch is successfully applied, but it seems to require to be rebased.
        Also, please remove an unused variable, initTime, in QueryStatus.

        Show
        jihoonson Jihoon Son added a comment - +1. This patch looks good to me. As Jinho said, however, the patch is successfully applied, but it seems to require to be rebased. Also, please remove an unused variable, initTime, in QueryStatus.
        Hide
        hyunsik Hyunsik Choi added a comment -

        Thank you for the reviews. I've just committed this patch to master.

        Show
        hyunsik Hyunsik Choi added a comment - Thank you for the reviews. I've just committed this patch to master.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-trunk-postcommit #591 (See https://builds.apache.org/job/Tajo-trunk-postcommit/591/)
        TAJO-387: Query is hanging when errors occurs in Query or SubQuery class. (hyunsik) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=da8cd2984317b61273ccd0d415401873c4eb4d7c)

        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/SubQueryEventType.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/SubQueryDiagnosticsUpdateEvent.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/cli/TajoCli.java
        • tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/AbstractStorageManager.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalEngine.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/QueryStatus.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/Query.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/SubQueryState.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/YarnContainerProxy.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoContainerProxy.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/QueryEventType.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/QueryDiagnosticsUpdateEvent.java
        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/statistics/StatisticsUtil.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoMasterClientService.java
        • tajo-core/tajo-core-backend/src/main/proto/ClientProtos.proto
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryMasterTask.java
          TAJO-387: Query is hanging when errors occurs in Query or SubQuery class. (Added a missed changelog) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=988fff90c9084762666635d558a203c6903f3dc1)
        • CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-trunk-postcommit #591 (See https://builds.apache.org/job/Tajo-trunk-postcommit/591/ ) TAJO-387 : Query is hanging when errors occurs in Query or SubQuery class. (hyunsik) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=da8cd2984317b61273ccd0d415401873c4eb4d7c ) tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/SubQueryEventType.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/SubQueryDiagnosticsUpdateEvent.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/cli/TajoCli.java tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/AbstractStorageManager.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalEngine.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/QueryStatus.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/Query.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/SubQueryState.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/YarnContainerProxy.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoContainerProxy.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/QueryEventType.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/QueryDiagnosticsUpdateEvent.java tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/statistics/StatisticsUtil.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoMasterClientService.java tajo-core/tajo-core-backend/src/main/proto/ClientProtos.proto tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryMasterTask.java TAJO-387 : Query is hanging when errors occurs in Query or SubQuery class. (Added a missed changelog) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=988fff90c9084762666635d558a203c6903f3dc1 ) CHANGES.txt

          People

          • Assignee:
            hyunsik Hyunsik Choi
            Reporter:
            hyunsik Hyunsik Choi
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development