Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1935

Some Tasks don't work after they become TA_ASSIGNED

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.10.1
    • Fix Version/s: 0.12.0, 0.11.1
    • Component/s: QueryMaster
    • Labels:
      None

      Description

      The query is like following :

      insert overwrite into lineitem_tmp
      select
        l_partkey as t_partkey, 0.2 * avg(l_quantity) as t_avg_quantity
      from
        lineitem
      group by l_partkey;
      

      It is from query #17 of TPC-H.

      From eb2, only 2 tasks were not running after TA_ASSIGNED and query was in infinite loop.
      It ran on TPC-H 10TB.

      Corresponding web pages and log are attached.

      1. issue_log.tar.gz
        6.80 MB
        Jongyoung Park
      2. screenshots.zip
        91 kB
        Jongyoung Park

        Activity

        Hide
        jhkim Jinho Kim added a comment -

        Thank you for the reporting.
        Some rpc operation timeout is very short and it is unnecessarily operation. If GC pause are over than 10 second, a task can lost.
        I will remove the future timeout because we already use ping timeout in rpc internal.

        Show
        jhkim Jinho Kim added a comment - Thank you for the reporting. Some rpc operation timeout is very short and it is unnecessarily operation. If GC pause are over than 10 second, a task can lost. I will remove the future timeout because we already use ping timeout in rpc internal.
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user jinossy opened a pull request:

        https://github.com/apache/tajo/pull/864

        TAJO-1935: Some Tasks don't work after they become TA_ASSIGNED.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/jinossy/tajo TAJO-1935

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/864.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #864


        commit 80efb7710e84ad3a3bc2362a4d18654a35c181ee
        Author: Jinho Kim <jhkim@apache.org>
        Date: 2015-11-18T02:29:16Z

        TAJO-1935: Some Tasks don't work after they become TA_ASSIGNED.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user jinossy opened a pull request: https://github.com/apache/tajo/pull/864 TAJO-1935 : Some Tasks don't work after they become TA_ASSIGNED. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jinossy/tajo TAJO-1935 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/864.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #864 commit 80efb7710e84ad3a3bc2362a4d18654a35c181ee Author: Jinho Kim <jhkim@apache.org> Date: 2015-11-18T02:29:16Z TAJO-1935 : Some Tasks don't work after they become TA_ASSIGNED.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/864#issuecomment-157597597

        +1 LGTM!

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/864#issuecomment-157597597 +1 LGTM!
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/864

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/864
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-CODEGEN-build #595 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/595/)
        TAJO-1935: Some Tasks don't work after they become TA_ASSIGNED. (jhkim: rev 4fd1739d491f547757eaa9d26e37a496eca02d92)

        • tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/RpcConstants.java
        • tajo-core/src/main/java/org/apache/tajo/worker/NodeStatusUpdater.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java
        • tajo-core/src/main/java/org/apache/tajo/worker/ExecutionBlockContext.java
        • CHANGES
        • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java
        • tajo-core/src/main/java/org/apache/tajo/worker/TaskManager.java
        • tajo-core/src/main/java/org/apache/tajo/master/QueryInProgress.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #595 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/595/ ) TAJO-1935 : Some Tasks don't work after they become TA_ASSIGNED. (jhkim: rev 4fd1739d491f547757eaa9d26e37a496eca02d92) tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/RpcConstants.java tajo-core/src/main/java/org/apache/tajo/worker/NodeStatusUpdater.java tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java tajo-core/src/main/java/org/apache/tajo/worker/ExecutionBlockContext.java CHANGES tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java tajo-core/src/main/java/org/apache/tajo/worker/TaskManager.java tajo-core/src/main/java/org/apache/tajo/master/QueryInProgress.java
        Hide
        jhkim Jinho Kim added a comment -

        Thanks for your quick review!

        Show
        jhkim Jinho Kim added a comment - Thanks for your quick review!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #979 (See https://builds.apache.org/job/Tajo-master-build/979/)
        TAJO-1935: Some Tasks don't work after they become TA_ASSIGNED. (jhkim: rev 4fd1739d491f547757eaa9d26e37a496eca02d92)

        • tajo-core/src/main/java/org/apache/tajo/master/QueryInProgress.java
        • tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/RpcConstants.java
        • tajo-core/src/main/java/org/apache/tajo/worker/ExecutionBlockContext.java
        • tajo-core/src/main/java/org/apache/tajo/worker/NodeStatusUpdater.java
        • CHANGES
        • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java
        • tajo-core/src/main/java/org/apache/tajo/worker/TaskManager.java
        • tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #979 (See https://builds.apache.org/job/Tajo-master-build/979/ ) TAJO-1935 : Some Tasks don't work after they become TA_ASSIGNED. (jhkim: rev 4fd1739d491f547757eaa9d26e37a496eca02d92) tajo-core/src/main/java/org/apache/tajo/master/QueryInProgress.java tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/RpcConstants.java tajo-core/src/main/java/org/apache/tajo/worker/ExecutionBlockContext.java tajo-core/src/main/java/org/apache/tajo/worker/NodeStatusUpdater.java CHANGES tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java tajo-core/src/main/java/org/apache/tajo/worker/TaskManager.java tajo-core/src/main/java/org/apache/tajo/querymaster/DefaultTaskScheduler.java

          People

          • Assignee:
            jhkim Jinho Kim
            Reporter:
            eminency Jongyoung Park
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development