Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1696

Resource calculator should consider the requested disk resource at the first stage

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0
    • Component/s: Resource Manager
    • Labels:
      None

      Description

      Resource calculator currently considers only cpu and memory. However, disk is an important resource in the first stage, so should be considered for available resource calculation.

      In addition, in the current implementation, a worker sends a heartbeat whenever it finishes processing half of the assigned tasks. So, the problem of ignoring disk resource makes the first heartbeat delayed during the first stage, thereby making the query execution slow.

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user jihoonson opened a pull request:

        https://github.com/apache/tajo/pull/640

        TAJO-1696: Resource calculator should consider the requested disk resource at the first stage

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/jihoonson/tajo-2 TAJO-1696

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/640.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #640


        commit e231945aebb7c3302fa8b4af4934da21de46a999
        Author: Jihoon Son <jihoonson@apache.org>
        Date: 2015-07-22T02:47:26Z

        TAJO-1696


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user jihoonson opened a pull request: https://github.com/apache/tajo/pull/640 TAJO-1696 : Resource calculator should consider the requested disk resource at the first stage You can merge this pull request into a Git repository by running: $ git pull https://github.com/jihoonson/tajo-2 TAJO-1696 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/640.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #640 commit e231945aebb7c3302fa8b4af4934da21de46a999 Author: Jihoon Son <jihoonson@apache.org> Date: 2015-07-22T02:47:26Z TAJO-1696
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jinossy commented on the pull request:

        https://github.com/apache/tajo/pull/640#issuecomment-123558713

        +1
        Great! @jihoonson

        Show
        githubbot ASF GitHub Bot added a comment - Github user jinossy commented on the pull request: https://github.com/apache/tajo/pull/640#issuecomment-123558713 +1 Great! @jihoonson
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/640

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/640
        Hide
        hudson Hudson added a comment -

        ABORTED: Integrated in Tajo-master-CODEGEN-build #400 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/400/)
        TAJO-1696: Resource calculator should consider the requested disk resource at the first stage. (jihoon) (jihoonson: rev 95f708ac9e0f723780fac259110ec8a737ecd0b9)

        • CHANGES
        • tajo-core/src/main/java/org/apache/tajo/resource/DefaultResourceCalculator.java
        Show
        hudson Hudson added a comment - ABORTED: Integrated in Tajo-master-CODEGEN-build #400 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/400/ ) TAJO-1696 : Resource calculator should consider the requested disk resource at the first stage. (jihoon) (jihoonson: rev 95f708ac9e0f723780fac259110ec8a737ecd0b9) CHANGES tajo-core/src/main/java/org/apache/tajo/resource/DefaultResourceCalculator.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #760 (See https://builds.apache.org/job/Tajo-master-build/760/)
        TAJO-1696: Resource calculator should consider the requested disk resource at the first stage. (jihoon) (jihoonson: rev 95f708ac9e0f723780fac259110ec8a737ecd0b9)

        • tajo-core/src/main/java/org/apache/tajo/resource/DefaultResourceCalculator.java
        • CHANGES
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #760 (See https://builds.apache.org/job/Tajo-master-build/760/ ) TAJO-1696 : Resource calculator should consider the requested disk resource at the first stage. (jihoon) (jihoonson: rev 95f708ac9e0f723780fac259110ec8a737ecd0b9) tajo-core/src/main/java/org/apache/tajo/resource/DefaultResourceCalculator.java CHANGES

          People

          • Assignee:
            jihoonson Jihoon Son
            Reporter:
            jihoonson Jihoon Son
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development