Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1966

Decrease memory usage of TajoTestingCluster

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0
    • Fix Version/s: 0.12.0, 0.11.1
    • Component/s: Unit Test
    • Labels:
      None

      Description

      Currently, TajoTestingCluster uses high memory. we should decrease memory usage in order to stable testing.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user jinossy opened a pull request:

          https://github.com/apache/tajo/pull/850

          TAJO-1966: Decrease memory usage of TajoTestingCluster

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/jinossy/tajo TAJO-1966

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/tajo/pull/850.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #850


          commit 42c9d93069d58f9d6d8247e93913b958f086e5cc
          Author: Jinho Kim <jhkim@apache.org>
          Date: 2015-11-05T15:35:05Z

          TAJO-1966: Decrease memory usage of TajoTestingCluster


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user jinossy opened a pull request: https://github.com/apache/tajo/pull/850 TAJO-1966 : Decrease memory usage of TajoTestingCluster You can merge this pull request into a Git repository by running: $ git pull https://github.com/jinossy/tajo TAJO-1966 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/850.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #850 commit 42c9d93069d58f9d6d8247e93913b958f086e5cc Author: Jinho Kim <jhkim@apache.org> Date: 2015-11-05T15:35:05Z TAJO-1966 : Decrease memory usage of TajoTestingCluster
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jinossy commented on the pull request:

          https://github.com/apache/tajo/pull/850#issuecomment-154099154

          I've remove memory database immediately for the heap space and change netty allocator to unpooled allocator.
          When I was test, the memory usage is “heap 150~400MB, direct 1~3MB, mapped 0MB”
          https://db.apache.org/derby/docs/10.11/devguide/cdevdvlpinmemdb.html

          Show
          githubbot ASF GitHub Bot added a comment - Github user jinossy commented on the pull request: https://github.com/apache/tajo/pull/850#issuecomment-154099154 I've remove memory database immediately for the heap space and change netty allocator to unpooled allocator. When I was test, the memory usage is “heap 150~400MB, direct 1~3MB, mapped 0MB” https://db.apache.org/derby/docs/10.11/devguide/cdevdvlpinmemdb.html
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on the pull request:

          https://github.com/apache/tajo/pull/850#issuecomment-154271038

          +1, looks good to me.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/850#issuecomment-154271038 +1, looks good to me.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/tajo/pull/850

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/850
          Hide
          jhkim Jinho Kim added a comment -

          Thanks for your quick review!

          Show
          jhkim Jinho Kim added a comment - Thanks for your quick review!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-master-CODEGEN-build #584 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/584/)
          TAJO-1966: Decrease memory usage of TajoTestingCluster. (jhkim: rev 3b0b68375245c5123f9891e0f25bcbe15e9a2a20)

          • tajo-cluster-tests/src/test/java/org/apache/tajo/TajoTestingCluster.java
          • tajo-common/src/main/java/org/apache/tajo/storage/BufferPool.java
          • tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/NettyUtils.java
          • tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/RpcConstants.java
          • tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/DerbyStore.java
          • tajo-cluster-tests/src/test/java/org/apache/tajo/CatalogTestingUtil.java
          • tajo-core/src/main/java/org/apache/tajo/util/JvmPauseMonitor.java
          • tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/NettyServerBase.java
          • tajo-core/src/main/java/org/apache/tajo/worker/Fetcher.java
          • tajo-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java
          • tajo-rpc/tajo-rpc-protobuf/src/main/java/org/apache/tajo/rpc/NettyClientBase.java
          • tajo-project/pom.xml
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TablespaceManager.java
          • CHANGES
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #584 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/584/ ) TAJO-1966 : Decrease memory usage of TajoTestingCluster. (jhkim: rev 3b0b68375245c5123f9891e0f25bcbe15e9a2a20) tajo-cluster-tests/src/test/java/org/apache/tajo/TajoTestingCluster.java tajo-common/src/main/java/org/apache/tajo/storage/BufferPool.java tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/NettyUtils.java tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/RpcConstants.java tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/DerbyStore.java tajo-cluster-tests/src/test/java/org/apache/tajo/CatalogTestingUtil.java tajo-core/src/main/java/org/apache/tajo/util/JvmPauseMonitor.java tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/NettyServerBase.java tajo-core/src/main/java/org/apache/tajo/worker/Fetcher.java tajo-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java tajo-rpc/tajo-rpc-protobuf/src/main/java/org/apache/tajo/rpc/NettyClientBase.java tajo-project/pom.xml tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TablespaceManager.java CHANGES
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-master-build #964 (See https://builds.apache.org/job/Tajo-master-build/964/)
          TAJO-1966: Decrease memory usage of TajoTestingCluster. (jhkim: rev 3b0b68375245c5123f9891e0f25bcbe15e9a2a20)

          • tajo-core/src/main/java/org/apache/tajo/util/JvmPauseMonitor.java
          • tajo-project/pom.xml
          • CHANGES
          • tajo-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java
          • tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/RpcConstants.java
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/DerbyStore.java
          • tajo-cluster-tests/src/test/java/org/apache/tajo/CatalogTestingUtil.java
          • tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/NettyUtils.java
          • tajo-rpc/tajo-rpc-protobuf/src/main/java/org/apache/tajo/rpc/NettyClientBase.java
          • tajo-common/src/main/java/org/apache/tajo/storage/BufferPool.java
          • tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java
          • tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/NettyServerBase.java
          • tajo-core/src/main/java/org/apache/tajo/worker/Fetcher.java
          • tajo-cluster-tests/src/test/java/org/apache/tajo/TajoTestingCluster.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TablespaceManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-build #964 (See https://builds.apache.org/job/Tajo-master-build/964/ ) TAJO-1966 : Decrease memory usage of TajoTestingCluster. (jhkim: rev 3b0b68375245c5123f9891e0f25bcbe15e9a2a20) tajo-core/src/main/java/org/apache/tajo/util/JvmPauseMonitor.java tajo-project/pom.xml CHANGES tajo-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/RpcConstants.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/DerbyStore.java tajo-cluster-tests/src/test/java/org/apache/tajo/CatalogTestingUtil.java tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/NettyUtils.java tajo-rpc/tajo-rpc-protobuf/src/main/java/org/apache/tajo/rpc/NettyClientBase.java tajo-common/src/main/java/org/apache/tajo/storage/BufferPool.java tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java tajo-rpc/tajo-rpc-common/src/main/java/org/apache/tajo/rpc/NettyServerBase.java tajo-core/src/main/java/org/apache/tajo/worker/Fetcher.java tajo-cluster-tests/src/test/java/org/apache/tajo/TajoTestingCluster.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TablespaceManager.java

            People

            • Assignee:
              jhkim Jinho Kim
              Reporter:
              jhkim Jinho Kim
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development