Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-261

Rearrange default port numbers and config names.

    Details

      Description

      Tajo uses many port numbers and config properties. But, they are inconsistent. The main objective of this issue is to rearrange the default port numbers and config properties. Also, we should make an wiki page to describe them.

      1. TAJO-261_2.patch
        157 kB
        Hyunsik Choi
      2. TAJO-261_3.patch
        162 kB
        Hyunsik Choi
      3. TAJO-261.patch
        112 kB
        Hyunsik Choi

        Activity

        Hide
        hyunsik Hyunsik Choi added a comment -

        This patch rearranges a number of default address and ports of RPC and HTTP services. It also rearranges config properties names and their purpose.

        I've written the default listen address and its config properties at the wiki page (https://wiki.apache.org/tajo/Configuration#DefaultPortNumbers). I will also describe the other updated configs on the wiki page as soon as possible.

        Show
        hyunsik Hyunsik Choi added a comment - This patch rearranges a number of default address and ports of RPC and HTTP services. It also rearranges config properties names and their purpose. I've written the default listen address and its config properties at the wiki page ( https://wiki.apache.org/tajo/Configuration#DefaultPortNumbers ). I will also describe the other updated configs on the wiki page as soon as possible.
        Hide
        jhkim Jinho Kim added a comment -

        Hyunsik,

        Great work!! but I was getting an error.

        Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.004 sec <<< FAILURE!
        test(org.apache.tajo.storage.TestRowFile)  Time elapsed: 0.004 sec  <<< ERROR!
        java.lang.IllegalArgumentException: Can not create a Path from a null string
                at org.apache.hadoop.fs.Path.checkPathArg(Path.java:87)
                at org.apache.hadoop.fs.Path.<init>(Path.java:99)
                at org.apache.tajo.storage.TestRowFile.test(TestRowFile.java:73)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                at java.lang.reflect.Method.invoke(Method.java:597)
                at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
                at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
                at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
                at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
                at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
                at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
                at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
                at o
        
        Show
        jhkim Jinho Kim added a comment - Hyunsik, Great work!! but I was getting an error. Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.004 sec <<< FAILURE! test(org.apache.tajo.storage.TestRowFile) Time elapsed: 0.004 sec <<< ERROR! java.lang.IllegalArgumentException: Can not create a Path from a null string at org.apache.hadoop.fs.Path.checkPathArg(Path.java:87) at org.apache.hadoop.fs.Path.<init>(Path.java:99) at org.apache.tajo.storage.TestRowFile.test(TestRowFile.java:73) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at o
        Hide
        hyunsik Hyunsik Choi added a comment -

        Thank you for your comment. I'm digging this bug.

        Show
        hyunsik Hyunsik Choi added a comment - Thank you for your comment. I'm digging this bug.
        Hide
        hyunsik Hyunsik Choi added a comment -

        I'm uploading the second patch. This fixes the bug mentioned above.

        Show
        hyunsik Hyunsik Choi added a comment - I'm uploading the second patch. This fixes the bug mentioned above.
        Hide
        hyunsik Hyunsik Choi added a comment -
        Show
        hyunsik Hyunsik Choi added a comment - Regarding this update, I've updated the following wiki pages: https://wiki.apache.org/tajo/GettingStarted https://wiki.apache.org/tajo/tsql https://wiki.apache.org/tajo/Configuration
        Hide
        hyunsik Hyunsik Choi added a comment -

        This patch additionally separates TAJO_HEAPSIZE variable in tajo-env-sh into TAJO_MASTER_HEAPSIZE and TAJO_WORKER_HEAPSIZE. It would be useful for increasing workers' heap independently.

        Show
        hyunsik Hyunsik Choi added a comment - This patch additionally separates TAJO_HEAPSIZE variable in tajo-env-sh into TAJO_MASTER_HEAPSIZE and TAJO_WORKER_HEAPSIZE. It would be useful for increasing workers' heap independently.
        Hide
        jhkim Jinho Kim added a comment -

        +1
        Sorry for the late response
        Looks great for me. I've verified 'mvn clean install'.

        Show
        jhkim Jinho Kim added a comment - +1 Sorry for the late response Looks great for me. I've verified 'mvn clean install'.
        Hide
        hyunsik Hyunsik Choi added a comment -

        Thank you for the review. I've just committed it to master and branch-0.2.

        Show
        hyunsik Hyunsik Choi added a comment - Thank you for the review. I've just committed it to master and branch-0.2.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-trunk-postcommit #523 (See https://builds.apache.org/job/Tajo-trunk-postcommit/523/)
        TAJO-261: Rearrange default port numbers and config names. (hyunsik) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=d51283284688dfa25550cfe773dfc52391d6b266)

        • tajo-core/tajo-core-backend/src/main/resources/webapps/admin/cluster.jsp
        • tajo-core/tajo-core-storage/src/test/java/org/apache/tajo/storage/v2/TestStorages.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/LogicalPlan.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/TajoClient.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/webapp/StaticHttpServer.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryMasterTask.java
        • tajo-core/tajo-core-backend/src/main/resources/tajo-default.xml
        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/CatalogConstants.java
        • tajo-core/tajo-core-backend/src/main/resources/catalog-default.xml
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/YarnTaskRunnerLauncherImpl.java
        • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/client/TestTajoClient.java
        • tajo-catalog/tajo-catalog-common/src/main/resources/catalog-default.xml
        • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/LocalTajoTestingUtility.java
        • tajo-rpc/src/main/java/org/apache/tajo/util/NetUtils.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalEngine.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TaskRunnerManager.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryMaster.java
        • tajo-core/tajo-core-backend/src/main/resources/webapps/admin/query.jsp
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/TajoDump.java
        • tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/pullserver/PullServerAuxService.java
        • tajo-core/tajo-core-backend/src/test/resources/catalog-default.xml
        • tajo-core/tajo-core-backend/src/main/proto/TajoMasterClientProtocol.proto
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java
        • tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/StorageManagerFactory.java
        • tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java
        • tajo-core/tajo-core-backend/src/main/proto/ClientProtos.proto
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/util/JSPUtil.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoWorker.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoWorkerManagerService.java
        • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/storage/TestRowFile.java
        • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/BackendTestingUtil.java
        • tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/v2/RCFile.java
        • tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/AbstractStorageManager.java
        • tajo-catalog/tajo-catalog-server/src/test/resources/catalog-default.xml
        • tajo-common/src/main/java/org/apache/tajo/util/NetUtils.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/benchmark/BenchmarkSet.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoResourceAllocator.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/YarnContainerProxy.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoMasterService.java
        • CHANGES.txt
        • tajo-dist/src/main/bin/tajo
        • tajo-dist/src/main/conf/tajo-env.sh
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java
        • tajo-core/tajo-core-storage/src/test/java/org/apache/tajo/storage/v2/TestCSVScanner.java
        • tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
        • tajo-core/tajo-core-storage/src/test/java/org/apache/tajo/storage/v2/TestCSVCompression.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TaskRunner.java
        • tajo-core/tajo-core-backend/src/main/proto/ClientProtocol.proto
        • tajo-common/src/main/java/org/apache/tajo/TajoConstants.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoMasterClientService.java
        • tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/v2/DiskFileScanScheduler.java
        • tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/v2/ScanScheduler.java
        • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/TajoTestingCluster.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoMaster.java
        • tajo-core/tajo-core-backend/src/test/resources/tajo-default.xml
        • tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/v2/StorageManagerV2.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/cli/TajoCli.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/benchmark/TPCH.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/Task.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryUnitAttempt.java
        • tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/v2/RCFileScanner.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/YarnResourceAllocator.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/ExternalSortExec.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/WorkerResource.java
        • tajo-core/tajo-core-backend/src/main/resources/webapps/admin/index.jsp
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryJobManager.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java
        • tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/RawFile.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/SQLStates.java
        • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/MiniTajoYarnCluster.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-trunk-postcommit #523 (See https://builds.apache.org/job/Tajo-trunk-postcommit/523/ ) TAJO-261 : Rearrange default port numbers and config names. (hyunsik) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=d51283284688dfa25550cfe773dfc52391d6b266 ) tajo-core/tajo-core-backend/src/main/resources/webapps/admin/cluster.jsp tajo-core/tajo-core-storage/src/test/java/org/apache/tajo/storage/v2/TestStorages.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/LogicalPlan.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/TajoClient.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/webapp/StaticHttpServer.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryMasterTask.java tajo-core/tajo-core-backend/src/main/resources/tajo-default.xml tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/CatalogConstants.java tajo-core/tajo-core-backend/src/main/resources/catalog-default.xml tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/YarnTaskRunnerLauncherImpl.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/client/TestTajoClient.java tajo-catalog/tajo-catalog-common/src/main/resources/catalog-default.xml tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/LocalTajoTestingUtility.java tajo-rpc/src/main/java/org/apache/tajo/util/NetUtils.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalEngine.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TaskRunnerManager.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryMaster.java tajo-core/tajo-core-backend/src/main/resources/webapps/admin/query.jsp tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/TajoDump.java tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/pullserver/PullServerAuxService.java tajo-core/tajo-core-backend/src/test/resources/catalog-default.xml tajo-core/tajo-core-backend/src/main/proto/TajoMasterClientProtocol.proto tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/StorageManagerFactory.java tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java tajo-core/tajo-core-backend/src/main/proto/ClientProtos.proto tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/util/JSPUtil.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoWorker.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoWorkerManagerService.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/storage/TestRowFile.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/BackendTestingUtil.java tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/v2/RCFile.java tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/AbstractStorageManager.java tajo-catalog/tajo-catalog-server/src/test/resources/catalog-default.xml tajo-common/src/main/java/org/apache/tajo/util/NetUtils.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/benchmark/BenchmarkSet.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoResourceAllocator.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/YarnContainerProxy.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoMasterService.java CHANGES.txt tajo-dist/src/main/bin/tajo tajo-dist/src/main/conf/tajo-env.sh tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java tajo-core/tajo-core-storage/src/test/java/org/apache/tajo/storage/v2/TestCSVScanner.java tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java tajo-core/tajo-core-storage/src/test/java/org/apache/tajo/storage/v2/TestCSVCompression.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TaskRunner.java tajo-core/tajo-core-backend/src/main/proto/ClientProtocol.proto tajo-common/src/main/java/org/apache/tajo/TajoConstants.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoMasterClientService.java tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/v2/DiskFileScanScheduler.java tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/v2/ScanScheduler.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/TajoTestingCluster.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoMaster.java tajo-core/tajo-core-backend/src/test/resources/tajo-default.xml tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/v2/StorageManagerV2.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/cli/TajoCli.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/benchmark/TPCH.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/Task.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryUnitAttempt.java tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/v2/RCFileScanner.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/YarnResourceAllocator.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/ExternalSortExec.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/WorkerResource.java tajo-core/tajo-core-backend/src/main/resources/webapps/admin/index.jsp tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryJobManager.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java tajo-core/tajo-core-storage/src/main/java/org/apache/tajo/storage/RawFile.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/SQLStates.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/MiniTajoYarnCluster.java

          People

          • Assignee:
            hyunsik Hyunsik Choi
            Reporter:
            hyunsik Hyunsik Choi
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development