Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-456

Separate tajo-jdbc and tajo-client from tajo-core-backend

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: Build
    • Labels:
      None

      Description

      Currently, tajo-client and tajo-jdbc are included in tajo-core-backend, which depends on lots of third-party libraries. So, even just client programs should include unnecessary third-party libraries. This patch separates tajo-jdbc and tajo-client from tajo-core-backend to individual maven modules. As a result, the client and jdbc's dependencies are more simplified than before.

      After this patch, 'mvn package -Pdist' commands generates tajo jdbc drivers in ${TAJO_HOME}/tajo-dist/target/tajo-${tajo.version}/share/jdbc-dist. There will be the following files in the directory:

      joda-time-2.3.jar
      tajo-catalog-common-0.8.0-SNAPSHOT.jar
      tajo-client-0.8.0-SNAPSHOT.jar
      tajo-common-0.8.0-SNAPSHOT.jar
      tajo-jdbc-0.8.0-SNAPSHOT.jar
      tajo-rpc-0.8.0-SNAPSHOT.jar
      tajo-storage-0.8.0-SNAPSHOT.jar
      

      In order to load the Tajo JDBC driver, client programs must be able to locate the all above JAR files* and hadoop's JAR files.

      For that, users should set classpath for them. If the jar files are located in the directory '/usr/local/share/tajo-jdbc' and hadoop binary is located in '/opt/hadoop', you should set classpath as follows:

      export CLASSPATH=`/opt/hadoop/bin/hadoop classpath`:/usr/local/share/tajo-jdbc/*:$CLASSPATH
      

      Note that the command '${HADOOP_HOME}/bin/hadoop classpath' prints out hadoop's classpaths via stdout.

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-trunk-postcommit #651 (See https://builds.apache.org/job/Tajo-trunk-postcommit/651/)
        TAJO-456: Separate tajo-jdbc and tajo-client from tajo-core-backend. (hyunsik) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=b6a5ff0c39e3fd50fc27c2a3804218fc884135f5)

        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoResultSet.java
        • tajo-common/src/main/java/org/apache/tajo/util/TUtil.java
        • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoDriver.java
        • tajo-catalog/tajo-catalog-client/pom.xml
        • tajo-client/src/main/java/org/apache/tajo/cli/TajoCli.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoDriver.java
        • tajo-core/tajo-core-backend/src/main/proto/QueryMasterClientProtocol.proto
        • tajo-core/pom.xml
        • tajo-client/src/main/proto/ClientProtos.proto
        • tajo-client/src/main/resources/log4j.properties
        • tajo-client/src/main/proto/TajoMasterClientProtocol.proto
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoResultSetBase.java
        • tajo-core/tajo-core-backend/pom.xml
        • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoConnection.java
        • tajo-core/tajo-core-backend/src/main/proto/ClientProtos.proto
        • tajo-core/tajo-core-pullserver/pom.xml
        • pom.xml
        • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoStatement.java
        • tajo-client/src/main/java/org/apache/tajo/jdbc/TajoResultSetBase.java
        • tajo-catalog/tajo-catalog-server/pom.xml
        • tajo-client/src/main/proto/QueryMasterClientProtocol.proto
        • tajo-rpc/pom.xml
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/SQLStates.java
        • tajo-common/src/main/proto/tajo_protos.proto
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/TajoClient.java
        • tajo-common/pom.xml
        • tajo-rpc/src/main/java/org/apache/tajo/rpc/RpcConnectionPool.java
        • CHANGES.txt
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoConnection.java
        • tajo-client/src/main/java/org/apache/tajo/client/QueryStatus.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/MetaDataTuple.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/TajoDump.java
        • tajo-core/tajo-core-storage/pom.xml
        • tajo-client/src/main/java/org/apache/tajo/client/TajoDump.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/ResultSetUtil.java
        • tajo-jdbc/pom.xml
        • tajo-common/src/main/java/org/apache/tajo/storage/Tuple.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoResultSetMetaData.java
        • tajo-client/src/main/java/org/apache/tajo/jdbc/SQLStates.java
        • tajo-client/pom.xml
        • tajo-project/pom.xml
        • tajo-core/tajo-core-backend/src/main/proto/tajo_protos.proto
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoStatement.java
        • tajo-core/tajo-core-backend/src/main/proto/TajoMasterClientProtocol.proto
        • tajo-catalog/tajo-catalog-common/pom.xml
        • tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/storage/Tuple.java
        • tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoPreparedStatement.java
        • tajo-client/src/main/java/org/apache/tajo/jdbc/TajoResultSet.java
        • tajo-client/src/main/java/org/apache/tajo/client/TajoClient.java
        • tajo-client/src/main/java/org/apache/tajo/client/ResultSetUtil.java
        • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/MetaDataTuple.java
        • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoPreparedStatement.java
        • tajo-jdbc/src/main/resources/log4j.properties
        • tajo-dist/pom.xml
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/QueryStatus.java
        • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoDatabaseMetaData.java
        • tajo-client/src/main/java/org/apache/tajo/jdbc/TajoResultSetMetaData.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoMetaDataResultSet.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/cli/TajoCli.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoDatabaseMetaData.java
        • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoMetaDataResultSet.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-trunk-postcommit #651 (See https://builds.apache.org/job/Tajo-trunk-postcommit/651/ ) TAJO-456 : Separate tajo-jdbc and tajo-client from tajo-core-backend. (hyunsik) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=b6a5ff0c39e3fd50fc27c2a3804218fc884135f5 ) tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoResultSet.java tajo-common/src/main/java/org/apache/tajo/util/TUtil.java tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoDriver.java tajo-catalog/tajo-catalog-client/pom.xml tajo-client/src/main/java/org/apache/tajo/cli/TajoCli.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoDriver.java tajo-core/tajo-core-backend/src/main/proto/QueryMasterClientProtocol.proto tajo-core/pom.xml tajo-client/src/main/proto/ClientProtos.proto tajo-client/src/main/resources/log4j.properties tajo-client/src/main/proto/TajoMasterClientProtocol.proto tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoResultSetBase.java tajo-core/tajo-core-backend/pom.xml tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoConnection.java tajo-core/tajo-core-backend/src/main/proto/ClientProtos.proto tajo-core/tajo-core-pullserver/pom.xml pom.xml tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoStatement.java tajo-client/src/main/java/org/apache/tajo/jdbc/TajoResultSetBase.java tajo-catalog/tajo-catalog-server/pom.xml tajo-client/src/main/proto/QueryMasterClientProtocol.proto tajo-rpc/pom.xml tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/SQLStates.java tajo-common/src/main/proto/tajo_protos.proto tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/TajoClient.java tajo-common/pom.xml tajo-rpc/src/main/java/org/apache/tajo/rpc/RpcConnectionPool.java CHANGES.txt tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoConnection.java tajo-client/src/main/java/org/apache/tajo/client/QueryStatus.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/MetaDataTuple.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/TajoDump.java tajo-core/tajo-core-storage/pom.xml tajo-client/src/main/java/org/apache/tajo/client/TajoDump.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/ResultSetUtil.java tajo-jdbc/pom.xml tajo-common/src/main/java/org/apache/tajo/storage/Tuple.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoResultSetMetaData.java tajo-client/src/main/java/org/apache/tajo/jdbc/SQLStates.java tajo-client/pom.xml tajo-project/pom.xml tajo-core/tajo-core-backend/src/main/proto/tajo_protos.proto tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoStatement.java tajo-core/tajo-core-backend/src/main/proto/TajoMasterClientProtocol.proto tajo-catalog/tajo-catalog-common/pom.xml tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/storage/Tuple.java tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoPreparedStatement.java tajo-client/src/main/java/org/apache/tajo/jdbc/TajoResultSet.java tajo-client/src/main/java/org/apache/tajo/client/TajoClient.java tajo-client/src/main/java/org/apache/tajo/client/ResultSetUtil.java tajo-jdbc/src/main/java/org/apache/tajo/jdbc/MetaDataTuple.java tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoPreparedStatement.java tajo-jdbc/src/main/resources/log4j.properties tajo-dist/pom.xml tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/QueryStatus.java tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoDatabaseMetaData.java tajo-client/src/main/java/org/apache/tajo/jdbc/TajoResultSetMetaData.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoMetaDataResultSet.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/cli/TajoCli.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/jdbc/TajoDatabaseMetaData.java tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoMetaDataResultSet.java
        Hide
        hyunsik Hyunsik Choi added a comment -

        committed it to master. Thanks for the review.

        Show
        hyunsik Hyunsik Choi added a comment - committed it to master. Thanks for the review.
        Hide
        jihoonson Jihoon Son added a comment -

        Thanks!

        +1. I successfully compiled using 'git apply', and the patch passed 'mvn verify'.
        Ship it!

        Show
        jihoonson Jihoon Son added a comment - Thanks! +1. I successfully compiled using 'git apply', and the patch passed 'mvn verify'. Ship it!
        Hide
        hyunsik Hyunsik Choi added a comment -

        Could you try to use 'git apply' instead of 'patch' command?

        post-review, which is a reviewboard command line tool, uses 'git --no-renames' option that not allow git to copy the renamed files. But, the patch generated in such a way may not work in patch. Thanks!

        Show
        hyunsik Hyunsik Choi added a comment - Could you try to use 'git apply' instead of 'patch' command? post-review, which is a reviewboard command line tool, uses 'git --no-renames' option that not allow git to copy the renamed files. But, the patch generated in such a way may not work in patch. Thanks!
        Hide
        jihoonson Jihoon Son added a comment -

        I met a compile error as follows.

        src/main/proto/tajo_protos.proto: No such file or directory
        
        Show
        jihoonson Jihoon Son added a comment - I met a compile error as follows. src/main/proto/tajo_protos.proto: No such file or directory
        Hide
        jihoonson Jihoon Son added a comment -

        I'm reviewing the patch.

        Show
        jihoonson Jihoon Son added a comment - I'm reviewing the patch.
        Hide
        hyunsik Hyunsik Choi added a comment -

        I submitted the patch to the reviewboard.
        https://reviews.apache.org/r/16494

        Show
        hyunsik Hyunsik Choi added a comment - I submitted the patch to the reviewboard. https://reviews.apache.org/r/16494

          People

          • Assignee:
            hyunsik Hyunsik Choi
            Reporter:
            hyunsik Hyunsik Choi
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development