Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0
    • Component/s: Storage
    • Labels:
      None

      Description

      The main objective of this issue is to implement JDBC-based storage and its Tablespace implementation.

      1. TAJO-1730_3.patch
        268 kB
        Hyunsik Choi
      2. TAJO-1730_2.patch
        268 kB
        Hyunsik Choi
      3. TAJO-1730.patch
        268 kB
        Hyunsik Choi

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-0.11.0-build #54 (See https://builds.apache.org/job/Tajo-0.11.0-build/54/)
          TAJO-1730: JDBC Tablespace support. (missed version change) (hyunsik: rev 332da7e1f788476a45771a3ec5f6b5d9401dbaf2)

          • tajo-storage/tajo-storage-pgsql/pom.xml
          • tajo-storage/tajo-storage-jdbc/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-0.11.0-build #54 (See https://builds.apache.org/job/Tajo-0.11.0-build/54/ ) TAJO-1730 : JDBC Tablespace support. (missed version change) (hyunsik: rev 332da7e1f788476a45771a3ec5f6b5d9401dbaf2) tajo-storage/tajo-storage-pgsql/pom.xml tajo-storage/tajo-storage-jdbc/pom.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-master-build #876 (See https://builds.apache.org/job/Tajo-master-build/876/)
          TAJO-1730: JDBC Tablespace support. (hyunsik: rev 0e4ad5635be05bdde8dac5feb6c9a85fb3cf0e06)

          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q2_Part_MixedStorage.sql
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestFileTablespace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q3.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/datetime_types.txt
          • tajo-cluster-tests/src/test/java/org/apache/tajo/TpchTestBase.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/fragment/FragmentConvertor.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q5.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/nation.sql
          • tajo-client/src/main/java/org/apache/tajo/client/CatalogAdminClientImpl.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/AbstractScanner.java
          • tajo-common/src/main/java/org/apache/tajo/util/UriUtil.java
          • tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst
          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileTablespace.java
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLQueryTests.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCtasToHdfs.result
          • tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLMetadataProvider.java
          • tajo-cluster-tests/src/test/java/org/apache/tajo/TajoTestingCluster.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q3.result
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testVariableLengthFields.result
          • tajo-common/src/main/java/org/apache/tajo/util/FileUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q5.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q1.sql
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/OldStorageManager.java
          • tajo-dist/pom.xml
          • tajo-common/src/main/java/org/apache/tajo/exception/ExceptionUtil.java
          • tajo-common/src/main/java/org/apache/tajo/util/JavaResourceUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testInPredicateWithNumbers.result
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestJoinQuery.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestDelimitedTextFile.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testSimpleFilter.sql
          • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLSimpleQueryTests/testSelectAll.sql
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/PgSQLTestServer.java
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLSimpleQueryTests.java
          • tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLJdbcScanner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCaseWhenFilter.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testQueryWithConnProperties.sql
          • tajo-common/src/test/java/org/apache/tajo/util/TestFileUtils.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testDateTimeTypes.sql
          • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testInPredicateWithLiterals.result
          • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseTablespace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testInPredicateWithNumbers.sql
          • tajo-docs/src/main/sphinx/index.rst
          • tajo-core/src/main/java/org/apache/tajo/querymaster/Repartitioner.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLSimpleQueryTests/testSelectLimit.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/region.sql
          • tajo-storage/tajo-storage-pgsql/pom.xml
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCaseWhenFilter.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/datetime_types.sql
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestLineReader.java
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/LinkedMetadataManager.java
          • tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultFileScanner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testProjectedColumns.result
          • tajo-storage/tajo-storage-common/src/main/resources/storage-default.xml
          • tajo-plan/src/main/proto/Plan.proto
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLSimpleQueryTests/testSelectLimit.sql
          • tajo-storage/pom.xml
          • tajo-catalog/tajo-catalog-common/src/test/java/org/apache/tajo/catalog/TestFunctionDesc.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testFunctionWithinFilter.sql
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcTablespace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/orders.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q2_Part.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testFunctionWithinFilter.result
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageProperty.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/supplier.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/exprrewrite/rules/ConstantPropagation.java
          • tajo-storage/tajo-storage-common/src/main/resources/storage-default.json
          • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseScanner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testSimpleFilter.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/lineitem.sql
          • tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLMetadataProvider.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/expr/EvalType.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q2_Part_MixedStorage.result
          • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testFixedLengthFields.sql
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/SQLBuilder.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testBetweenDates.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testBetweenNumbers.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testBetweenDates.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/part.sql
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLEndPointTests.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/serder/EvalNodeSerializer.java
          • tajo-common/src/main/java/org/apache/tajo/storage/StorageService.java
          • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestLinkedMetadataManager.java
          • tajo-core-tests/src/test/java/org/apache/tajo/storage/TestRowFile.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Tablespace.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/customer.sql
          • CHANGES
          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileScanner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testProjectedColumns.sql
          • tajo-docs/src/main/sphinx/storage_plugins/overview.rst
          • tajo-docs/src/main/sphinx/storage_plugins.rst
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/SQLExpressionGenerator.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testQueryWithConnProperties.result
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TablespaceManager.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testVariableLengthFields.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/dataset/.marker
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/partsupp.sql
          • tajo-core/src/main/java/org/apache/tajo/master/exec/QueryExecutor.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testBetweenNumbers.result
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Scanner.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/json/TestJsonSerDe.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testInPredicateWithLiterals.sql
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java
          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/avro/TestAvroUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCountAsterisk.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLSimpleQueryTests/testSelectAll.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/exprrewrite/rules/ConstantFolding.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q1.result
          • tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java
          • tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLTablespace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q2_Part.sql
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/NullScanner.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/expr/SimpleEvalNodeVisitor.java
          • tajo-storage/tajo-storage-jdbc/src/main/test/java/org/apache/tajo/storage/jdbc/TestConnectionInfo.java
          • tajo-storage/tajo-storage-jdbc/pom.xml
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testDateTimeTypes.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/expr/AlgebraicUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testFixedLengthFields.result
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcMetadataProviderBase.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCtasToHdfs.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/logical/ScanNode.java
          • tajo-core-tests/src/test/java/org/apache/tajo/cli/tools/TestDDLBuilder.java
          • tajo-storage/tajo-storage-common/src/test/resources/storage-default.xml
          • tajo-storage/tajo-storage-jdbc/src/main/proto/JdbcFragmentProtos.proto
          • tajo-project/pom.xml
          • tajo-core-tests/src/test/java/org/apache/tajo/parser/sql/TestSQLAnalyzer.java
          • tajo-common/src/main/java/org/apache/tajo/exception/NotImplementedException.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/CommonConditionReduceRule.java
          • tajo-common/src/main/java/org/apache/tajo/exception/UnsupportedException.java
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcFragment.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java
          • tajo-docs/src/main/sphinx/storage_plugin.rst
          • tajo-core/src/main/java/org/apache/tajo/engine/codegen/EvalCodeGenerator.java
          • tajo-common/src/test/java/org/apache/tajo/util/TestUriUtil.java
          • tajo-storage/tajo-storage-hbase/src/test/java/org/apache/tajo/storage/hbase/TestHBaseTableSpace.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestHBaseTable.java
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLJdbcTableSpace.java
          • tajo-common/src/main/java/org/apache/tajo/util/StringUtils.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCountAsterisk.sql
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-build #876 (See https://builds.apache.org/job/Tajo-master-build/876/ ) TAJO-1730 : JDBC Tablespace support. (hyunsik: rev 0e4ad5635be05bdde8dac5feb6c9a85fb3cf0e06) tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q2_Part_MixedStorage.sql tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestFileTablespace.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q3.sql tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/datetime_types.txt tajo-cluster-tests/src/test/java/org/apache/tajo/TpchTestBase.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/fragment/FragmentConvertor.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q5.result tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/nation.sql tajo-client/src/main/java/org/apache/tajo/client/CatalogAdminClientImpl.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/AbstractScanner.java tajo-common/src/main/java/org/apache/tajo/util/UriUtil.java tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileTablespace.java tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLQueryTests.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCtasToHdfs.result tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLMetadataProvider.java tajo-cluster-tests/src/test/java/org/apache/tajo/TajoTestingCluster.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q3.result tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testVariableLengthFields.result tajo-common/src/main/java/org/apache/tajo/util/FileUtil.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q5.sql tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q1.sql tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/OldStorageManager.java tajo-dist/pom.xml tajo-common/src/main/java/org/apache/tajo/exception/ExceptionUtil.java tajo-common/src/main/java/org/apache/tajo/util/JavaResourceUtil.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testInPredicateWithNumbers.result tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestJoinQuery.java tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestDelimitedTextFile.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testSimpleFilter.sql tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableUtil.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLSimpleQueryTests/testSelectAll.sql tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/PgSQLTestServer.java tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLSimpleQueryTests.java tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLJdbcScanner.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCaseWhenFilter.result tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testQueryWithConnProperties.sql tajo-common/src/test/java/org/apache/tajo/util/TestFileUtils.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testDateTimeTypes.sql tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testInPredicateWithLiterals.result tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseTablespace.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testInPredicateWithNumbers.sql tajo-docs/src/main/sphinx/index.rst tajo-core/src/main/java/org/apache/tajo/querymaster/Repartitioner.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLSimpleQueryTests/testSelectLimit.result tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/region.sql tajo-storage/tajo-storage-pgsql/pom.xml tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCaseWhenFilter.sql tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/datetime_types.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestLineReader.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/LinkedMetadataManager.java tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultFileScanner.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testProjectedColumns.result tajo-storage/tajo-storage-common/src/main/resources/storage-default.xml tajo-plan/src/main/proto/Plan.proto tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLSimpleQueryTests/testSelectLimit.sql tajo-storage/pom.xml tajo-catalog/tajo-catalog-common/src/test/java/org/apache/tajo/catalog/TestFunctionDesc.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testFunctionWithinFilter.sql tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcTablespace.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/orders.sql tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q2_Part.result tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testFunctionWithinFilter.result tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageProperty.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/supplier.sql tajo-plan/src/main/java/org/apache/tajo/plan/exprrewrite/rules/ConstantPropagation.java tajo-storage/tajo-storage-common/src/main/resources/storage-default.json tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseScanner.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testSimpleFilter.result tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/lineitem.sql tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLMetadataProvider.java tajo-plan/src/main/java/org/apache/tajo/plan/expr/EvalType.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q2_Part_MixedStorage.result tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testFixedLengthFields.sql tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/SQLBuilder.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testBetweenDates.result tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testBetweenNumbers.sql tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testBetweenDates.sql tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/part.sql tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLEndPointTests.java tajo-plan/src/main/java/org/apache/tajo/plan/serder/EvalNodeSerializer.java tajo-common/src/main/java/org/apache/tajo/storage/StorageService.java tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestLinkedMetadataManager.java tajo-core-tests/src/test/java/org/apache/tajo/storage/TestRowFile.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Tablespace.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/customer.sql CHANGES tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileScanner.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testProjectedColumns.sql tajo-docs/src/main/sphinx/storage_plugins/overview.rst tajo-docs/src/main/sphinx/storage_plugins.rst tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/SQLExpressionGenerator.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testQueryWithConnProperties.result tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TablespaceManager.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testVariableLengthFields.sql tajo-storage/tajo-storage-pgsql/src/test/resources/dataset/.marker tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/partsupp.sql tajo-core/src/main/java/org/apache/tajo/master/exec/QueryExecutor.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testBetweenNumbers.result tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Scanner.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/json/TestJsonSerDe.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testInPredicateWithLiterals.sql tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/avro/TestAvroUtil.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCountAsterisk.result tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLSimpleQueryTests/testSelectAll.result tajo-plan/src/main/java/org/apache/tajo/plan/exprrewrite/rules/ConstantFolding.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q1.result tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLTablespace.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q2_Part.sql tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/NullScanner.java tajo-plan/src/main/java/org/apache/tajo/plan/expr/SimpleEvalNodeVisitor.java tajo-storage/tajo-storage-jdbc/src/main/test/java/org/apache/tajo/storage/jdbc/TestConnectionInfo.java tajo-storage/tajo-storage-jdbc/pom.xml tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testDateTimeTypes.result tajo-plan/src/main/java/org/apache/tajo/plan/expr/AlgebraicUtil.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testFixedLengthFields.result tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcMetadataProviderBase.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCtasToHdfs.sql tajo-plan/src/main/java/org/apache/tajo/plan/logical/ScanNode.java tajo-core-tests/src/test/java/org/apache/tajo/cli/tools/TestDDLBuilder.java tajo-storage/tajo-storage-common/src/test/resources/storage-default.xml tajo-storage/tajo-storage-jdbc/src/main/proto/JdbcFragmentProtos.proto tajo-project/pom.xml tajo-core-tests/src/test/java/org/apache/tajo/parser/sql/TestSQLAnalyzer.java tajo-common/src/main/java/org/apache/tajo/exception/NotImplementedException.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/CommonConditionReduceRule.java tajo-common/src/main/java/org/apache/tajo/exception/UnsupportedException.java tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcFragment.java tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java tajo-docs/src/main/sphinx/storage_plugin.rst tajo-core/src/main/java/org/apache/tajo/engine/codegen/EvalCodeGenerator.java tajo-common/src/test/java/org/apache/tajo/util/TestUriUtil.java tajo-storage/tajo-storage-hbase/src/test/java/org/apache/tajo/storage/hbase/TestHBaseTableSpace.java tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestHBaseTable.java tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLJdbcTableSpace.java tajo-common/src/main/java/org/apache/tajo/util/StringUtils.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCountAsterisk.sql
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-0.11.0-build #53 (See https://builds.apache.org/job/Tajo-0.11.0-build/53/)
          TAJO-1730: JDBC Tablespace support. (hyunsik: rev 144a02e342d256f3c58a2f69356ea08c6e43ac75)

          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/SQLBuilder.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q2_Part_MixedStorage.sql
          • tajo-core/src/main/java/org/apache/tajo/querymaster/Repartitioner.java
          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileScanner.java
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/SQLExpressionGenerator.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/datetime_types.sql
          • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCountAsterisk.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/CommonConditionReduceRule.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/orders.sql
          • tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLEndPointTests.java
          • tajo-common/src/main/java/org/apache/tajo/exception/UnsupportedException.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q2_Part.result
          • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseScanner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testQueryWithConnProperties.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testBetweenNumbers.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLSimpleQueryTests/testSelectAll.sql
          • tajo-storage/tajo-storage-common/src/test/resources/storage-default.xml
          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileTablespace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCountAsterisk.sql
          • tajo-common/src/main/java/org/apache/tajo/exception/NotImplementedException.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/expr/AlgebraicUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q3.sql
          • tajo-common/src/main/java/org/apache/tajo/util/StringUtils.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/AbstractScanner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testInPredicateWithNumbers.result
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcMetadataProviderBase.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java
          • tajo-plan/src/main/proto/Plan.proto
          • tajo-storage/tajo-storage-jdbc/pom.xml
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q5.result
          • tajo-storage/tajo-storage-jdbc/src/main/proto/JdbcFragmentProtos.proto
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testFixedLengthFields.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testDateTimeTypes.sql
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestJoinQuery.java
          • tajo-common/src/test/java/org/apache/tajo/util/TestFileUtils.java
          • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseTablespace.java
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/PgSQLTestServer.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/nation.sql
          • tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCtasToHdfs.result
          • tajo-core-tests/src/test/java/org/apache/tajo/cli/tools/TestDDLBuilder.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q2_Part.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testInPredicateWithLiterals.sql
          • tajo-storage/pom.xml
          • tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLTablespace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testBetweenDates.result
          • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestLinkedMetadataManager.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testFixedLengthFields.sql
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcTablespace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/lineitem.sql
          • tajo-common/src/main/java/org/apache/tajo/util/JavaResourceUtil.java
          • tajo-common/src/main/java/org/apache/tajo/util/UriUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testProjectedColumns.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/logical/ScanNode.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLSimpleQueryTests/testSelectLimit.result
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Scanner.java
          • tajo-docs/src/main/sphinx/storage_plugins.rst
          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/expr/EvalType.java
          • tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultFileScanner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/datetime_types.txt
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCaseWhenFilter.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/serder/EvalNodeSerializer.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testBetweenDates.sql
          • tajo-catalog/tajo-catalog-common/src/test/java/org/apache/tajo/catalog/TestFunctionDesc.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q3.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/exprrewrite/rules/ConstantFolding.java
          • tajo-project/pom.xml
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLQueryTests.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/customer.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/dataset/.marker
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/LinkedMetadataManager.java
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcFragment.java
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLSimpleQueryTests.java
          • tajo-dist/pom.xml
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testVariableLengthFields.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testInPredicateWithLiterals.result
          • tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLMetadataProvider.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCaseWhenFilter.sql
          • tajo-common/src/main/java/org/apache/tajo/exception/ExceptionUtil.java
          • tajo-client/src/main/java/org/apache/tajo/client/CatalogAdminClientImpl.java
          • tajo-core/src/main/java/org/apache/tajo/master/exec/QueryExecutor.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testDateTimeTypes.result
          • tajo-core-tests/src/test/java/org/apache/tajo/storage/TestRowFile.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLSimpleQueryTests/testSelectLimit.sql
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/fragment/FragmentConvertor.java
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLMetadataProvider.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testVariableLengthFields.sql
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestLineReader.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q1.sql
          • tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLJdbcScanner.java
          • tajo-cluster-tests/src/test/java/org/apache/tajo/TpchTestBase.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q5.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/exprrewrite/rules/ConstantPropagation.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageProperty.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TablespaceManager.java
          • tajo-docs/src/main/sphinx/storage_plugin.rst
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestDelimitedTextFile.java
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java
          • tajo-storage/tajo-storage-pgsql/pom.xml
          • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java
          • tajo-docs/src/main/sphinx/storage_plugins/overview.rst
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/OldStorageManager.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCtasToHdfs.sql
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/expr/SimpleEvalNodeVisitor.java
          • tajo-core-tests/src/test/java/org/apache/tajo/parser/sql/TestSQLAnalyzer.java
          • tajo-common/src/main/java/org/apache/tajo/storage/StorageService.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testSimpleFilter.result
          • tajo-storage/tajo-storage-common/src/main/resources/storage-default.xml
          • tajo-storage/tajo-storage-jdbc/src/main/test/java/org/apache/tajo/storage/jdbc/TestConnectionInfo.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/supplier.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q2_Part_MixedStorage.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLSimpleQueryTests/testSelectAll.result
          • tajo-common/src/test/java/org/apache/tajo/util/TestUriUtil.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestFileTablespace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/region.sql
          • tajo-cluster-tests/src/test/java/org/apache/tajo/TajoTestingCluster.java
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testSimpleFilter.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q1.result
          • tajo-storage/tajo-storage-hbase/src/test/java/org/apache/tajo/storage/hbase/TestHBaseTableSpace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testQueryWithConnProperties.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testBetweenNumbers.sql
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestHBaseTable.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testInPredicateWithNumbers.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/part.sql
          • tajo-core/src/main/java/org/apache/tajo/engine/codegen/EvalCodeGenerator.java
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLJdbcTableSpace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testFunctionWithinFilter.result
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/NullScanner.java
          • CHANGES
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/json/TestJsonSerDe.java
          • tajo-common/src/main/java/org/apache/tajo/util/FileUtil.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/avro/TestAvroUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testProjectedColumns.result
          • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testFunctionWithinFilter.sql
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Tablespace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/partsupp.sql
          • tajo-storage/tajo-storage-common/src/main/resources/storage-default.json
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java
          • tajo-docs/src/main/sphinx/index.rst
          • tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-0.11.0-build #53 (See https://builds.apache.org/job/Tajo-0.11.0-build/53/ ) TAJO-1730 : JDBC Tablespace support. (hyunsik: rev 144a02e342d256f3c58a2f69356ea08c6e43ac75) tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/SQLBuilder.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q2_Part_MixedStorage.sql tajo-core/src/main/java/org/apache/tajo/querymaster/Repartitioner.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileScanner.java tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/SQLExpressionGenerator.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/datetime_types.sql tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCountAsterisk.result tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/CommonConditionReduceRule.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/orders.sql tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLEndPointTests.java tajo-common/src/main/java/org/apache/tajo/exception/UnsupportedException.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q2_Part.result tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseScanner.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testQueryWithConnProperties.result tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testBetweenNumbers.result tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLSimpleQueryTests/testSelectAll.sql tajo-storage/tajo-storage-common/src/test/resources/storage-default.xml tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileTablespace.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCountAsterisk.sql tajo-common/src/main/java/org/apache/tajo/exception/NotImplementedException.java tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java tajo-plan/src/main/java/org/apache/tajo/plan/expr/AlgebraicUtil.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q3.sql tajo-common/src/main/java/org/apache/tajo/util/StringUtils.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/AbstractScanner.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testInPredicateWithNumbers.result tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcMetadataProviderBase.java tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java tajo-plan/src/main/proto/Plan.proto tajo-storage/tajo-storage-jdbc/pom.xml tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q5.result tajo-storage/tajo-storage-jdbc/src/main/proto/JdbcFragmentProtos.proto tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testFixedLengthFields.result tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testDateTimeTypes.sql tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestJoinQuery.java tajo-common/src/test/java/org/apache/tajo/util/TestFileUtils.java tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseTablespace.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/PgSQLTestServer.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/nation.sql tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCtasToHdfs.result tajo-core-tests/src/test/java/org/apache/tajo/cli/tools/TestDDLBuilder.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q2_Part.sql tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testInPredicateWithLiterals.sql tajo-storage/pom.xml tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLTablespace.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testBetweenDates.result tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestLinkedMetadataManager.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testFixedLengthFields.sql tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcTablespace.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/lineitem.sql tajo-common/src/main/java/org/apache/tajo/util/JavaResourceUtil.java tajo-common/src/main/java/org/apache/tajo/util/UriUtil.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testProjectedColumns.sql tajo-plan/src/main/java/org/apache/tajo/plan/logical/ScanNode.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLSimpleQueryTests/testSelectLimit.result tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Scanner.java tajo-docs/src/main/sphinx/storage_plugins.rst tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java tajo-plan/src/main/java/org/apache/tajo/plan/expr/EvalType.java tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultFileScanner.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/datetime_types.txt tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCaseWhenFilter.result tajo-plan/src/main/java/org/apache/tajo/plan/serder/EvalNodeSerializer.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testBetweenDates.sql tajo-catalog/tajo-catalog-common/src/test/java/org/apache/tajo/catalog/TestFunctionDesc.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q3.result tajo-plan/src/main/java/org/apache/tajo/plan/exprrewrite/rules/ConstantFolding.java tajo-project/pom.xml tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLQueryTests.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/customer.sql tajo-storage/tajo-storage-pgsql/src/test/resources/dataset/.marker tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/LinkedMetadataManager.java tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcFragment.java tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLSimpleQueryTests.java tajo-dist/pom.xml tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testVariableLengthFields.result tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testInPredicateWithLiterals.result tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLMetadataProvider.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCaseWhenFilter.sql tajo-common/src/main/java/org/apache/tajo/exception/ExceptionUtil.java tajo-client/src/main/java/org/apache/tajo/client/CatalogAdminClientImpl.java tajo-core/src/main/java/org/apache/tajo/master/exec/QueryExecutor.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testDateTimeTypes.result tajo-core-tests/src/test/java/org/apache/tajo/storage/TestRowFile.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLSimpleQueryTests/testSelectLimit.sql tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/fragment/FragmentConvertor.java tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLMetadataProvider.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testVariableLengthFields.sql tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestLineReader.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q1.sql tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLJdbcScanner.java tajo-cluster-tests/src/test/java/org/apache/tajo/TpchTestBase.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q5.sql tajo-plan/src/main/java/org/apache/tajo/plan/exprrewrite/rules/ConstantPropagation.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageProperty.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TablespaceManager.java tajo-docs/src/main/sphinx/storage_plugin.rst tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestDelimitedTextFile.java tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java tajo-storage/tajo-storage-pgsql/pom.xml tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java tajo-docs/src/main/sphinx/storage_plugins/overview.rst tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/OldStorageManager.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCtasToHdfs.sql tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java tajo-plan/src/main/java/org/apache/tajo/plan/expr/SimpleEvalNodeVisitor.java tajo-core-tests/src/test/java/org/apache/tajo/parser/sql/TestSQLAnalyzer.java tajo-common/src/main/java/org/apache/tajo/storage/StorageService.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testSimpleFilter.result tajo-storage/tajo-storage-common/src/main/resources/storage-default.xml tajo-storage/tajo-storage-jdbc/src/main/test/java/org/apache/tajo/storage/jdbc/TestConnectionInfo.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/supplier.sql tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q2_Part_MixedStorage.result tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLSimpleQueryTests/testSelectAll.result tajo-common/src/test/java/org/apache/tajo/util/TestUriUtil.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestFileTablespace.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/region.sql tajo-cluster-tests/src/test/java/org/apache/tajo/TajoTestingCluster.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testSimpleFilter.sql tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q1.result tajo-storage/tajo-storage-hbase/src/test/java/org/apache/tajo/storage/hbase/TestHBaseTableSpace.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testQueryWithConnProperties.sql tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testBetweenNumbers.sql tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestHBaseTable.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testInPredicateWithNumbers.sql tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/part.sql tajo-core/src/main/java/org/apache/tajo/engine/codegen/EvalCodeGenerator.java tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLJdbcTableSpace.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testFunctionWithinFilter.result tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/NullScanner.java CHANGES tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/json/TestJsonSerDe.java tajo-common/src/main/java/org/apache/tajo/util/FileUtil.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/avro/TestAvroUtil.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testProjectedColumns.result tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableUtil.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testFunctionWithinFilter.sql tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Tablespace.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/partsupp.sql tajo-storage/tajo-storage-common/src/main/resources/storage-default.json tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java tajo-docs/src/main/sphinx/index.rst tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java
          Hide
          hyunsik Hyunsik Choi added a comment -

          Thank you for the reviews. I've just committed this feature to master and branch-0.11.0 branches.

          Show
          hyunsik Hyunsik Choi added a comment - Thank you for the reviews. I've just committed this feature to master and branch-0.11.0 branches.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-master-CODEGEN-build #518 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/518/)
          TAJO-1730: JDBC Tablespace support. (hyunsik: rev 0e4ad5635be05bdde8dac5feb6c9a85fb3cf0e06)

          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcTablespace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q1.sql
          • tajo-common/src/main/java/org/apache/tajo/storage/StorageService.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCountAsterisk.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCaseWhenFilter.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/lineitem.sql
          • tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLJdbcScanner.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/expr/SimpleEvalNodeVisitor.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testFixedLengthFields.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q2_Part_MixedStorage.sql
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/json/TestJsonSerDe.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testSimpleFilter.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/CommonConditionReduceRule.java
          • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseScanner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testInPredicateWithNumbers.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/nation.sql
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLQueryTests.java
          • tajo-common/src/test/java/org/apache/tajo/util/TestFileUtils.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/serder/EvalNodeSerializer.java
          • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLMetadataProvider.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q5.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testBetweenNumbers.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q3.result
          • tajo-common/src/test/java/org/apache/tajo/util/TestUriUtil.java
          • tajo-common/src/main/java/org/apache/tajo/util/FileUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testQueryWithConnProperties.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCountAsterisk.sql
          • tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/exprrewrite/rules/ConstantPropagation.java
          • tajo-docs/src/main/sphinx/index.rst
          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileTablespace.java
          • tajo-storage/tajo-storage-jdbc/src/main/test/java/org/apache/tajo/storage/jdbc/TestConnectionInfo.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java
          • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseTablespace.java
          • tajo-storage/tajo-storage-pgsql/pom.xml
          • tajo-dist/pom.xml
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TablespaceManager.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/expr/AlgebraicUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testBetweenDates.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/exprrewrite/rules/ConstantFolding.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/expr/EvalType.java
          • tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java
          • tajo-storage/tajo-storage-jdbc/pom.xml
          • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java
          • tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLMetadataProvider.java
          • tajo-core/src/main/java/org/apache/tajo/querymaster/Repartitioner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/orders.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testFixedLengthFields.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/dataset/.marker
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/part.sql
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/OldStorageManager.java
          • tajo-storage/tajo-storage-common/src/main/resources/storage-default.json
          • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testQueryWithConnProperties.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testSimpleFilter.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q5.result
          • tajo-common/src/main/java/org/apache/tajo/util/JavaResourceUtil.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Scanner.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCaseWhenFilter.sql
          • tajo-plan/src/main/proto/Plan.proto
          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/SQLBuilder.java
          • tajo-docs/src/main/sphinx/storage_plugin.rst
          • tajo-core/src/main/java/org/apache/tajo/engine/codegen/EvalCodeGenerator.java
          • tajo-docs/src/main/sphinx/storage_plugins.rst
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testDateTimeTypes.sql
          • tajo-storage/pom.xml
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testBetweenDates.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testInPredicateWithNumbers.result
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestLineReader.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/supplier.sql
          • tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultFileScanner.java
          • tajo-common/src/main/java/org/apache/tajo/exception/NotImplementedException.java
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/SQLExpressionGenerator.java
          • tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLTablespace.java
          • tajo-catalog/tajo-catalog-common/src/test/java/org/apache/tajo/catalog/TestFunctionDesc.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Tablespace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLSimpleQueryTests/testSelectLimit.result
          • tajo-core-tests/src/test/java/org/apache/tajo/storage/TestRowFile.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/fragment/FragmentConvertor.java
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcFragment.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testBetweenNumbers.result
          • tajo-common/src/main/java/org/apache/tajo/exception/UnsupportedException.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/customer.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLSimpleQueryTests/testSelectLimit.sql
          • tajo-core-tests/src/test/java/org/apache/tajo/cli/tools/TestDDLBuilder.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestJoinQuery.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/region.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q1.result
          • tajo-project/pom.xml
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testVariableLengthFields.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLSimpleQueryTests/testSelectAll.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testFunctionWithinFilter.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCtasToHdfs.result
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcMetadataProviderBase.java
          • tajo-core-tests/src/test/java/org/apache/tajo/parser/sql/TestSQLAnalyzer.java
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLSimpleQueryTests/testSelectAll.result
          • tajo-storage/tajo-storage-common/src/test/resources/storage-default.xml
          • tajo-common/src/main/java/org/apache/tajo/util/StringUtils.java
          • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java
          • CHANGES
          • tajo-client/src/main/java/org/apache/tajo/client/CatalogAdminClientImpl.java
          • tajo-core/src/main/java/org/apache/tajo/master/exec/QueryExecutor.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestDelimitedTextFile.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/datetime_types.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCtasToHdfs.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testFunctionWithinFilter.sql
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLSimpleQueryTests.java
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/PgSQLTestServer.java
          • tajo-storage/tajo-storage-common/src/main/resources/storage-default.xml
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testVariableLengthFields.sql
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testProjectedColumns.sql
          • tajo-storage/tajo-storage-jdbc/src/main/proto/JdbcFragmentProtos.proto
          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileScanner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q3.sql
          • tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/AbstractScanner.java
          • tajo-cluster-tests/src/test/java/org/apache/tajo/TpchTestBase.java
          • tajo-cluster-tests/src/test/java/org/apache/tajo/TajoTestingCluster.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testProjectedColumns.result
          • tajo-common/src/main/java/org/apache/tajo/exception/ExceptionUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q2_Part.sql
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLJdbcTableSpace.java
          • tajo-common/src/main/java/org/apache/tajo/util/UriUtil.java
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestHBaseTable.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageProperty.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testInPredicateWithLiterals.sql
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/datetime_types.txt
          • tajo-docs/src/main/sphinx/storage_plugins/overview.rst
          • tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLEndPointTests.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/avro/TestAvroUtil.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q2_Part_MixedStorage.result
          • tajo-storage/tajo-storage-hbase/src/test/java/org/apache/tajo/storage/hbase/TestHBaseTableSpace.java
          • tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testInPredicateWithLiterals.result
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testDateTimeTypes.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/logical/ScanNode.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q2_Part.result
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java
          • tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestLinkedMetadataManager.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestFileTablespace.java
          • tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/partsupp.sql
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/LinkedMetadataManager.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/NullScanner.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #518 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/518/ ) TAJO-1730 : JDBC Tablespace support. (hyunsik: rev 0e4ad5635be05bdde8dac5feb6c9a85fb3cf0e06) tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcTablespace.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q1.sql tajo-common/src/main/java/org/apache/tajo/storage/StorageService.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCountAsterisk.result tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCaseWhenFilter.result tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/lineitem.sql tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLJdbcScanner.java tajo-plan/src/main/java/org/apache/tajo/plan/expr/SimpleEvalNodeVisitor.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testFixedLengthFields.result tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q2_Part_MixedStorage.sql tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/json/TestJsonSerDe.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testSimpleFilter.sql tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/CommonConditionReduceRule.java tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseScanner.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testInPredicateWithNumbers.sql tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/nation.sql tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLQueryTests.java tajo-common/src/test/java/org/apache/tajo/util/TestFileUtils.java tajo-plan/src/main/java/org/apache/tajo/plan/serder/EvalNodeSerializer.java tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableUtil.java tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLMetadataProvider.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q5.sql tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testBetweenNumbers.sql tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q3.result tajo-common/src/test/java/org/apache/tajo/util/TestUriUtil.java tajo-common/src/main/java/org/apache/tajo/util/FileUtil.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testQueryWithConnProperties.sql tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCountAsterisk.sql tajo-core/src/main/java/org/apache/tajo/querymaster/Stage.java tajo-plan/src/main/java/org/apache/tajo/plan/exprrewrite/rules/ConstantPropagation.java tajo-docs/src/main/sphinx/index.rst tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileTablespace.java tajo-storage/tajo-storage-jdbc/src/main/test/java/org/apache/tajo/storage/jdbc/TestConnectionInfo.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseTablespace.java tajo-storage/tajo-storage-pgsql/pom.xml tajo-dist/pom.xml tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/TablespaceManager.java tajo-plan/src/main/java/org/apache/tajo/plan/expr/AlgebraicUtil.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testBetweenDates.sql tajo-plan/src/main/java/org/apache/tajo/plan/exprrewrite/rules/ConstantFolding.java tajo-plan/src/main/java/org/apache/tajo/plan/expr/EvalType.java tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java tajo-storage/tajo-storage-jdbc/pom.xml tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLMetadataProvider.java tajo-core/src/main/java/org/apache/tajo/querymaster/Repartitioner.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/orders.sql tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testFixedLengthFields.sql tajo-storage/tajo-storage-pgsql/src/test/resources/dataset/.marker tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/part.sql tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/OldStorageManager.java tajo-storage/tajo-storage-common/src/main/resources/storage-default.json tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testQueryWithConnProperties.result tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testSimpleFilter.result tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q5.result tajo-common/src/main/java/org/apache/tajo/util/JavaResourceUtil.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Scanner.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCaseWhenFilter.sql tajo-plan/src/main/proto/Plan.proto tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/SQLBuilder.java tajo-docs/src/main/sphinx/storage_plugin.rst tajo-core/src/main/java/org/apache/tajo/engine/codegen/EvalCodeGenerator.java tajo-docs/src/main/sphinx/storage_plugins.rst tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testDateTimeTypes.sql tajo-storage/pom.xml tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testBetweenDates.result tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testInPredicateWithNumbers.result tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestLineReader.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/supplier.sql tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultFileScanner.java tajo-common/src/main/java/org/apache/tajo/exception/NotImplementedException.java tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/SQLExpressionGenerator.java tajo-storage/tajo-storage-pgsql/src/main/java/org/apache/tajo/storage/pgsql/PgSQLTablespace.java tajo-catalog/tajo-catalog-common/src/test/java/org/apache/tajo/catalog/TestFunctionDesc.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Tablespace.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLSimpleQueryTests/testSelectLimit.result tajo-core-tests/src/test/java/org/apache/tajo/storage/TestRowFile.java tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/fragment/FragmentConvertor.java tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcFragment.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testBetweenNumbers.result tajo-common/src/main/java/org/apache/tajo/exception/UnsupportedException.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/customer.sql tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLSimpleQueryTests/testSelectLimit.sql tajo-core-tests/src/test/java/org/apache/tajo/cli/tools/TestDDLBuilder.java tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestJoinQuery.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/region.sql tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q1.result tajo-project/pom.xml tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testVariableLengthFields.result tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLSimpleQueryTests/testSelectAll.sql tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testFunctionWithinFilter.result tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testCtasToHdfs.result tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcMetadataProviderBase.java tajo-core-tests/src/test/java/org/apache/tajo/parser/sql/TestSQLAnalyzer.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLSimpleQueryTests/testSelectAll.result tajo-storage/tajo-storage-common/src/test/resources/storage-default.xml tajo-common/src/main/java/org/apache/tajo/util/StringUtils.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java CHANGES tajo-client/src/main/java/org/apache/tajo/client/CatalogAdminClientImpl.java tajo-core/src/main/java/org/apache/tajo/master/exec/QueryExecutor.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestDelimitedTextFile.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/datetime_types.sql tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testCtasToHdfs.sql tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testFunctionWithinFilter.sql tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLSimpleQueryTests.java tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/PgSQLTestServer.java tajo-storage/tajo-storage-common/src/main/resources/storage-default.xml tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testVariableLengthFields.sql tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testProjectedColumns.sql tajo-storage/tajo-storage-jdbc/src/main/proto/JdbcFragmentProtos.proto tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileScanner.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q3.sql tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/AbstractScanner.java tajo-cluster-tests/src/test/java/org/apache/tajo/TpchTestBase.java tajo-cluster-tests/src/test/java/org/apache/tajo/TajoTestingCluster.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testProjectedColumns.result tajo-common/src/main/java/org/apache/tajo/exception/ExceptionUtil.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testTPCH_Q2_Part.sql tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLJdbcTableSpace.java tajo-common/src/main/java/org/apache/tajo/util/UriUtil.java tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestHBaseTable.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageProperty.java tajo-storage/tajo-storage-pgsql/src/test/resources/queries/TestPgSQLQueryTests/testInPredicateWithLiterals.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/datetime_types.txt tajo-docs/src/main/sphinx/storage_plugins/overview.rst tajo-storage/tajo-storage-pgsql/src/test/java/org/apache/tajo/storage/pgsql/TestPgSQLEndPointTests.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/avro/TestAvroUtil.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q2_Part_MixedStorage.result tajo-storage/tajo-storage-hbase/src/test/java/org/apache/tajo/storage/hbase/TestHBaseTableSpace.java tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testInPredicateWithLiterals.result tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testDateTimeTypes.result tajo-plan/src/main/java/org/apache/tajo/plan/logical/ScanNode.java tajo-storage/tajo-storage-pgsql/src/test/resources/results/TestPgSQLQueryTests/testTPCH_Q2_Part.result tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java tajo-catalog/tajo-catalog-server/src/test/java/org/apache/tajo/catalog/TestLinkedMetadataManager.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestFileTablespace.java tajo-storage/tajo-storage-pgsql/src/test/resources/pgsql/partsupp.sql tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/LinkedMetadataManager.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/NullScanner.java
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/tajo/pull/719

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/719
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on the pull request:

          https://github.com/apache/tajo/pull/719#issuecomment-140797602

          +1, thank you for the effort.
          I'm very excited with that we can finally use this nice feature!

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/719#issuecomment-140797602 +1, thank you for the effort. I'm very excited with that we can finally use this nice feature!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/719#issuecomment-140789501

          I reflected your comments, and I rebased the patch. That's nice idea. I'll do like that in another ticket later.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/719#issuecomment-140789501 I reflected your comments, and I rebased the patch. That's nice idea. I'll do like that in another ticket later.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on the pull request:

          https://github.com/apache/tajo/pull/719#issuecomment-140642251

          The latest patch looks good to me! Thank you for quick update.
          I have two more comments.

          First, as you know, there is a problem of precision of floating numbers. We can simply ignore this problem until we support decimal or numeric types.

          Second, I think that it would be nicer and easier to comprehend the overall logic related to tablespace if you separate its implementation. More specifically, in the current implementation, Tablespace is a single class, and contains information about itself and provides some functionalities to interact with underlying storage. This makes the implementation quite complicated. So, how about separating it into Tablespace and TablespaceHandler? I'm just suggesting you, and you don't have to do it in this ticket even though you agree. Maybe it causes huge changes.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/719#issuecomment-140642251 The latest patch looks good to me! Thank you for quick update. I have two more comments. First, as you know, there is a problem of precision of floating numbers. We can simply ignore this problem until we support decimal or numeric types. Second, I think that it would be nicer and easier to comprehend the overall logic related to tablespace if you separate its implementation. More specifically, in the current implementation, Tablespace is a single class, and contains information about itself and provides some functionalities to interact with underlying storage. This makes the implementation quite complicated. So, how about separating it into Tablespace and TablespaceHandler? I'm just suggesting you, and you don't have to do it in this ticket even though you agree. Maybe it causes huge changes.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r39596553

          — Diff: tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Tablespace.java —
          @@ -18,31 +18,28 @@

          package org.apache.tajo.storage;

          +import net.minidev.json.JSONObject;
          import org.apache.hadoop.fs.Path;
          import org.apache.tajo.ExecutionBlockId;
          import org.apache.tajo.OverridableConf;
          import org.apache.tajo.TaskAttemptId;
          -import org.apache.tajo.catalog.Schema;
          -import org.apache.tajo.catalog.SortSpec;
          -import org.apache.tajo.catalog.TableDesc;
          -import org.apache.tajo.catalog.TableMeta;
          +import org.apache.tajo.catalog.*;
          import org.apache.tajo.catalog.proto.CatalogProtos.FragmentProto;
          import org.apache.tajo.conf.TajoConf;
          import org.apache.tajo.exception.TajoException;
          import org.apache.tajo.exception.TajoRuntimeException;
          import org.apache.tajo.exception.UnsupportedException;
          import org.apache.tajo.plan.LogicalPlan;
          +import org.apache.tajo.plan.expr.EvalNode;
          import org.apache.tajo.plan.logical.LogicalNode;
          import org.apache.tajo.plan.logical.ScanNode;
          — End diff –

          Unused import.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r39596553 — Diff: tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Tablespace.java — @@ -18,31 +18,28 @@ package org.apache.tajo.storage; +import net.minidev.json.JSONObject; import org.apache.hadoop.fs.Path; import org.apache.tajo.ExecutionBlockId; import org.apache.tajo.OverridableConf; import org.apache.tajo.TaskAttemptId; -import org.apache.tajo.catalog.Schema; -import org.apache.tajo.catalog.SortSpec; -import org.apache.tajo.catalog.TableDesc; -import org.apache.tajo.catalog.TableMeta; +import org.apache.tajo.catalog.*; import org.apache.tajo.catalog.proto.CatalogProtos.FragmentProto; import org.apache.tajo.conf.TajoConf; import org.apache.tajo.exception.TajoException; import org.apache.tajo.exception.TajoRuntimeException; import org.apache.tajo.exception.UnsupportedException; import org.apache.tajo.plan.LogicalPlan; +import org.apache.tajo.plan.expr.EvalNode; import org.apache.tajo.plan.logical.LogicalNode; import org.apache.tajo.plan.logical.ScanNode; — End diff – Unused import.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r39596419

          — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java —
          @@ -1398,24 +1398,18 @@ public ScanNode visitRelation(PlanContext context, Stack<Expr> stack, Relation e
          return targets;
          }

          • private void updatePhysicalInfo(PlanContext planContext, TableDesc desc) {
          • if (desc.getUri() != null &&
          • !desc.getMeta().getStoreType().equals("SYSTEM") &&
          • !desc.getMeta().getStoreType().equals("FAKEFILE") && // FAKEFILE is used for test
          • PlannerUtil.isFileStorageType(desc.getMeta().getStoreType())) {
            + private void updatePhysicalInfo(TableDesc desc) {
            +
            + // FAKEFILE is used for test
            + if (!desc.getMeta().getStoreType().equals("SYSTEM") && !desc.getMeta().getStoreType().equals("FAKEFILE")) {
            try {
          • Path path = new Path(desc.getUri());
          • FileSystem fs = path.getFileSystem(planContext.queryContext.getConf());
          • FileStatus status = fs.getFileStatus(path);
          • if (desc.getStats() != null && (status.isDirectory() || status.isFile())) {
          • ContentSummary summary = fs.getContentSummary(path);
          • if (summary != null) { - long volume = summary.getLength(); - desc.getStats().setNumBytes(volume); - }

            + if (desc.getStats() != null)

            { + desc.getStats().setNumBytes(storage.getTableVolumn(desc.getUri())); }

            } catch (Throwable t) {

              • End diff –

          This line seems to be changed to catch UnsupportedException.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r39596419 — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java — @@ -1398,24 +1398,18 @@ public ScanNode visitRelation(PlanContext context, Stack<Expr> stack, Relation e return targets; } private void updatePhysicalInfo(PlanContext planContext, TableDesc desc) { if (desc.getUri() != null && !desc.getMeta().getStoreType().equals("SYSTEM") && !desc.getMeta().getStoreType().equals("FAKEFILE") && // FAKEFILE is used for test PlannerUtil.isFileStorageType(desc.getMeta().getStoreType())) { + private void updatePhysicalInfo(TableDesc desc) { + + // FAKEFILE is used for test + if (!desc.getMeta().getStoreType().equals("SYSTEM") && !desc.getMeta().getStoreType().equals("FAKEFILE")) { try { Path path = new Path(desc.getUri()); FileSystem fs = path.getFileSystem(planContext.queryContext.getConf()); FileStatus status = fs.getFileStatus(path); if (desc.getStats() != null && (status.isDirectory() || status.isFile())) { ContentSummary summary = fs.getContentSummary(path); if (summary != null) { - long volume = summary.getLength(); - desc.getStats().setNumBytes(volume); - } + if (desc.getStats() != null) { + desc.getStats().setNumBytes(storage.getTableVolumn(desc.getUri())); } } catch (Throwable t) { End diff – This line seems to be changed to catch UnsupportedException.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/719#issuecomment-140209546

          I fixed some bugs. I also add ``connection_properties`` key to allow user to set JDBC connection parameter, avoiding to add username and password in a JDBC connection url.

          Example:
          ```
          {
          "spaces": {
          "pgsql": {
          "uri": "jdbc:postgresql://localhost:5432/hyunsik",
          "configs": {
          "mapped_database": "pgsql",
          "connection_properties":

          { "user": "hyunsik", "password": "xxxxxx" }

          }
          }
          }
          }
          ```

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/719#issuecomment-140209546 I fixed some bugs. I also add ``connection_properties`` key to allow user to set JDBC connection parameter, avoiding to add username and password in a JDBC connection url. Example: ``` { "spaces": { "pgsql": { "uri": "jdbc:postgresql://localhost:5432/hyunsik", "configs": { "mapped_database": "pgsql", "connection_properties": { "user": "hyunsik", "password": "xxxxxx" } } } } } ```
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on the pull request:

          https://github.com/apache/tajo/pull/719#issuecomment-139851982

          There is another issue as follows.
          ```
          postgres> \d
          customer
          postgres> select count from customer;
          ERROR: No error message
          ```

          Here is the query master log.
          ```
          ...
          2015-09-13 17:34:29,422 INFO org.apache.tajo.engine.planner.PhysicalPlannerImpl: The planner chooses [Hash Aggregation]
          2015-09-13 17:34:29,429 INFO org.apache.tajo.storage.jdbc.JdbcScanner: Generated SQL: SELECT 1 FROM null
          2015-09-13 17:34:29,434 ERROR org.apache.tajo.worker.TaskImpl: internal error: FATAL: database "jihoonson&table=customer" does not exist
          org.apache.tajo.exception.TajoInternalError: internal error: FATAL: database "jihoonson&table=customer" does not exist
          at org.apache.tajo.storage.jdbc.JdbcScanner.executeQueryAndGetIter(JdbcScanner.java:249)
          at org.apache.tajo.storage.jdbc.JdbcScanner.next(JdbcScanner.java:102)
          at org.apache.tajo.engine.planner.physical.FullScanIterator.hasNext(FullScanIterator.java:39)
          at org.apache.tajo.engine.planner.physical.SeqScanExec.next(SeqScanExec.java:258)
          at org.apache.tajo.engine.planner.physical.HashAggregateExec.compute(HashAggregateExec.java:52)
          at org.apache.tajo.engine.planner.physical.HashAggregateExec.next(HashAggregateExec.java:84)
          at org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:102)
          at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:403)
          at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
          at java.util.concurrent.FutureTask.run(FutureTask.java:262)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:745)
          2015-09-13 17:34:29,435 INFO org.apache.tajo.worker.TaskAttemptContext: Query status of ta_1442133232061_0002_000001_000000_00 is changed to TA_FAILED
          2015-09-13 17:34:29,437 INFO org.apache.tajo.worker.TaskImpl: ta_1442133232061_0002_000001_000000_00 completed. Worker's task counter - total:1, succeeded: 0, killed: 0, failed: 1
          ...
          ```
          In the above error message of ```database "jihoonson&table=customer" does not exist```, ```jihoonson``` is the name of the database which is mapped from pgsql to tajo.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/719#issuecomment-139851982 There is another issue as follows. ``` postgres> \d customer postgres> select count from customer; ERROR: No error message ``` Here is the query master log. ``` ... 2015-09-13 17:34:29,422 INFO org.apache.tajo.engine.planner.PhysicalPlannerImpl: The planner chooses [Hash Aggregation] 2015-09-13 17:34:29,429 INFO org.apache.tajo.storage.jdbc.JdbcScanner: Generated SQL: SELECT 1 FROM null 2015-09-13 17:34:29,434 ERROR org.apache.tajo.worker.TaskImpl: internal error: FATAL: database "jihoonson&table=customer" does not exist org.apache.tajo.exception.TajoInternalError: internal error: FATAL: database "jihoonson&table=customer" does not exist at org.apache.tajo.storage.jdbc.JdbcScanner.executeQueryAndGetIter(JdbcScanner.java:249) at org.apache.tajo.storage.jdbc.JdbcScanner.next(JdbcScanner.java:102) at org.apache.tajo.engine.planner.physical.FullScanIterator.hasNext(FullScanIterator.java:39) at org.apache.tajo.engine.planner.physical.SeqScanExec.next(SeqScanExec.java:258) at org.apache.tajo.engine.planner.physical.HashAggregateExec.compute(HashAggregateExec.java:52) at org.apache.tajo.engine.planner.physical.HashAggregateExec.next(HashAggregateExec.java:84) at org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:102) at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:403) at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 2015-09-13 17:34:29,435 INFO org.apache.tajo.worker.TaskAttemptContext: Query status of ta_1442133232061_0002_000001_000000_00 is changed to TA_FAILED 2015-09-13 17:34:29,437 INFO org.apache.tajo.worker.TaskImpl: ta_1442133232061_0002_000001_000000_00 completed. Worker's task counter - total:1, succeeded: 0, killed: 0, failed: 1 ... ``` In the above error message of ```database "jihoonson&table=customer" does not exist```, ```jihoonson``` is the name of the database which is mapped from pgsql to tajo.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/719#issuecomment-139491371

          I updated the patch, including the followings:

          • Add CASE WHEN, function, Between expression support and Boolean constant support
          • Add more unit tests for various filter expressions
          • Fix many bugs
          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/719#issuecomment-139491371 I updated the patch, including the followings: Add CASE WHEN, function, Between expression support and Boolean constant support Add more unit tests for various filter expressions Fix many bugs
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on the pull request:

          https://github.com/apache/tajo/pull/719#issuecomment-137692131

          The result of the following query is different from pgsql.

                1. Tajo
                  ```
                  postgres> select
                  > n_name,
                  > sum(l_extendedprice * (1 - l_discount)) as revenue
                  > from
                  > customer,
                  > orders,
                  > lineitem,
                  > supplier,
                  > nation,
                  > region
                  > where
                  > c_custkey = o_custkey
                  > and l_orderkey = o_orderkey
                  > and l_suppkey = s_suppkey
                  > and c_nationkey = s_nationkey
                  > and s_nationkey = n_nationkey
                  > and n_regionkey = r_regionkey
                  > and r_name = 'ASIA'
                  > and o_orderdate >= '1994-01-01'::date
                  > and o_orderdate < '1995-01-01'::timestamp
                  > group by
                  > n_name
                  > order by
                  > revenue desc;
                  Progress: 100%, response time: 0.559 sec
                  n_name, revenue
                  -------------------------------
                  (0 rows, 0.559 sec, 0 B selected)
                  ```
                1. PostgreSQL
                  ```
                  postgres=# select
                  postgres-# n_name,
                  postgres-# sum(l_extendedprice * (1 - l_discount)) as revenue
                  postgres-# from
                  postgres-# customer,
                  postgres-# orders,
                  postgres-# lineitem,
                  postgres-# supplier,
                  postgres-# nation,
                  postgres-# region
                  postgres-# where
                  postgres-# c_custkey = o_custkey
                  postgres-# and l_orderkey = o_orderkey
                  postgres-# and l_suppkey = s_suppkey
                  postgres-# and c_nationkey = s_nationkey
                  postgres-# and s_nationkey = n_nationkey
                  postgres-# and n_regionkey = r_regionkey
                  postgres-# and r_name = 'ASIA'
                  postgres-# and o_orderdate >= '1994-01-01'::date
                  postgres-# and o_orderdate < '1995-01-01'::timestamp
                  postgres-# group by
                  postgres-# n_name
                  postgres-# order by
                  postgres-# revenue desc;
                  n_name | revenue
                  ----------+-----------------
                  INDONESIA | 55502041.1697
                  VIETNAM | 55295086.9967
                  CHINA | 53724494.2566
                  INDIA | 52035512.0002
                  JAPAN | 45410175.6954001
                  (5 rows)
                  ```
          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/719#issuecomment-137692131 The result of the following query is different from pgsql. Tajo ``` postgres> select > n_name, > sum(l_extendedprice * (1 - l_discount)) as revenue > from > customer, > orders, > lineitem, > supplier, > nation, > region > where > c_custkey = o_custkey > and l_orderkey = o_orderkey > and l_suppkey = s_suppkey > and c_nationkey = s_nationkey > and s_nationkey = n_nationkey > and n_regionkey = r_regionkey > and r_name = 'ASIA' > and o_orderdate >= '1994-01-01'::date > and o_orderdate < '1995-01-01'::timestamp > group by > n_name > order by > revenue desc; Progress: 100%, response time: 0.559 sec n_name, revenue ------------------------------- (0 rows, 0.559 sec, 0 B selected) ``` PostgreSQL ``` postgres=# select postgres-# n_name, postgres-# sum(l_extendedprice * (1 - l_discount)) as revenue postgres-# from postgres-# customer, postgres-# orders, postgres-# lineitem, postgres-# supplier, postgres-# nation, postgres-# region postgres-# where postgres-# c_custkey = o_custkey postgres-# and l_orderkey = o_orderkey postgres-# and l_suppkey = s_suppkey postgres-# and c_nationkey = s_nationkey postgres-# and s_nationkey = n_nationkey postgres-# and n_regionkey = r_regionkey postgres-# and r_name = 'ASIA' postgres-# and o_orderdate >= '1994-01-01'::date postgres-# and o_orderdate < '1995-01-01'::timestamp postgres-# group by postgres-# n_name postgres-# order by postgres-# revenue desc; n_name | revenue ---------- + ----------------- INDONESIA | 55502041.1697 VIETNAM | 55295086.9967 CHINA | 53724494.2566 INDIA | 52035512.0002 JAPAN | 45410175.6954001 (5 rows) ```
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on the pull request:

          https://github.com/apache/tajo/pull/719#issuecomment-137691732

          The following query on JDBC table is failed.
          ```
          2015-09-04 18:42:24,723 INFO org.apache.tajo.storage.jdbc.JdbcScanner: Executed SQL Statement: SELECT FROM customer
          2015-09-04 18:42:24,728 ERROR org.apache.tajo.worker.TaskImpl: internal error: ERROR: syntax error at or near "FROM"
          Position: 9
          org.apache.tajo.exception.TajoInternalError: internal error: ERROR: syntax error at or near "FROM"
          Position: 9
          at org.apache.tajo.storage.jdbc.JdbcScanner.executeQueryAndGetIter(JdbcScanner.java:249)
          at org.apache.tajo.storage.jdbc.JdbcScanner.next(JdbcScanner.java:102)
          at org.apache.tajo.engine.planner.physical.FullScanIterator.hasNext(FullScanIterator.java:39)
          at org.apache.tajo.engine.planner.physical.SeqScanExec.next(SeqScanExec.java:258)
          at org.apache.tajo.engine.planner.physical.HashAggregateExec.compute(HashAggregateExec.java:52)
          at org.apache.tajo.engine.planner.physical.HashAggregateExec.next(HashAggregateExec.java:84)
          at org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:102)
          at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:403)
          at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
          at java.util.concurrent.FutureTask.run(FutureTask.java:262)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:745)
          2015-09-04 18:42:24,728 INFO org.apache.tajo.worker.TaskAttemptContext: Query status of ta_1441359373738_0003_000001_000000_02 is changed to TA_FAILED
          ```

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/719#issuecomment-137691732 The following query on JDBC table is failed. ``` 2015-09-04 18:42:24,723 INFO org.apache.tajo.storage.jdbc.JdbcScanner: Executed SQL Statement: SELECT FROM customer 2015-09-04 18:42:24,728 ERROR org.apache.tajo.worker.TaskImpl: internal error: ERROR: syntax error at or near "FROM" Position: 9 org.apache.tajo.exception.TajoInternalError: internal error: ERROR: syntax error at or near "FROM" Position: 9 at org.apache.tajo.storage.jdbc.JdbcScanner.executeQueryAndGetIter(JdbcScanner.java:249) at org.apache.tajo.storage.jdbc.JdbcScanner.next(JdbcScanner.java:102) at org.apache.tajo.engine.planner.physical.FullScanIterator.hasNext(FullScanIterator.java:39) at org.apache.tajo.engine.planner.physical.SeqScanExec.next(SeqScanExec.java:258) at org.apache.tajo.engine.planner.physical.HashAggregateExec.compute(HashAggregateExec.java:52) at org.apache.tajo.engine.planner.physical.HashAggregateExec.next(HashAggregateExec.java:84) at org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:102) at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:403) at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 2015-09-04 18:42:24,728 INFO org.apache.tajo.worker.TaskAttemptContext: Query status of ta_1441359373738_0003_000001_000000_02 is changed to TA_FAILED ```
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on the pull request:

          https://github.com/apache/tajo/pull/719#issuecomment-137691470

          When I submitted a query on HDFS table, the query is hanged with the following error.
          ```
          2015-09-04 18:36:30,726 INFO org.apache.tajo.storage.FileTablespace: Total # of splits: 1
          2015-09-04 18:36:30,727 ERROR org.apache.tajo.querymaster.Stage: Stage (eb_1441359373738_0001_000007) ERROR:
          java.lang.ClassCastException: org.apache.tajo.storage.pgsql.PgSQLTablespace cannot be cast to org.apache.tajo.storage.FileTablespace
          at org.apache.tajo.querymaster.Repartitioner.scheduleLeafTasksWithBroadcastTable(Repartitioner.java:507)
          at org.apache.tajo.querymaster.Repartitioner.scheduleFragmentsForJoinQuery(Repartitioner.java:231)
          at org.apache.tajo.querymaster.Stage$InitAndRequestContainer.schedule(Stage.java:1037)
          at org.apache.tajo.querymaster.Stage$InitAndRequestContainer.access$800(Stage.java:817)
          at org.apache.tajo.querymaster.Stage$InitAndRequestContainer$1.run(Stage.java:843)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
          at java.util.concurrent.FutureTask.run(FutureTask.java:262)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:745)
          ```
          Query hang is because the type cast exception is not handled, so it may not be related to this issue.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/719#issuecomment-137691470 When I submitted a query on HDFS table, the query is hanged with the following error. ``` 2015-09-04 18:36:30,726 INFO org.apache.tajo.storage.FileTablespace: Total # of splits: 1 2015-09-04 18:36:30,727 ERROR org.apache.tajo.querymaster.Stage: Stage (eb_1441359373738_0001_000007) ERROR: java.lang.ClassCastException: org.apache.tajo.storage.pgsql.PgSQLTablespace cannot be cast to org.apache.tajo.storage.FileTablespace at org.apache.tajo.querymaster.Repartitioner.scheduleLeafTasksWithBroadcastTable(Repartitioner.java:507) at org.apache.tajo.querymaster.Repartitioner.scheduleFragmentsForJoinQuery(Repartitioner.java:231) at org.apache.tajo.querymaster.Stage$InitAndRequestContainer.schedule(Stage.java:1037) at org.apache.tajo.querymaster.Stage$InitAndRequestContainer.access$800(Stage.java:817) at org.apache.tajo.querymaster.Stage$InitAndRequestContainer$1.run(Stage.java:843) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) ``` Query hang is because the type cast exception is not handled, so it may not be related to this issue.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38729285

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcTablespace.java —
          @@ -0,0 +1,161 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import com.google.common.collect.Lists;
          +import net.minidev.json.JSONObject;
          +import org.apache.hadoop.fs.Path;
          +import org.apache.tajo.ExecutionBlockId;
          +import org.apache.tajo.OverridableConf;
          +import org.apache.tajo.catalog.*;
          +import org.apache.tajo.exception.NotImplementedException;
          +import org.apache.tajo.exception.TajoInternalError;
          +import org.apache.tajo.exception.TajoRuntimeException;
          +import org.apache.tajo.exception.UnsupportedException;
          +import org.apache.tajo.plan.LogicalPlan;
          +import org.apache.tajo.plan.expr.EvalNode;
          +import org.apache.tajo.plan.logical.LogicalNode;
          +import org.apache.tajo.storage.*;
          +import org.apache.tajo.storage.fragment.Fragment;
          +
          +import javax.annotation.Nullable;
          +import java.io.IOException;
          +import java.net.URI;
          +import java.sql.Connection;
          +import java.sql.DatabaseMetaData;
          +import java.sql.DriverManager;
          +import java.sql.SQLException;
          +import java.util.List;
          +
          +/**
          + * JDBC Tablespace
          + */
          +public abstract class JdbcTablespace extends Tablespace {
          +
          + static final StorageProperty STORAGE_PROPERTY = new StorageProperty("rowstore", false, true, false, true);
          + static final FormatProperty FORMAT_PROPERTY = new FormatProperty(false, false, false);
          + public static final String MAPPED_DATABASE_CONFIG_KEY = "mapped_database";
          +
          + private Connection conn;
          +
          + public JdbcTablespace(String name, URI uri, JSONObject config)

          { + super(name, uri, config); + }

          +
          + @Override
          + protected void storageInit() throws IOException {
          + try

          { + this.conn = DriverManager.getConnection(uri.toASCIIString()); + }

          catch (SQLException e)

          { + throw new IOException(e); + }

          + }
          +
          + @Override
          + public long getTableVolume(URI uri) throws UnsupportedException

          { + throw new UnsupportedException(); + }

          +
          + @Override
          + public URI getTableUri(String databaseName, String tableName)

          { + return URI.create(getUri() + "&table=" + tableName); + }

          +
          + @Override
          + public List<Fragment> getSplits(String inputSourceId,
          + TableDesc tableDesc,
          + @Nullable EvalNode filterCondition) throws IOException

          { + return Lists.newArrayList((Fragment)new JdbcFragment(inputSourceId, tableDesc.getUri().toASCIIString())); + }

          +
          + @Override
          + public StorageProperty getProperty()

          { + return STORAGE_PROPERTY; + }

          +
          + @Override
          + public FormatProperty getFormatProperty(TableMeta meta)

          { + return FORMAT_PROPERTY; + }

          +
          + @Override
          + public void close() {
          — End diff –

          fixed.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38729285 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcTablespace.java — @@ -0,0 +1,161 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import com.google.common.collect.Lists; +import net.minidev.json.JSONObject; +import org.apache.hadoop.fs.Path; +import org.apache.tajo.ExecutionBlockId; +import org.apache.tajo.OverridableConf; +import org.apache.tajo.catalog.*; +import org.apache.tajo.exception.NotImplementedException; +import org.apache.tajo.exception.TajoInternalError; +import org.apache.tajo.exception.TajoRuntimeException; +import org.apache.tajo.exception.UnsupportedException; +import org.apache.tajo.plan.LogicalPlan; +import org.apache.tajo.plan.expr.EvalNode; +import org.apache.tajo.plan.logical.LogicalNode; +import org.apache.tajo.storage.*; +import org.apache.tajo.storage.fragment.Fragment; + +import javax.annotation.Nullable; +import java.io.IOException; +import java.net.URI; +import java.sql.Connection; +import java.sql.DatabaseMetaData; +import java.sql.DriverManager; +import java.sql.SQLException; +import java.util.List; + +/** + * JDBC Tablespace + */ +public abstract class JdbcTablespace extends Tablespace { + + static final StorageProperty STORAGE_PROPERTY = new StorageProperty("rowstore", false, true, false, true); + static final FormatProperty FORMAT_PROPERTY = new FormatProperty(false, false, false); + public static final String MAPPED_DATABASE_CONFIG_KEY = "mapped_database"; + + private Connection conn; + + public JdbcTablespace(String name, URI uri, JSONObject config) { + super(name, uri, config); + } + + @Override + protected void storageInit() throws IOException { + try { + this.conn = DriverManager.getConnection(uri.toASCIIString()); + } catch (SQLException e) { + throw new IOException(e); + } + } + + @Override + public long getTableVolume(URI uri) throws UnsupportedException { + throw new UnsupportedException(); + } + + @Override + public URI getTableUri(String databaseName, String tableName) { + return URI.create(getUri() + "&table=" + tableName); + } + + @Override + public List<Fragment> getSplits(String inputSourceId, + TableDesc tableDesc, + @Nullable EvalNode filterCondition) throws IOException { + return Lists.newArrayList((Fragment)new JdbcFragment(inputSourceId, tableDesc.getUri().toASCIIString())); + } + + @Override + public StorageProperty getProperty() { + return STORAGE_PROPERTY; + } + + @Override + public FormatProperty getFormatProperty(TableMeta meta) { + return FORMAT_PROPERTY; + } + + @Override + public void close() { — End diff – fixed.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38729237

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java —
          @@ -0,0 +1,112 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import org.apache.tajo.exception.TajoInternalError;
          +
          +import java.net.URI;
          +import java.util.HashMap;
          +import java.util.Map;
          +
          +public class ConnectionInfo {
          + String scheme;
          + String dbName;
          + String tableName;
          + String user;
          + String password;
          + Map<String, String> params;
          +
          + public String scheme()

          { + return scheme; + }

          +
          + public String database()

          { + return dbName; + }

          +
          + public String table()

          { + return tableName; + }

          +
          + public String user()

          { + return user; + }

          +
          + public String password()

          { + return password; + }

          +
          + public static ConnectionInfo fromURI(String originalUri)

          { + return fromURI(URI.create(originalUri)); + }

          +
          + public static ConnectionInfo fromURI(URI originalUri) {
          + final String uriStr = originalUri.toASCIIString();
          + URI uri = originalUri;
          +
          + final ConnectionInfo connInfo = new ConnectionInfo();
          + connInfo.scheme = uriStr.substring(0, uriStr.indexOf("://"));
          — End diff –

          Thanks for your reply.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38729237 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java — @@ -0,0 +1,112 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import org.apache.tajo.exception.TajoInternalError; + +import java.net.URI; +import java.util.HashMap; +import java.util.Map; + +public class ConnectionInfo { + String scheme; + String dbName; + String tableName; + String user; + String password; + Map<String, String> params; + + public String scheme() { + return scheme; + } + + public String database() { + return dbName; + } + + public String table() { + return tableName; + } + + public String user() { + return user; + } + + public String password() { + return password; + } + + public static ConnectionInfo fromURI(String originalUri) { + return fromURI(URI.create(originalUri)); + } + + public static ConnectionInfo fromURI(URI originalUri) { + final String uriStr = originalUri.toASCIIString(); + URI uri = originalUri; + + final ConnectionInfo connInfo = new ConnectionInfo(); + connInfo.scheme = uriStr.substring(0, uriStr.indexOf("://")); — End diff – Thanks for your reply.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38729200

          — Diff: tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseTablespace.java —
          @@ -93,21 +94,13 @@ public void storageInit() throws IOException

          { hbaseConf.set(HConstants.ZOOKEEPER_CLIENT_PORT, splits[1]); }
          • @Override
          • public void setConfig(String name, String value) { - }
            -
            - @Override
            - public void setConfigs(Map<String, String> configs) { - }

            -
            public Configuration getHbaseConf()

            { return hbaseConf; }

          @Override

          • public long getTableVolume(URI uri) throws IOException {
          • return 0;
            + public long getTableVolume(URI uri) throws UnsupportedException {
            + throw new UnsupportedException();
              • End diff –

          Yes, you are right. NotImplementedException should be thrown when we will implement later.
          I found a workaround to get the total volume of hbase table, that is using the HDFS command ```hadoop fs -du```. This will be inefficient, but may be very helpful for query planning. So, I think we need to implement this function someday. What do you think?

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38729200 — Diff: tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseTablespace.java — @@ -93,21 +94,13 @@ public void storageInit() throws IOException { hbaseConf.set(HConstants.ZOOKEEPER_CLIENT_PORT, splits[1]); } @Override public void setConfig(String name, String value) { - } - - @Override - public void setConfigs(Map<String, String> configs) { - } - public Configuration getHbaseConf() { return hbaseConf; } @Override public long getTableVolume(URI uri) throws IOException { return 0; + public long getTableVolume(URI uri) throws UnsupportedException { + throw new UnsupportedException(); End diff – Yes, you are right. NotImplementedException should be thrown when we will implement later. I found a workaround to get the total volume of hbase table, that is using the HDFS command ```hadoop fs -du```. This will be inefficient, but may be very helpful for query planning. So, I think we need to implement this function someday. What do you think?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38729198

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java —
          @@ -0,0 +1,313 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import com.google.common.base.Preconditions;
          +import org.apache.commons.logging.Log;
          +import org.apache.commons.logging.LogFactory;
          +import org.apache.tajo.catalog.Column;
          +import org.apache.tajo.catalog.Schema;
          +import org.apache.tajo.catalog.TableMeta;
          +import org.apache.tajo.catalog.statistics.TableStats;
          +import org.apache.tajo.datum.DatumFactory;
          +import org.apache.tajo.datum.TimeDatum;
          +import org.apache.tajo.exception.TajoInternalError;
          +import org.apache.tajo.exception.TajoRuntimeException;
          +import org.apache.tajo.exception.UnsupportedDataTypeException;
          +import org.apache.tajo.exception.UnsupportedException;
          +import org.apache.tajo.plan.expr.EvalNode;
          +import org.apache.tajo.plan.logical.LogicalNode;
          +import org.apache.tajo.storage.Scanner;
          +import org.apache.tajo.storage.Tuple;
          +import org.apache.tajo.storage.VTuple;
          +
          +import java.io.Closeable;
          +import java.io.IOException;
          +import java.sql.*;
          +import java.util.Iterator;
          +
          +public abstract class JdbcScanner implements Scanner {
          + private static final Log LOG = LogFactory.getLog(JdbcScanner.class);
          +
          + protected final DatabaseMetaData dbMetaData;
          + protected final String tableName;
          + protected final Schema schema;
          + protected final TableMeta tableMeta;
          + protected final JdbcFragment fragment;
          + protected final TableStats stats;
          — End diff –

          Fixed.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38729198 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java — @@ -0,0 +1,313 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import com.google.common.base.Preconditions; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.tajo.catalog.Column; +import org.apache.tajo.catalog.Schema; +import org.apache.tajo.catalog.TableMeta; +import org.apache.tajo.catalog.statistics.TableStats; +import org.apache.tajo.datum.DatumFactory; +import org.apache.tajo.datum.TimeDatum; +import org.apache.tajo.exception.TajoInternalError; +import org.apache.tajo.exception.TajoRuntimeException; +import org.apache.tajo.exception.UnsupportedDataTypeException; +import org.apache.tajo.exception.UnsupportedException; +import org.apache.tajo.plan.expr.EvalNode; +import org.apache.tajo.plan.logical.LogicalNode; +import org.apache.tajo.storage.Scanner; +import org.apache.tajo.storage.Tuple; +import org.apache.tajo.storage.VTuple; + +import java.io.Closeable; +import java.io.IOException; +import java.sql.*; +import java.util.Iterator; + +public abstract class JdbcScanner implements Scanner { + private static final Log LOG = LogFactory.getLog(JdbcScanner.class); + + protected final DatabaseMetaData dbMetaData; + protected final String tableName; + protected final Schema schema; + protected final TableMeta tableMeta; + protected final JdbcFragment fragment; + protected final TableStats stats; — End diff – Fixed.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38729004

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java —
          @@ -0,0 +1,313 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import com.google.common.base.Preconditions;
          +import org.apache.commons.logging.Log;
          +import org.apache.commons.logging.LogFactory;
          +import org.apache.tajo.catalog.Column;
          +import org.apache.tajo.catalog.Schema;
          +import org.apache.tajo.catalog.TableMeta;
          +import org.apache.tajo.catalog.statistics.TableStats;
          +import org.apache.tajo.datum.DatumFactory;
          +import org.apache.tajo.datum.TimeDatum;
          +import org.apache.tajo.exception.TajoInternalError;
          +import org.apache.tajo.exception.TajoRuntimeException;
          +import org.apache.tajo.exception.UnsupportedDataTypeException;
          +import org.apache.tajo.exception.UnsupportedException;
          +import org.apache.tajo.plan.expr.EvalNode;
          +import org.apache.tajo.plan.logical.LogicalNode;
          +import org.apache.tajo.storage.Scanner;
          +import org.apache.tajo.storage.Tuple;
          +import org.apache.tajo.storage.VTuple;
          +
          +import java.io.Closeable;
          +import java.io.IOException;
          +import java.sql.*;
          +import java.util.Iterator;
          +
          +public abstract class JdbcScanner implements Scanner {
          + private static final Log LOG = LogFactory.getLog(JdbcScanner.class);
          +
          + protected final DatabaseMetaData dbMetaData;
          + protected final String tableName;
          + protected final Schema schema;
          + protected final TableMeta tableMeta;
          + protected final JdbcFragment fragment;
          + protected final TableStats stats;
          + protected final SQLBuilder builder;
          — End diff –

          getSQLBuilder() can be overridden by its subclass class because some SQL generation can be specified depending on RDBMS. So, I made it as a member variable.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38729004 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java — @@ -0,0 +1,313 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import com.google.common.base.Preconditions; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.tajo.catalog.Column; +import org.apache.tajo.catalog.Schema; +import org.apache.tajo.catalog.TableMeta; +import org.apache.tajo.catalog.statistics.TableStats; +import org.apache.tajo.datum.DatumFactory; +import org.apache.tajo.datum.TimeDatum; +import org.apache.tajo.exception.TajoInternalError; +import org.apache.tajo.exception.TajoRuntimeException; +import org.apache.tajo.exception.UnsupportedDataTypeException; +import org.apache.tajo.exception.UnsupportedException; +import org.apache.tajo.plan.expr.EvalNode; +import org.apache.tajo.plan.logical.LogicalNode; +import org.apache.tajo.storage.Scanner; +import org.apache.tajo.storage.Tuple; +import org.apache.tajo.storage.VTuple; + +import java.io.Closeable; +import java.io.IOException; +import java.sql.*; +import java.util.Iterator; + +public abstract class JdbcScanner implements Scanner { + private static final Log LOG = LogFactory.getLog(JdbcScanner.class); + + protected final DatabaseMetaData dbMetaData; + protected final String tableName; + protected final Schema schema; + protected final TableMeta tableMeta; + protected final JdbcFragment fragment; + protected final TableStats stats; + protected final SQLBuilder builder; — End diff – getSQLBuilder() can be overridden by its subclass class because some SQL generation can be specified depending on RDBMS. So, I made it as a member variable.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38728750

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java —
          @@ -0,0 +1,112 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import org.apache.tajo.exception.TajoInternalError;
          +
          +import java.net.URI;
          +import java.util.HashMap;
          +import java.util.Map;
          +
          +public class ConnectionInfo {
          + String scheme;
          + String dbName;
          + String tableName;
          + String user;
          + String password;
          + Map<String, String> params;
          +
          + public String scheme()

          { + return scheme; + }

          +
          + public String database()

          { + return dbName; + }

          +
          + public String table()

          { + return tableName; + }

          +
          + public String user()

          { + return user; + }

          +
          + public String password()

          { + return password; + }

          +
          + public static ConnectionInfo fromURI(String originalUri)

          { + return fromURI(URI.create(originalUri)); + }

          +
          + public static ConnectionInfo fromURI(URI originalUri) {
          + final String uriStr = originalUri.toASCIIString();
          + URI uri = originalUri;
          +
          + final ConnectionInfo connInfo = new ConnectionInfo();
          + connInfo.scheme = uriStr.substring(0, uriStr.indexOf("://"));
          — End diff –

          Actually, URI format we used here is not standard format. So, this method needs its own code. Thanks.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38728750 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java — @@ -0,0 +1,112 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import org.apache.tajo.exception.TajoInternalError; + +import java.net.URI; +import java.util.HashMap; +import java.util.Map; + +public class ConnectionInfo { + String scheme; + String dbName; + String tableName; + String user; + String password; + Map<String, String> params; + + public String scheme() { + return scheme; + } + + public String database() { + return dbName; + } + + public String table() { + return tableName; + } + + public String user() { + return user; + } + + public String password() { + return password; + } + + public static ConnectionInfo fromURI(String originalUri) { + return fromURI(URI.create(originalUri)); + } + + public static ConnectionInfo fromURI(URI originalUri) { + final String uriStr = originalUri.toASCIIString(); + URI uri = originalUri; + + final ConnectionInfo connInfo = new ConnectionInfo(); + connInfo.scheme = uriStr.substring(0, uriStr.indexOf("://")); — End diff – Actually, URI format we used here is not standard format. So, this method needs its own code. Thanks.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38728599

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java —
          @@ -0,0 +1,313 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import com.google.common.base.Preconditions;
          +import org.apache.commons.logging.Log;
          +import org.apache.commons.logging.LogFactory;
          +import org.apache.tajo.catalog.Column;
          +import org.apache.tajo.catalog.Schema;
          +import org.apache.tajo.catalog.TableMeta;
          +import org.apache.tajo.catalog.statistics.TableStats;
          +import org.apache.tajo.datum.DatumFactory;
          +import org.apache.tajo.datum.TimeDatum;
          +import org.apache.tajo.exception.TajoInternalError;
          +import org.apache.tajo.exception.TajoRuntimeException;
          +import org.apache.tajo.exception.UnsupportedDataTypeException;
          +import org.apache.tajo.exception.UnsupportedException;
          +import org.apache.tajo.plan.expr.EvalNode;
          +import org.apache.tajo.plan.logical.LogicalNode;
          +import org.apache.tajo.storage.Scanner;
          +import org.apache.tajo.storage.Tuple;
          +import org.apache.tajo.storage.VTuple;
          +
          +import java.io.Closeable;
          +import java.io.IOException;
          +import java.sql.*;
          +import java.util.Iterator;
          +
          +public abstract class JdbcScanner implements Scanner {
          + private static final Log LOG = LogFactory.getLog(JdbcScanner.class);
          +
          + protected final DatabaseMetaData dbMetaData;
          + protected final String tableName;
          + protected final Schema schema;
          + protected final TableMeta tableMeta;
          + protected final JdbcFragment fragment;
          + protected final TableStats stats;
          + protected final SQLBuilder builder;
          +
          + protected Column [] targets;
          + protected EvalNode filter;
          + protected Long limit;
          + protected LogicalNode planPart;
          + protected VTuple outTuple;
          + protected String generatedSql;
          + protected ResultSetIterator iter;
          +
          + public JdbcScanner(final DatabaseMetaData dbMetaData,
          + final Schema tableSchema,
          + final TableMeta tableMeta,
          + final JdbcFragment fragment)

          { + + Preconditions.checkNotNull(dbMetaData); + Preconditions.checkNotNull(tableSchema); + Preconditions.checkNotNull(tableMeta); + Preconditions.checkNotNull(fragment); + + this.dbMetaData = dbMetaData; + this.tableName = ConnectionInfo.fromURI(fragment.getUri()).tableName; + this.schema = tableSchema; + this.tableMeta = tableMeta; + this.fragment = fragment; + this.stats = new TableStats(); + builder = getSQLBuilder(getSQLExprBuilder()); + }

          +
          + @Override
          + public void init() throws IOException {
          + if (targets == null)

          { + targets = schema.toArray(); + }

          + outTuple = new VTuple(targets.length);
          +
          + if (planPart == null)

          { + generatedSql = builder.build(tableName, targets, filter, limit); + }

          else

          { + generatedSql = builder.build(planPart); + }

          + }
          +
          + @Override
          + public Tuple next() throws IOException {
          + if (iter == null)

          { + iter = executeQueryAndGetIter(); + }

          +
          + if (iter.hasNext())

          { + return iter.next(); + }

          else

          { + return null; + }

          + }
          +
          + @Override
          + public void reset() throws IOException {
          + if (iter != null)

          { + iter.rewind(); + }

          + }
          +
          + @Override
          + public void close() throws IOException {
          + if (iter != null)

          { + iter.close(); + }

          + }
          +
          + @Override
          + public void pushOperators(LogicalNode planPart)

          { + this.planPart = planPart; + }

          +
          +
          + @Override
          + public boolean isProjectable()

          { + return true; + }
          +
          + @Override
          + public void setTarget(Column [] targets) { + this.targets = targets; + }
          +
          + @Override
          + public boolean isSelectable() { + return true; + }

          +
          + @Override
          + public void setFilter(EvalNode filter)

          { + this.filter = filter; + }

          +
          + @Override
          + public void setLimit(long num)

          { + this.limit = num; + }

          +
          + @Override
          + public boolean isSplittable()

          { + return false; + }

          +
          + @Override
          + public float getProgress()

          { + return 0; + }

          +
          + @Override
          + public TableStats getInputStats()

          { + return stats; + }

          +
          + @Override
          + public Schema getSchema()

          { + return schema; + }

          +
          + protected SQLBuilder getSQLBuilder(SQLExpressionGenerator exprBuilder) {
          + return new SQLBuilder(dbMetaData, getSQLExprBuilder());
          — End diff –

          Thanks!

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38728599 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java — @@ -0,0 +1,313 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import com.google.common.base.Preconditions; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.tajo.catalog.Column; +import org.apache.tajo.catalog.Schema; +import org.apache.tajo.catalog.TableMeta; +import org.apache.tajo.catalog.statistics.TableStats; +import org.apache.tajo.datum.DatumFactory; +import org.apache.tajo.datum.TimeDatum; +import org.apache.tajo.exception.TajoInternalError; +import org.apache.tajo.exception.TajoRuntimeException; +import org.apache.tajo.exception.UnsupportedDataTypeException; +import org.apache.tajo.exception.UnsupportedException; +import org.apache.tajo.plan.expr.EvalNode; +import org.apache.tajo.plan.logical.LogicalNode; +import org.apache.tajo.storage.Scanner; +import org.apache.tajo.storage.Tuple; +import org.apache.tajo.storage.VTuple; + +import java.io.Closeable; +import java.io.IOException; +import java.sql.*; +import java.util.Iterator; + +public abstract class JdbcScanner implements Scanner { + private static final Log LOG = LogFactory.getLog(JdbcScanner.class); + + protected final DatabaseMetaData dbMetaData; + protected final String tableName; + protected final Schema schema; + protected final TableMeta tableMeta; + protected final JdbcFragment fragment; + protected final TableStats stats; + protected final SQLBuilder builder; + + protected Column [] targets; + protected EvalNode filter; + protected Long limit; + protected LogicalNode planPart; + protected VTuple outTuple; + protected String generatedSql; + protected ResultSetIterator iter; + + public JdbcScanner(final DatabaseMetaData dbMetaData, + final Schema tableSchema, + final TableMeta tableMeta, + final JdbcFragment fragment) { + + Preconditions.checkNotNull(dbMetaData); + Preconditions.checkNotNull(tableSchema); + Preconditions.checkNotNull(tableMeta); + Preconditions.checkNotNull(fragment); + + this.dbMetaData = dbMetaData; + this.tableName = ConnectionInfo.fromURI(fragment.getUri()).tableName; + this.schema = tableSchema; + this.tableMeta = tableMeta; + this.fragment = fragment; + this.stats = new TableStats(); + builder = getSQLBuilder(getSQLExprBuilder()); + } + + @Override + public void init() throws IOException { + if (targets == null) { + targets = schema.toArray(); + } + outTuple = new VTuple(targets.length); + + if (planPart == null) { + generatedSql = builder.build(tableName, targets, filter, limit); + } else { + generatedSql = builder.build(planPart); + } + } + + @Override + public Tuple next() throws IOException { + if (iter == null) { + iter = executeQueryAndGetIter(); + } + + if (iter.hasNext()) { + return iter.next(); + } else { + return null; + } + } + + @Override + public void reset() throws IOException { + if (iter != null) { + iter.rewind(); + } + } + + @Override + public void close() throws IOException { + if (iter != null) { + iter.close(); + } + } + + @Override + public void pushOperators(LogicalNode planPart) { + this.planPart = planPart; + } + + + @Override + public boolean isProjectable() { + return true; + } + + @Override + public void setTarget(Column [] targets) { + this.targets = targets; + } + + @Override + public boolean isSelectable() { + return true; + } + + @Override + public void setFilter(EvalNode filter) { + this.filter = filter; + } + + @Override + public void setLimit(long num) { + this.limit = num; + } + + @Override + public boolean isSplittable() { + return false; + } + + @Override + public float getProgress() { + return 0; + } + + @Override + public TableStats getInputStats() { + return stats; + } + + @Override + public Schema getSchema() { + return schema; + } + + protected SQLBuilder getSQLBuilder(SQLExpressionGenerator exprBuilder) { + return new SQLBuilder(dbMetaData, getSQLExprBuilder()); — End diff – Thanks!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38728586

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcMetadataProviderBase.java —
          @@ -0,0 +1,253 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import com.google.common.base.Function;
          +import com.google.common.collect.Collections2;
          +import com.google.common.collect.Lists;
          +import org.apache.commons.logging.Log;
          +import org.apache.commons.logging.LogFactory;
          +import org.apache.tajo.catalog.*;
          +import org.apache.tajo.catalog.statistics.TableStats;
          +import org.apache.tajo.common.TajoDataTypes.Type;
          +import org.apache.tajo.exception.*;
          +import org.apache.tajo.util.KeyValueSet;
          +import org.apache.tajo.util.Pair;
          +
          +import javax.annotation.Nullable;
          +import java.net.URI;
          +import java.sql.*;
          +import java.util.Collection;
          +import java.util.Collections;
          +import java.util.Comparator;
          +import java.util.List;
          +
          +import static org.apache.tajo.catalog.CatalogUtil.newSimpleDataType;
          +
          +public abstract class JdbcMetadataProviderBase implements MetadataProvider {
          + protected static final Log LOG = LogFactory.getLog(JdbcMetadataProviderBase.class);
          +
          + protected final JdbcTablespace space;
          + protected final String databaseName;
          +
          + protected final String jdbcUri;
          + protected final String username;
          + protected final String password;
          +
          + protected final Connection connection;
          +
          + public JdbcMetadataProviderBase(JdbcTablespace space, String dbName) {
          + this.space = space;
          + this.databaseName = dbName;
          +
          + ConnectionInfo connInfo = ConnectionInfo.fromURI(space.getUri());
          + this.jdbcUri = space.getUri().toASCIIString();
          + this.username = connInfo.user();
          + this.password = connInfo.password();
          +
          + try

          { + Class.forName(getJdbcDriverName()).newInstance(); + LOG.info(getJdbcDriverName() + " is loaded..."); + }

          catch (Exception e)

          { + throw new TajoInternalError(e); + }
          +
          + try { + connection = DriverManager.getConnection(jdbcUri, this.username, this.password); + } catch (SQLException e) { + throw new TajoInternalError(e); + }

          + }
          +
          + @Override
          + public String getTablespaceName()

          { + return space.getName(); + }

          +
          + @Override
          + public URI getTablespaceUri()

          { + return space.getUri(); + }

          +
          + @Override
          + public String getDatabaseName()

          { + return databaseName; + }

          +
          + @Override
          + public Collection<String> getSchemas()

          { + return Collections.EMPTY_SET; + }

          +
          + @Override
          + public Collection<String> getTables(@Nullable String schemaPattern, @Nullable String tablePattern) {
          + ResultSet res = null;
          + List<String> tableNames = Lists.newArrayList();
          + try {
          + res = connection.getMetaData().getTables(databaseName, schemaPattern, tablePattern, null);
          + while(res.next())

          { + tableNames.add(res.getString("TABLE_NAME")); + }

          + } catch (SQLException e)

          { + throw new TajoInternalError(e); + }

          finally {
          + try {
          + if (res != null)

          { + res.close(); + }

          + } catch (SQLException e)

          { + LOG.warn(e); + }

          + }
          +
          + return tableNames;
          + }
          +
          + private TypeDesc convertDataType(ResultSet res) throws SQLException {
          + final int typeId = res.getInt("DATA_TYPE");
          +
          + switch (typeId ) {
          + case Types.BOOLEAN:
          + return new TypeDesc(newSimpleDataType(Type.BOOLEAN));
          — End diff –

          As far as I know, JDBC drivers guarantees the type consistency, meaning that ResultSet will result in the same typed value for a certain JDBC data type regardless of kinds of database systems.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38728586 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcMetadataProviderBase.java — @@ -0,0 +1,253 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import com.google.common.base.Function; +import com.google.common.collect.Collections2; +import com.google.common.collect.Lists; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.tajo.catalog.*; +import org.apache.tajo.catalog.statistics.TableStats; +import org.apache.tajo.common.TajoDataTypes.Type; +import org.apache.tajo.exception.*; +import org.apache.tajo.util.KeyValueSet; +import org.apache.tajo.util.Pair; + +import javax.annotation.Nullable; +import java.net.URI; +import java.sql.*; +import java.util.Collection; +import java.util.Collections; +import java.util.Comparator; +import java.util.List; + +import static org.apache.tajo.catalog.CatalogUtil.newSimpleDataType; + +public abstract class JdbcMetadataProviderBase implements MetadataProvider { + protected static final Log LOG = LogFactory.getLog(JdbcMetadataProviderBase.class); + + protected final JdbcTablespace space; + protected final String databaseName; + + protected final String jdbcUri; + protected final String username; + protected final String password; + + protected final Connection connection; + + public JdbcMetadataProviderBase(JdbcTablespace space, String dbName) { + this.space = space; + this.databaseName = dbName; + + ConnectionInfo connInfo = ConnectionInfo.fromURI(space.getUri()); + this.jdbcUri = space.getUri().toASCIIString(); + this.username = connInfo.user(); + this.password = connInfo.password(); + + try { + Class.forName(getJdbcDriverName()).newInstance(); + LOG.info(getJdbcDriverName() + " is loaded..."); + } catch (Exception e) { + throw new TajoInternalError(e); + } + + try { + connection = DriverManager.getConnection(jdbcUri, this.username, this.password); + } catch (SQLException e) { + throw new TajoInternalError(e); + } + } + + @Override + public String getTablespaceName() { + return space.getName(); + } + + @Override + public URI getTablespaceUri() { + return space.getUri(); + } + + @Override + public String getDatabaseName() { + return databaseName; + } + + @Override + public Collection<String> getSchemas() { + return Collections.EMPTY_SET; + } + + @Override + public Collection<String> getTables(@Nullable String schemaPattern, @Nullable String tablePattern) { + ResultSet res = null; + List<String> tableNames = Lists.newArrayList(); + try { + res = connection.getMetaData().getTables(databaseName, schemaPattern, tablePattern, null); + while(res.next()) { + tableNames.add(res.getString("TABLE_NAME")); + } + } catch (SQLException e) { + throw new TajoInternalError(e); + } finally { + try { + if (res != null) { + res.close(); + } + } catch (SQLException e) { + LOG.warn(e); + } + } + + return tableNames; + } + + private TypeDesc convertDataType(ResultSet res) throws SQLException { + final int typeId = res.getInt("DATA_TYPE"); + + switch (typeId ) { + case Types.BOOLEAN: + return new TypeDesc(newSimpleDataType(Type.BOOLEAN)); — End diff – As far as I know, JDBC drivers guarantees the type consistency, meaning that ResultSet will result in the same typed value for a certain JDBC data type regardless of kinds of database systems.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38728340

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcFragment.java —
          @@ -0,0 +1,106 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import com.google.protobuf.ByteString;
          +import com.google.protobuf.InvalidProtocolBufferException;
          +import org.apache.tajo.catalog.proto.CatalogProtos;
          +import org.apache.tajo.storage.fragment.Fragment;
          +import org.apache.tajo.storage.jdbc.JdbcFragmentProtos.JdbcFragmentProto;
          +import org.apache.tajo.util.TUtil;
          +
          +public class JdbcFragment implements Fragment, Comparable<JdbcFragment>, Cloneable {
          + String uri;
          + String inputSourceId;
          + String [] hostNames;
          +
          +
          + public JdbcFragment(ByteString raw) throws InvalidProtocolBufferException

          { + JdbcFragmentProto.Builder builder = JdbcFragmentProto.newBuilder(); + builder.mergeFrom(raw); + builder.build(); + init(builder.build()); + }

          +
          + public JdbcFragment(String inputSourceId, String uri)

          { + this.inputSourceId = inputSourceId; + this.uri = uri; + this.hostNames = extractHosts(uri); + }

          +
          + private void init(JdbcFragmentProto proto)

          { + this.uri = proto.getUri(); + this.inputSourceId = proto.getInputSourceId(); + this.hostNames = proto.getHostsList().toArray(new String [proto.getHostsCount()]); + }

          +
          + private String [] extractHosts(String uri) {
          + return new String [] {};
          — End diff –

          Fixed. Thanks!

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38728340 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcFragment.java — @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import com.google.protobuf.ByteString; +import com.google.protobuf.InvalidProtocolBufferException; +import org.apache.tajo.catalog.proto.CatalogProtos; +import org.apache.tajo.storage.fragment.Fragment; +import org.apache.tajo.storage.jdbc.JdbcFragmentProtos.JdbcFragmentProto; +import org.apache.tajo.util.TUtil; + +public class JdbcFragment implements Fragment, Comparable<JdbcFragment>, Cloneable { + String uri; + String inputSourceId; + String [] hostNames; + + + public JdbcFragment(ByteString raw) throws InvalidProtocolBufferException { + JdbcFragmentProto.Builder builder = JdbcFragmentProto.newBuilder(); + builder.mergeFrom(raw); + builder.build(); + init(builder.build()); + } + + public JdbcFragment(String inputSourceId, String uri) { + this.inputSourceId = inputSourceId; + this.uri = uri; + this.hostNames = extractHosts(uri); + } + + private void init(JdbcFragmentProto proto) { + this.uri = proto.getUri(); + this.inputSourceId = proto.getInputSourceId(); + this.hostNames = proto.getHostsList().toArray(new String [proto.getHostsCount()]); + } + + private String [] extractHosts(String uri) { + return new String [] {}; — End diff – Fixed. Thanks!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38728091

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java —
          @@ -0,0 +1,112 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import org.apache.tajo.exception.TajoInternalError;
          +
          +import java.net.URI;
          +import java.util.HashMap;
          +import java.util.Map;
          +
          +public class ConnectionInfo {
          + String scheme;
          + String dbName;
          + String tableName;
          + String user;
          + String password;
          + Map<String, String> params;
          +
          + public String scheme()

          { + return scheme; + }

          +
          + public String database()

          { + return dbName; + }

          +
          + public String table()

          { + return tableName; + }

          +
          + public String user()

          { + return user; + }

          +
          + public String password()

          { + return password; + }

          +
          + public static ConnectionInfo fromURI(String originalUri)

          { + return fromURI(URI.create(originalUri)); + }

          +
          + public static ConnectionInfo fromURI(URI originalUri) {
          + final String uriStr = originalUri.toASCIIString();
          + URI uri = originalUri;
          +
          + final ConnectionInfo connInfo = new ConnectionInfo();
          + connInfo.scheme = uriStr.substring(0, uriStr.indexOf("://"));
          +
          + if (connInfo.scheme.split(":").length > 1)

          { + int idx = uriStr.indexOf(':'); + uri = URI.create(uriStr.substring(idx + 1)); + }

          +
          + String path = uri.getPath();
          + if (path != null && !path.isEmpty()) {
          + String [] pathElements = path.substring(1).split("/");
          + if (pathElements.length != 1)

          { + throw new TajoInternalError("Invalid JDBC path: " + path); + }

          + connInfo.dbName = pathElements[0];
          + }
          +
          + Map<String, String> params = new HashMap<>();
          +
          + int paramIndex = uriStr.indexOf("?");
          + if (paramIndex > 0) {
          + String parameterPart = uriStr.substring(paramIndex+1, uriStr.length());
          +
          + String [] eachParam = parameterPart.split("&");
          +
          + for (String each: eachParam) {
          + String [] keyValues = each.split("=");
          + if (keyValues.length != 2)

          { + throw new TajoInternalError("Invalid URI Parameters: " + parameterPart); + }

          + params.put(keyValues[0], keyValues[1]);
          + }
          + }
          +
          + if (params.containsKey("table")) {
          — End diff –

          In my opinion, literal string is more intuitive and they are used only here. So, static constants seems to be too verbose to me.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38728091 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java — @@ -0,0 +1,112 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import org.apache.tajo.exception.TajoInternalError; + +import java.net.URI; +import java.util.HashMap; +import java.util.Map; + +public class ConnectionInfo { + String scheme; + String dbName; + String tableName; + String user; + String password; + Map<String, String> params; + + public String scheme() { + return scheme; + } + + public String database() { + return dbName; + } + + public String table() { + return tableName; + } + + public String user() { + return user; + } + + public String password() { + return password; + } + + public static ConnectionInfo fromURI(String originalUri) { + return fromURI(URI.create(originalUri)); + } + + public static ConnectionInfo fromURI(URI originalUri) { + final String uriStr = originalUri.toASCIIString(); + URI uri = originalUri; + + final ConnectionInfo connInfo = new ConnectionInfo(); + connInfo.scheme = uriStr.substring(0, uriStr.indexOf("://")); + + if (connInfo.scheme.split(":").length > 1) { + int idx = uriStr.indexOf(':'); + uri = URI.create(uriStr.substring(idx + 1)); + } + + String path = uri.getPath(); + if (path != null && !path.isEmpty()) { + String [] pathElements = path.substring(1).split("/"); + if (pathElements.length != 1) { + throw new TajoInternalError("Invalid JDBC path: " + path); + } + connInfo.dbName = pathElements [0] ; + } + + Map<String, String> params = new HashMap<>(); + + int paramIndex = uriStr.indexOf("?"); + if (paramIndex > 0) { + String parameterPart = uriStr.substring(paramIndex+1, uriStr.length()); + + String [] eachParam = parameterPart.split("&"); + + for (String each: eachParam) { + String [] keyValues = each.split("="); + if (keyValues.length != 2) { + throw new TajoInternalError("Invalid URI Parameters: " + parameterPart); + } + params.put(keyValues [0] , keyValues [1] ); + } + } + + if (params.containsKey("table")) { — End diff – In my opinion, literal string is more intuitive and they are used only here. So, static constants seems to be too verbose to me.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38727547

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcTablespace.java —
          @@ -0,0 +1,161 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import com.google.common.collect.Lists;
          +import net.minidev.json.JSONObject;
          +import org.apache.hadoop.fs.Path;
          +import org.apache.tajo.ExecutionBlockId;
          +import org.apache.tajo.OverridableConf;
          +import org.apache.tajo.catalog.*;
          +import org.apache.tajo.exception.NotImplementedException;
          +import org.apache.tajo.exception.TajoInternalError;
          +import org.apache.tajo.exception.TajoRuntimeException;
          +import org.apache.tajo.exception.UnsupportedException;
          +import org.apache.tajo.plan.LogicalPlan;
          +import org.apache.tajo.plan.expr.EvalNode;
          +import org.apache.tajo.plan.logical.LogicalNode;
          +import org.apache.tajo.storage.*;
          +import org.apache.tajo.storage.fragment.Fragment;
          +
          +import javax.annotation.Nullable;
          +import java.io.IOException;
          +import java.net.URI;
          +import java.sql.Connection;
          +import java.sql.DatabaseMetaData;
          +import java.sql.DriverManager;
          +import java.sql.SQLException;
          +import java.util.List;
          +
          +/**
          + * JDBC Tablespace
          + */
          +public abstract class JdbcTablespace extends Tablespace {
          +
          + static final StorageProperty STORAGE_PROPERTY = new StorageProperty("rowstore", false, true, false, true);
          + static final FormatProperty FORMAT_PROPERTY = new FormatProperty(false, false, false);
          + public static final String MAPPED_DATABASE_CONFIG_KEY = "mapped_database";
          +
          + private Connection conn;
          +
          + public JdbcTablespace(String name, URI uri, JSONObject config)

          { + super(name, uri, config); + }

          +
          + @Override
          + protected void storageInit() throws IOException {
          + try

          { + this.conn = DriverManager.getConnection(uri.toASCIIString()); + }

          catch (SQLException e)

          { + throw new IOException(e); + }

          + }
          +
          + @Override
          + public long getTableVolume(URI uri) throws UnsupportedException

          { + throw new UnsupportedException(); + }

          +
          + @Override
          + public URI getTableUri(String databaseName, String tableName)

          { + return URI.create(getUri() + "&table=" + tableName); + }

          +
          + @Override
          + public List<Fragment> getSplits(String inputSourceId,
          + TableDesc tableDesc,
          + @Nullable EvalNode filterCondition) throws IOException

          { + return Lists.newArrayList((Fragment)new JdbcFragment(inputSourceId, tableDesc.getUri().toASCIIString())); + }

          +
          + @Override
          + public StorageProperty getProperty()

          { + return STORAGE_PROPERTY; + }

          +
          + @Override
          + public FormatProperty getFormatProperty(TableMeta meta)

          { + return FORMAT_PROPERTY; + }

          +
          + @Override
          + public void close() {
          — End diff –

          Would you share why ```conn``` is not closed?

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38727547 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcTablespace.java — @@ -0,0 +1,161 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import com.google.common.collect.Lists; +import net.minidev.json.JSONObject; +import org.apache.hadoop.fs.Path; +import org.apache.tajo.ExecutionBlockId; +import org.apache.tajo.OverridableConf; +import org.apache.tajo.catalog.*; +import org.apache.tajo.exception.NotImplementedException; +import org.apache.tajo.exception.TajoInternalError; +import org.apache.tajo.exception.TajoRuntimeException; +import org.apache.tajo.exception.UnsupportedException; +import org.apache.tajo.plan.LogicalPlan; +import org.apache.tajo.plan.expr.EvalNode; +import org.apache.tajo.plan.logical.LogicalNode; +import org.apache.tajo.storage.*; +import org.apache.tajo.storage.fragment.Fragment; + +import javax.annotation.Nullable; +import java.io.IOException; +import java.net.URI; +import java.sql.Connection; +import java.sql.DatabaseMetaData; +import java.sql.DriverManager; +import java.sql.SQLException; +import java.util.List; + +/** + * JDBC Tablespace + */ +public abstract class JdbcTablespace extends Tablespace { + + static final StorageProperty STORAGE_PROPERTY = new StorageProperty("rowstore", false, true, false, true); + static final FormatProperty FORMAT_PROPERTY = new FormatProperty(false, false, false); + public static final String MAPPED_DATABASE_CONFIG_KEY = "mapped_database"; + + private Connection conn; + + public JdbcTablespace(String name, URI uri, JSONObject config) { + super(name, uri, config); + } + + @Override + protected void storageInit() throws IOException { + try { + this.conn = DriverManager.getConnection(uri.toASCIIString()); + } catch (SQLException e) { + throw new IOException(e); + } + } + + @Override + public long getTableVolume(URI uri) throws UnsupportedException { + throw new UnsupportedException(); + } + + @Override + public URI getTableUri(String databaseName, String tableName) { + return URI.create(getUri() + "&table=" + tableName); + } + + @Override + public List<Fragment> getSplits(String inputSourceId, + TableDesc tableDesc, + @Nullable EvalNode filterCondition) throws IOException { + return Lists.newArrayList((Fragment)new JdbcFragment(inputSourceId, tableDesc.getUri().toASCIIString())); + } + + @Override + public StorageProperty getProperty() { + return STORAGE_PROPERTY; + } + + @Override + public FormatProperty getFormatProperty(TableMeta meta) { + return FORMAT_PROPERTY; + } + + @Override + public void close() { — End diff – Would you share why ```conn``` is not closed?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38727469

          — Diff: tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseTablespace.java —
          @@ -93,21 +94,13 @@ public void storageInit() throws IOException

          { hbaseConf.set(HConstants.ZOOKEEPER_CLIENT_PORT, splits[1]); }
          • @Override
          • public void setConfig(String name, String value) { - }
            -
            - @Override
            - public void setConfigs(Map<String, String> configs) { - }

            -
            public Configuration getHbaseConf()

            { return hbaseConf; }

          @Override

          • public long getTableVolume(URI uri) throws IOException {
          • return 0;
            + public long getTableVolume(URI uri) throws UnsupportedException {
            + throw new UnsupportedException();
              • End diff –

          NotImplemented and UnsupportedException seems to be similar but they are different. NotImplemented means that we will implement it later. UnsupportedException means that this feature is not available.

          With Hbase, there is no way to make a guess about table volume. So, we should keep UnsupportedException here.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38727469 — Diff: tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseTablespace.java — @@ -93,21 +94,13 @@ public void storageInit() throws IOException { hbaseConf.set(HConstants.ZOOKEEPER_CLIENT_PORT, splits[1]); } @Override public void setConfig(String name, String value) { - } - - @Override - public void setConfigs(Map<String, String> configs) { - } - public Configuration getHbaseConf() { return hbaseConf; } @Override public long getTableVolume(URI uri) throws IOException { return 0; + public long getTableVolume(URI uri) throws UnsupportedException { + throw new UnsupportedException(); End diff – NotImplemented and UnsupportedException seems to be similar but they are different. NotImplemented means that we will implement it later. UnsupportedException means that this feature is not available. With Hbase, there is no way to make a guess about table volume. So, we should keep UnsupportedException here.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38727277

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java —
          @@ -0,0 +1,313 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import com.google.common.base.Preconditions;
          +import org.apache.commons.logging.Log;
          +import org.apache.commons.logging.LogFactory;
          +import org.apache.tajo.catalog.Column;
          +import org.apache.tajo.catalog.Schema;
          +import org.apache.tajo.catalog.TableMeta;
          +import org.apache.tajo.catalog.statistics.TableStats;
          +import org.apache.tajo.datum.DatumFactory;
          +import org.apache.tajo.datum.TimeDatum;
          +import org.apache.tajo.exception.TajoInternalError;
          +import org.apache.tajo.exception.TajoRuntimeException;
          +import org.apache.tajo.exception.UnsupportedDataTypeException;
          +import org.apache.tajo.exception.UnsupportedException;
          +import org.apache.tajo.plan.expr.EvalNode;
          +import org.apache.tajo.plan.logical.LogicalNode;
          +import org.apache.tajo.storage.Scanner;
          +import org.apache.tajo.storage.Tuple;
          +import org.apache.tajo.storage.VTuple;
          +
          +import java.io.Closeable;
          +import java.io.IOException;
          +import java.sql.*;
          +import java.util.Iterator;
          +
          +public abstract class JdbcScanner implements Scanner {
          + private static final Log LOG = LogFactory.getLog(JdbcScanner.class);
          +
          + protected final DatabaseMetaData dbMetaData;
          + protected final String tableName;
          + protected final Schema schema;
          + protected final TableMeta tableMeta;
          + protected final JdbcFragment fragment;
          + protected final TableStats stats;
          — End diff –

          ```stats``` is not updated properly.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38727277 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java — @@ -0,0 +1,313 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import com.google.common.base.Preconditions; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.tajo.catalog.Column; +import org.apache.tajo.catalog.Schema; +import org.apache.tajo.catalog.TableMeta; +import org.apache.tajo.catalog.statistics.TableStats; +import org.apache.tajo.datum.DatumFactory; +import org.apache.tajo.datum.TimeDatum; +import org.apache.tajo.exception.TajoInternalError; +import org.apache.tajo.exception.TajoRuntimeException; +import org.apache.tajo.exception.UnsupportedDataTypeException; +import org.apache.tajo.exception.UnsupportedException; +import org.apache.tajo.plan.expr.EvalNode; +import org.apache.tajo.plan.logical.LogicalNode; +import org.apache.tajo.storage.Scanner; +import org.apache.tajo.storage.Tuple; +import org.apache.tajo.storage.VTuple; + +import java.io.Closeable; +import java.io.IOException; +import java.sql.*; +import java.util.Iterator; + +public abstract class JdbcScanner implements Scanner { + private static final Log LOG = LogFactory.getLog(JdbcScanner.class); + + protected final DatabaseMetaData dbMetaData; + protected final String tableName; + protected final Schema schema; + protected final TableMeta tableMeta; + protected final JdbcFragment fragment; + protected final TableStats stats; — End diff – ```stats``` is not updated properly.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38727232

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java —
          @@ -0,0 +1,313 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import com.google.common.base.Preconditions;
          +import org.apache.commons.logging.Log;
          +import org.apache.commons.logging.LogFactory;
          +import org.apache.tajo.catalog.Column;
          +import org.apache.tajo.catalog.Schema;
          +import org.apache.tajo.catalog.TableMeta;
          +import org.apache.tajo.catalog.statistics.TableStats;
          +import org.apache.tajo.datum.DatumFactory;
          +import org.apache.tajo.datum.TimeDatum;
          +import org.apache.tajo.exception.TajoInternalError;
          +import org.apache.tajo.exception.TajoRuntimeException;
          +import org.apache.tajo.exception.UnsupportedDataTypeException;
          +import org.apache.tajo.exception.UnsupportedException;
          +import org.apache.tajo.plan.expr.EvalNode;
          +import org.apache.tajo.plan.logical.LogicalNode;
          +import org.apache.tajo.storage.Scanner;
          +import org.apache.tajo.storage.Tuple;
          +import org.apache.tajo.storage.VTuple;
          +
          +import java.io.Closeable;
          +import java.io.IOException;
          +import java.sql.*;
          +import java.util.Iterator;
          +
          +public abstract class JdbcScanner implements Scanner {
          + private static final Log LOG = LogFactory.getLog(JdbcScanner.class);
          +
          + protected final DatabaseMetaData dbMetaData;
          + protected final String tableName;
          + protected final Schema schema;
          + protected final TableMeta tableMeta;
          + protected final JdbcFragment fragment;
          + protected final TableStats stats;
          + protected final SQLBuilder builder;
          — End diff –

          ```builder``` don't have to be a member variable.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38727232 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java — @@ -0,0 +1,313 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import com.google.common.base.Preconditions; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.tajo.catalog.Column; +import org.apache.tajo.catalog.Schema; +import org.apache.tajo.catalog.TableMeta; +import org.apache.tajo.catalog.statistics.TableStats; +import org.apache.tajo.datum.DatumFactory; +import org.apache.tajo.datum.TimeDatum; +import org.apache.tajo.exception.TajoInternalError; +import org.apache.tajo.exception.TajoRuntimeException; +import org.apache.tajo.exception.UnsupportedDataTypeException; +import org.apache.tajo.exception.UnsupportedException; +import org.apache.tajo.plan.expr.EvalNode; +import org.apache.tajo.plan.logical.LogicalNode; +import org.apache.tajo.storage.Scanner; +import org.apache.tajo.storage.Tuple; +import org.apache.tajo.storage.VTuple; + +import java.io.Closeable; +import java.io.IOException; +import java.sql.*; +import java.util.Iterator; + +public abstract class JdbcScanner implements Scanner { + private static final Log LOG = LogFactory.getLog(JdbcScanner.class); + + protected final DatabaseMetaData dbMetaData; + protected final String tableName; + protected final Schema schema; + protected final TableMeta tableMeta; + protected final JdbcFragment fragment; + protected final TableStats stats; + protected final SQLBuilder builder; — End diff – ```builder``` don't have to be a member variable.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38727175

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java —
          @@ -0,0 +1,313 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import com.google.common.base.Preconditions;
          +import org.apache.commons.logging.Log;
          +import org.apache.commons.logging.LogFactory;
          +import org.apache.tajo.catalog.Column;
          +import org.apache.tajo.catalog.Schema;
          +import org.apache.tajo.catalog.TableMeta;
          +import org.apache.tajo.catalog.statistics.TableStats;
          +import org.apache.tajo.datum.DatumFactory;
          +import org.apache.tajo.datum.TimeDatum;
          +import org.apache.tajo.exception.TajoInternalError;
          +import org.apache.tajo.exception.TajoRuntimeException;
          +import org.apache.tajo.exception.UnsupportedDataTypeException;
          +import org.apache.tajo.exception.UnsupportedException;
          +import org.apache.tajo.plan.expr.EvalNode;
          +import org.apache.tajo.plan.logical.LogicalNode;
          +import org.apache.tajo.storage.Scanner;
          +import org.apache.tajo.storage.Tuple;
          +import org.apache.tajo.storage.VTuple;
          +
          +import java.io.Closeable;
          +import java.io.IOException;
          +import java.sql.*;
          +import java.util.Iterator;
          +
          +public abstract class JdbcScanner implements Scanner {
          + private static final Log LOG = LogFactory.getLog(JdbcScanner.class);
          +
          + protected final DatabaseMetaData dbMetaData;
          + protected final String tableName;
          + protected final Schema schema;
          + protected final TableMeta tableMeta;
          + protected final JdbcFragment fragment;
          + protected final TableStats stats;
          + protected final SQLBuilder builder;
          +
          + protected Column [] targets;
          + protected EvalNode filter;
          + protected Long limit;
          + protected LogicalNode planPart;
          + protected VTuple outTuple;
          + protected String generatedSql;
          + protected ResultSetIterator iter;
          +
          + public JdbcScanner(final DatabaseMetaData dbMetaData,
          + final Schema tableSchema,
          + final TableMeta tableMeta,
          + final JdbcFragment fragment)

          { + + Preconditions.checkNotNull(dbMetaData); + Preconditions.checkNotNull(tableSchema); + Preconditions.checkNotNull(tableMeta); + Preconditions.checkNotNull(fragment); + + this.dbMetaData = dbMetaData; + this.tableName = ConnectionInfo.fromURI(fragment.getUri()).tableName; + this.schema = tableSchema; + this.tableMeta = tableMeta; + this.fragment = fragment; + this.stats = new TableStats(); + builder = getSQLBuilder(getSQLExprBuilder()); + }

          +
          + @Override
          + public void init() throws IOException {
          + if (targets == null)

          { + targets = schema.toArray(); + }

          + outTuple = new VTuple(targets.length);
          +
          + if (planPart == null)

          { + generatedSql = builder.build(tableName, targets, filter, limit); + }

          else

          { + generatedSql = builder.build(planPart); + }

          + }
          +
          + @Override
          + public Tuple next() throws IOException {
          + if (iter == null)

          { + iter = executeQueryAndGetIter(); + }

          +
          + if (iter.hasNext())

          { + return iter.next(); + }

          else

          { + return null; + }

          + }
          +
          + @Override
          + public void reset() throws IOException {
          + if (iter != null)

          { + iter.rewind(); + }

          + }
          +
          + @Override
          + public void close() throws IOException {
          + if (iter != null)

          { + iter.close(); + }

          + }
          +
          + @Override
          + public void pushOperators(LogicalNode planPart)

          { + this.planPart = planPart; + }

          +
          +
          + @Override
          + public boolean isProjectable()

          { + return true; + }
          +
          + @Override
          + public void setTarget(Column [] targets) { + this.targets = targets; + }
          +
          + @Override
          + public boolean isSelectable() { + return true; + }

          +
          + @Override
          + public void setFilter(EvalNode filter)

          { + this.filter = filter; + }

          +
          + @Override
          + public void setLimit(long num)

          { + this.limit = num; + }

          +
          + @Override
          + public boolean isSplittable()

          { + return false; + }

          +
          + @Override
          + public float getProgress()

          { + return 0; + }

          +
          + @Override
          + public TableStats getInputStats()

          { + return stats; + }

          +
          + @Override
          + public Schema getSchema()

          { + return schema; + }

          +
          + protected SQLBuilder getSQLBuilder(SQLExpressionGenerator exprBuilder) {
          + return new SQLBuilder(dbMetaData, getSQLExprBuilder());
          — End diff –

          ```exprBuilder``` parameter is not used.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38727175 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcScanner.java — @@ -0,0 +1,313 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import com.google.common.base.Preconditions; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.tajo.catalog.Column; +import org.apache.tajo.catalog.Schema; +import org.apache.tajo.catalog.TableMeta; +import org.apache.tajo.catalog.statistics.TableStats; +import org.apache.tajo.datum.DatumFactory; +import org.apache.tajo.datum.TimeDatum; +import org.apache.tajo.exception.TajoInternalError; +import org.apache.tajo.exception.TajoRuntimeException; +import org.apache.tajo.exception.UnsupportedDataTypeException; +import org.apache.tajo.exception.UnsupportedException; +import org.apache.tajo.plan.expr.EvalNode; +import org.apache.tajo.plan.logical.LogicalNode; +import org.apache.tajo.storage.Scanner; +import org.apache.tajo.storage.Tuple; +import org.apache.tajo.storage.VTuple; + +import java.io.Closeable; +import java.io.IOException; +import java.sql.*; +import java.util.Iterator; + +public abstract class JdbcScanner implements Scanner { + private static final Log LOG = LogFactory.getLog(JdbcScanner.class); + + protected final DatabaseMetaData dbMetaData; + protected final String tableName; + protected final Schema schema; + protected final TableMeta tableMeta; + protected final JdbcFragment fragment; + protected final TableStats stats; + protected final SQLBuilder builder; + + protected Column [] targets; + protected EvalNode filter; + protected Long limit; + protected LogicalNode planPart; + protected VTuple outTuple; + protected String generatedSql; + protected ResultSetIterator iter; + + public JdbcScanner(final DatabaseMetaData dbMetaData, + final Schema tableSchema, + final TableMeta tableMeta, + final JdbcFragment fragment) { + + Preconditions.checkNotNull(dbMetaData); + Preconditions.checkNotNull(tableSchema); + Preconditions.checkNotNull(tableMeta); + Preconditions.checkNotNull(fragment); + + this.dbMetaData = dbMetaData; + this.tableName = ConnectionInfo.fromURI(fragment.getUri()).tableName; + this.schema = tableSchema; + this.tableMeta = tableMeta; + this.fragment = fragment; + this.stats = new TableStats(); + builder = getSQLBuilder(getSQLExprBuilder()); + } + + @Override + public void init() throws IOException { + if (targets == null) { + targets = schema.toArray(); + } + outTuple = new VTuple(targets.length); + + if (planPart == null) { + generatedSql = builder.build(tableName, targets, filter, limit); + } else { + generatedSql = builder.build(planPart); + } + } + + @Override + public Tuple next() throws IOException { + if (iter == null) { + iter = executeQueryAndGetIter(); + } + + if (iter.hasNext()) { + return iter.next(); + } else { + return null; + } + } + + @Override + public void reset() throws IOException { + if (iter != null) { + iter.rewind(); + } + } + + @Override + public void close() throws IOException { + if (iter != null) { + iter.close(); + } + } + + @Override + public void pushOperators(LogicalNode planPart) { + this.planPart = planPart; + } + + + @Override + public boolean isProjectable() { + return true; + } + + @Override + public void setTarget(Column [] targets) { + this.targets = targets; + } + + @Override + public boolean isSelectable() { + return true; + } + + @Override + public void setFilter(EvalNode filter) { + this.filter = filter; + } + + @Override + public void setLimit(long num) { + this.limit = num; + } + + @Override + public boolean isSplittable() { + return false; + } + + @Override + public float getProgress() { + return 0; + } + + @Override + public TableStats getInputStats() { + return stats; + } + + @Override + public Schema getSchema() { + return schema; + } + + protected SQLBuilder getSQLBuilder(SQLExpressionGenerator exprBuilder) { + return new SQLBuilder(dbMetaData, getSQLExprBuilder()); — End diff – ```exprBuilder``` parameter is not used.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38726773

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcMetadataProviderBase.java —
          @@ -0,0 +1,253 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import com.google.common.base.Function;
          +import com.google.common.collect.Collections2;
          +import com.google.common.collect.Lists;
          +import org.apache.commons.logging.Log;
          +import org.apache.commons.logging.LogFactory;
          +import org.apache.tajo.catalog.*;
          +import org.apache.tajo.catalog.statistics.TableStats;
          +import org.apache.tajo.common.TajoDataTypes.Type;
          +import org.apache.tajo.exception.*;
          +import org.apache.tajo.util.KeyValueSet;
          +import org.apache.tajo.util.Pair;
          +
          +import javax.annotation.Nullable;
          +import java.net.URI;
          +import java.sql.*;
          +import java.util.Collection;
          +import java.util.Collections;
          +import java.util.Comparator;
          +import java.util.List;
          +
          +import static org.apache.tajo.catalog.CatalogUtil.newSimpleDataType;
          +
          +public abstract class JdbcMetadataProviderBase implements MetadataProvider {
          + protected static final Log LOG = LogFactory.getLog(JdbcMetadataProviderBase.class);
          +
          + protected final JdbcTablespace space;
          + protected final String databaseName;
          +
          + protected final String jdbcUri;
          + protected final String username;
          + protected final String password;
          +
          + protected final Connection connection;
          +
          + public JdbcMetadataProviderBase(JdbcTablespace space, String dbName) {
          + this.space = space;
          + this.databaseName = dbName;
          +
          + ConnectionInfo connInfo = ConnectionInfo.fromURI(space.getUri());
          + this.jdbcUri = space.getUri().toASCIIString();
          + this.username = connInfo.user();
          + this.password = connInfo.password();
          +
          + try

          { + Class.forName(getJdbcDriverName()).newInstance(); + LOG.info(getJdbcDriverName() + " is loaded..."); + }

          catch (Exception e)

          { + throw new TajoInternalError(e); + }
          +
          + try { + connection = DriverManager.getConnection(jdbcUri, this.username, this.password); + } catch (SQLException e) { + throw new TajoInternalError(e); + }

          + }
          +
          + @Override
          + public String getTablespaceName()

          { + return space.getName(); + }

          +
          + @Override
          + public URI getTablespaceUri()

          { + return space.getUri(); + }

          +
          + @Override
          + public String getDatabaseName()

          { + return databaseName; + }

          +
          + @Override
          + public Collection<String> getSchemas()

          { + return Collections.EMPTY_SET; + }

          +
          + @Override
          + public Collection<String> getTables(@Nullable String schemaPattern, @Nullable String tablePattern) {
          + ResultSet res = null;
          + List<String> tableNames = Lists.newArrayList();
          + try {
          + res = connection.getMetaData().getTables(databaseName, schemaPattern, tablePattern, null);
          + while(res.next())

          { + tableNames.add(res.getString("TABLE_NAME")); + }

          + } catch (SQLException e)

          { + throw new TajoInternalError(e); + }

          finally {
          + try {
          + if (res != null)

          { + res.close(); + }

          + } catch (SQLException e)

          { + LOG.warn(e); + }

          + }
          +
          + return tableNames;
          + }
          +
          + private TypeDesc convertDataType(ResultSet res) throws SQLException {
          + final int typeId = res.getInt("DATA_TYPE");
          +
          + switch (typeId ) {
          + case Types.BOOLEAN:
          + return new TypeDesc(newSimpleDataType(Type.BOOLEAN));
          — End diff –

          This conversion code currently looks good, but should be improved later to support various types of database systems.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38726773 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcMetadataProviderBase.java — @@ -0,0 +1,253 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import com.google.common.base.Function; +import com.google.common.collect.Collections2; +import com.google.common.collect.Lists; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.tajo.catalog.*; +import org.apache.tajo.catalog.statistics.TableStats; +import org.apache.tajo.common.TajoDataTypes.Type; +import org.apache.tajo.exception.*; +import org.apache.tajo.util.KeyValueSet; +import org.apache.tajo.util.Pair; + +import javax.annotation.Nullable; +import java.net.URI; +import java.sql.*; +import java.util.Collection; +import java.util.Collections; +import java.util.Comparator; +import java.util.List; + +import static org.apache.tajo.catalog.CatalogUtil.newSimpleDataType; + +public abstract class JdbcMetadataProviderBase implements MetadataProvider { + protected static final Log LOG = LogFactory.getLog(JdbcMetadataProviderBase.class); + + protected final JdbcTablespace space; + protected final String databaseName; + + protected final String jdbcUri; + protected final String username; + protected final String password; + + protected final Connection connection; + + public JdbcMetadataProviderBase(JdbcTablespace space, String dbName) { + this.space = space; + this.databaseName = dbName; + + ConnectionInfo connInfo = ConnectionInfo.fromURI(space.getUri()); + this.jdbcUri = space.getUri().toASCIIString(); + this.username = connInfo.user(); + this.password = connInfo.password(); + + try { + Class.forName(getJdbcDriverName()).newInstance(); + LOG.info(getJdbcDriverName() + " is loaded..."); + } catch (Exception e) { + throw new TajoInternalError(e); + } + + try { + connection = DriverManager.getConnection(jdbcUri, this.username, this.password); + } catch (SQLException e) { + throw new TajoInternalError(e); + } + } + + @Override + public String getTablespaceName() { + return space.getName(); + } + + @Override + public URI getTablespaceUri() { + return space.getUri(); + } + + @Override + public String getDatabaseName() { + return databaseName; + } + + @Override + public Collection<String> getSchemas() { + return Collections.EMPTY_SET; + } + + @Override + public Collection<String> getTables(@Nullable String schemaPattern, @Nullable String tablePattern) { + ResultSet res = null; + List<String> tableNames = Lists.newArrayList(); + try { + res = connection.getMetaData().getTables(databaseName, schemaPattern, tablePattern, null); + while(res.next()) { + tableNames.add(res.getString("TABLE_NAME")); + } + } catch (SQLException e) { + throw new TajoInternalError(e); + } finally { + try { + if (res != null) { + res.close(); + } + } catch (SQLException e) { + LOG.warn(e); + } + } + + return tableNames; + } + + private TypeDesc convertDataType(ResultSet res) throws SQLException { + final int typeId = res.getInt("DATA_TYPE"); + + switch (typeId ) { + case Types.BOOLEAN: + return new TypeDesc(newSimpleDataType(Type.BOOLEAN)); — End diff – This conversion code currently looks good, but should be improved later to support various types of database systems.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38725244

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcFragment.java —
          @@ -0,0 +1,106 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import com.google.protobuf.ByteString;
          +import com.google.protobuf.InvalidProtocolBufferException;
          +import org.apache.tajo.catalog.proto.CatalogProtos;
          +import org.apache.tajo.storage.fragment.Fragment;
          +import org.apache.tajo.storage.jdbc.JdbcFragmentProtos.JdbcFragmentProto;
          +import org.apache.tajo.util.TUtil;
          +
          +public class JdbcFragment implements Fragment, Comparable<JdbcFragment>, Cloneable {
          + String uri;
          + String inputSourceId;
          + String [] hostNames;
          +
          +
          + public JdbcFragment(ByteString raw) throws InvalidProtocolBufferException

          { + JdbcFragmentProto.Builder builder = JdbcFragmentProto.newBuilder(); + builder.mergeFrom(raw); + builder.build(); + init(builder.build()); + }

          +
          + public JdbcFragment(String inputSourceId, String uri)

          { + this.inputSourceId = inputSourceId; + this.uri = uri; + this.hostNames = extractHosts(uri); + }

          +
          + private void init(JdbcFragmentProto proto)

          { + this.uri = proto.getUri(); + this.inputSourceId = proto.getInputSourceId(); + this.hostNames = proto.getHostsList().toArray(new String [proto.getHostsCount()]); + }

          +
          + private String [] extractHosts(String uri) {
          + return new String [] {};
          — End diff –

          Missing codes.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38725244 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/JdbcFragment.java — @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import com.google.protobuf.ByteString; +import com.google.protobuf.InvalidProtocolBufferException; +import org.apache.tajo.catalog.proto.CatalogProtos; +import org.apache.tajo.storage.fragment.Fragment; +import org.apache.tajo.storage.jdbc.JdbcFragmentProtos.JdbcFragmentProto; +import org.apache.tajo.util.TUtil; + +public class JdbcFragment implements Fragment, Comparable<JdbcFragment>, Cloneable { + String uri; + String inputSourceId; + String [] hostNames; + + + public JdbcFragment(ByteString raw) throws InvalidProtocolBufferException { + JdbcFragmentProto.Builder builder = JdbcFragmentProto.newBuilder(); + builder.mergeFrom(raw); + builder.build(); + init(builder.build()); + } + + public JdbcFragment(String inputSourceId, String uri) { + this.inputSourceId = inputSourceId; + this.uri = uri; + this.hostNames = extractHosts(uri); + } + + private void init(JdbcFragmentProto proto) { + this.uri = proto.getUri(); + this.inputSourceId = proto.getInputSourceId(); + this.hostNames = proto.getHostsList().toArray(new String [proto.getHostsCount()]); + } + + private String [] extractHosts(String uri) { + return new String [] {}; — End diff – Missing codes.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38725163

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java —
          @@ -0,0 +1,112 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import org.apache.tajo.exception.TajoInternalError;
          +
          +import java.net.URI;
          +import java.util.HashMap;
          +import java.util.Map;
          +
          +public class ConnectionInfo {
          + String scheme;
          + String dbName;
          + String tableName;
          + String user;
          + String password;
          + Map<String, String> params;
          +
          + public String scheme()

          { + return scheme; + }

          +
          + public String database()

          { + return dbName; + }

          +
          + public String table()

          { + return tableName; + }

          +
          + public String user()

          { + return user; + }

          +
          + public String password()

          { + return password; + }

          +
          + public static ConnectionInfo fromURI(String originalUri)

          { + return fromURI(URI.create(originalUri)); + }

          +
          + public static ConnectionInfo fromURI(URI originalUri) {
          + final String uriStr = originalUri.toASCIIString();
          + URI uri = originalUri;
          +
          + final ConnectionInfo connInfo = new ConnectionInfo();
          + connInfo.scheme = uriStr.substring(0, uriStr.indexOf("://"));
          +
          + if (connInfo.scheme.split(":").length > 1)

          { + int idx = uriStr.indexOf(':'); + uri = URI.create(uriStr.substring(idx + 1)); + }

          +
          + String path = uri.getPath();
          + if (path != null && !path.isEmpty()) {
          + String [] pathElements = path.substring(1).split("/");
          + if (pathElements.length != 1)

          { + throw new TajoInternalError("Invalid JDBC path: " + path); + }

          + connInfo.dbName = pathElements[0];
          + }
          +
          + Map<String, String> params = new HashMap<>();
          +
          + int paramIndex = uriStr.indexOf("?");
          + if (paramIndex > 0) {
          + String parameterPart = uriStr.substring(paramIndex+1, uriStr.length());
          +
          + String [] eachParam = parameterPart.split("&");
          +
          + for (String each: eachParam) {
          + String [] keyValues = each.split("=");
          + if (keyValues.length != 2)

          { + throw new TajoInternalError("Invalid URI Parameters: " + parameterPart); + }

          + params.put(keyValues[0], keyValues[1]);
          + }
          + }
          +
          + if (params.containsKey("table")) {
          — End diff –

          How about using static constants instead of the raw string?

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38725163 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java — @@ -0,0 +1,112 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import org.apache.tajo.exception.TajoInternalError; + +import java.net.URI; +import java.util.HashMap; +import java.util.Map; + +public class ConnectionInfo { + String scheme; + String dbName; + String tableName; + String user; + String password; + Map<String, String> params; + + public String scheme() { + return scheme; + } + + public String database() { + return dbName; + } + + public String table() { + return tableName; + } + + public String user() { + return user; + } + + public String password() { + return password; + } + + public static ConnectionInfo fromURI(String originalUri) { + return fromURI(URI.create(originalUri)); + } + + public static ConnectionInfo fromURI(URI originalUri) { + final String uriStr = originalUri.toASCIIString(); + URI uri = originalUri; + + final ConnectionInfo connInfo = new ConnectionInfo(); + connInfo.scheme = uriStr.substring(0, uriStr.indexOf("://")); + + if (connInfo.scheme.split(":").length > 1) { + int idx = uriStr.indexOf(':'); + uri = URI.create(uriStr.substring(idx + 1)); + } + + String path = uri.getPath(); + if (path != null && !path.isEmpty()) { + String [] pathElements = path.substring(1).split("/"); + if (pathElements.length != 1) { + throw new TajoInternalError("Invalid JDBC path: " + path); + } + connInfo.dbName = pathElements [0] ; + } + + Map<String, String> params = new HashMap<>(); + + int paramIndex = uriStr.indexOf("?"); + if (paramIndex > 0) { + String parameterPart = uriStr.substring(paramIndex+1, uriStr.length()); + + String [] eachParam = parameterPart.split("&"); + + for (String each: eachParam) { + String [] keyValues = each.split("="); + if (keyValues.length != 2) { + throw new TajoInternalError("Invalid URI Parameters: " + parameterPart); + } + params.put(keyValues [0] , keyValues [1] ); + } + } + + if (params.containsKey("table")) { — End diff – How about using static constants instead of the raw string?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38724978

          — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java —
          @@ -0,0 +1,112 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.storage.jdbc;
          +
          +import org.apache.tajo.exception.TajoInternalError;
          +
          +import java.net.URI;
          +import java.util.HashMap;
          +import java.util.Map;
          +
          +public class ConnectionInfo {
          + String scheme;
          + String dbName;
          + String tableName;
          + String user;
          + String password;
          + Map<String, String> params;
          +
          + public String scheme()

          { + return scheme; + }

          +
          + public String database()

          { + return dbName; + }

          +
          + public String table()

          { + return tableName; + }

          +
          + public String user()

          { + return user; + }

          +
          + public String password()

          { + return password; + }

          +
          + public static ConnectionInfo fromURI(String originalUri)

          { + return fromURI(URI.create(originalUri)); + }

          +
          + public static ConnectionInfo fromURI(URI originalUri) {
          + final String uriStr = originalUri.toASCIIString();
          + URI uri = originalUri;
          +
          + final ConnectionInfo connInfo = new ConnectionInfo();
          + connInfo.scheme = uriStr.substring(0, uriStr.indexOf("://"));
          — End diff –

          Please use UriUtil.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38724978 — Diff: tajo-storage/tajo-storage-jdbc/src/main/java/org/apache/tajo/storage/jdbc/ConnectionInfo.java — @@ -0,0 +1,112 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.storage.jdbc; + +import org.apache.tajo.exception.TajoInternalError; + +import java.net.URI; +import java.util.HashMap; +import java.util.Map; + +public class ConnectionInfo { + String scheme; + String dbName; + String tableName; + String user; + String password; + Map<String, String> params; + + public String scheme() { + return scheme; + } + + public String database() { + return dbName; + } + + public String table() { + return tableName; + } + + public String user() { + return user; + } + + public String password() { + return password; + } + + public static ConnectionInfo fromURI(String originalUri) { + return fromURI(URI.create(originalUri)); + } + + public static ConnectionInfo fromURI(URI originalUri) { + final String uriStr = originalUri.toASCIIString(); + URI uri = originalUri; + + final ConnectionInfo connInfo = new ConnectionInfo(); + connInfo.scheme = uriStr.substring(0, uriStr.indexOf("://")); — End diff – Please use UriUtil.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38723199

          — Diff: tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseTablespace.java —
          @@ -93,21 +94,13 @@ public void storageInit() throws IOException

          { hbaseConf.set(HConstants.ZOOKEEPER_CLIENT_PORT, splits[1]); }
          • @Override
          • public void setConfig(String name, String value) { - }
            -
            - @Override
            - public void setConfigs(Map<String, String> configs) { - }

            -
            public Configuration getHbaseConf()

            { return hbaseConf; }

          @Override

          • public long getTableVolume(URI uri) throws IOException {
          • return 0;
            + public long getTableVolume(URI uri) throws UnsupportedException {
            + throw new UnsupportedException();
              • End diff –

          Maybe NotImplementedException?

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38723199 — Diff: tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseTablespace.java — @@ -93,21 +94,13 @@ public void storageInit() throws IOException { hbaseConf.set(HConstants.ZOOKEEPER_CLIENT_PORT, splits[1]); } @Override public void setConfig(String name, String value) { - } - - @Override - public void setConfigs(Map<String, String> configs) { - } - public Configuration getHbaseConf() { return hbaseConf; } @Override public long getTableVolume(URI uri) throws IOException { return 0; + public long getTableVolume(URI uri) throws UnsupportedException { + throw new UnsupportedException(); End diff – Maybe NotImplementedException?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38723049

          — Diff: tajo-storage/tajo-storage-common/src/main/resources/storage-default.json —
          @@ -12,9 +12,13 @@
          "handler": "org.apache.tajo.storage.FileTablespace",
          "default-format": "text"
          },

          • "hbase": {
            + "hbase:zk": { "handler": "org.apache.tajo.storage.hbase.HBaseTablespace", "default-format": "hbase" + }

            ,
            + "jdbc:postgresql": {
            + "handler": "org.apache.tajo.storage.pgsql.PgSQLTablespace",
            + "default-format": "rowstore"

              • End diff –

          ```rowstore``` looks not intuitive for me. How about jdbc-store?

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38723049 — Diff: tajo-storage/tajo-storage-common/src/main/resources/storage-default.json — @@ -12,9 +12,13 @@ "handler": "org.apache.tajo.storage.FileTablespace", "default-format": "text" }, "hbase": { + "hbase:zk": { "handler": "org.apache.tajo.storage.hbase.HBaseTablespace", "default-format": "hbase" + } , + "jdbc:postgresql": { + "handler": "org.apache.tajo.storage.pgsql.PgSQLTablespace", + "default-format": "rowstore" End diff – ```rowstore``` looks not intuitive for me. How about jdbc-store?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/719#issuecomment-137649680

          rebased and reflected the comments.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/719#issuecomment-137649680 rebased and reflected the comments.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38721641

          — Diff: tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java —
          @@ -31,9 +31,9 @@

          String getDatabaseName();

          • Collection<String> getCatalogs();
            + Collection<String> getSchemas();
          • Collection<String> getTables(@Nullable String catalog);
            + Collection<String> getTables(@Nullable String schemaPattern, @Nullable String tablePattern);
          • TableDesc getTableDescriptor(String catalogName, String tableName) throws UndefinedTablespaceException;
            + TableDesc getTableDescriptor(String schemaName, String tableName) throws UndefinedTablespaceException;
              • End diff –

          TableDesc was a abbr of TableDescriptor. For consistent, I'll change it to TableDesc.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38721641 — Diff: tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java — @@ -31,9 +31,9 @@ String getDatabaseName(); Collection<String> getCatalogs(); + Collection<String> getSchemas(); Collection<String> getTables(@Nullable String catalog); + Collection<String> getTables(@Nullable String schemaPattern, @Nullable String tablePattern); TableDesc getTableDescriptor(String catalogName, String tableName) throws UndefinedTablespaceException; + TableDesc getTableDescriptor(String schemaName, String tableName) throws UndefinedTablespaceException; End diff – TableDesc was a abbr of TableDescriptor. For consistent, I'll change it to TableDesc.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38721593

          — Diff: tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java —
          @@ -31,9 +31,9 @@

          String getDatabaseName();

          • Collection<String> getCatalogs();
            + Collection<String> getSchemas();
              • End diff –

          getSchemas() will return database schema names. I'll add documentation.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38721593 — Diff: tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java — @@ -31,9 +31,9 @@ String getDatabaseName(); Collection<String> getCatalogs(); + Collection<String> getSchemas(); End diff – getSchemas() will return database schema names. I'll add documentation.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38721568

          — Diff: tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java —
          @@ -173,6 +179,11 @@ public void setFilter(EvalNode filter) {
          }

          @Override
          + public void setLimit(long num) {
          + throw new TajoRuntimeException(new UnsupportedException());
          — End diff –

          The runtime exception should be removed.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38721568 — Diff: tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java — @@ -173,6 +179,11 @@ public void setFilter(EvalNode filter) { } @Override + public void setLimit(long num) { + throw new TajoRuntimeException(new UnsupportedException()); — End diff – The runtime exception should be removed.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38721543

          — Diff: tajo-jdbc/pom.xml —
          @@ -120,7 +120,7 @@
          <dependency>
          <groupId>org.apache.hadoop</groupId>
          <artifactId>hadoop-common</artifactId>

          • <scope>test</scope>
            + <scope>provided</scope>
              • End diff –

          Fixed.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38721543 — Diff: tajo-jdbc/pom.xml — @@ -120,7 +120,7 @@ <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <scope>test</scope> + <scope>provided</scope> End diff – Fixed.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38721509

          — Diff: tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst —
          @@ -0,0 +1,29 @@
          +*************************************
          +PostgreSQL Storage Handler
          +*************************************
          +
          +Overview
          +========
          +
          +PostgreSQL storage handler available by default in Tajo. It enables users' queries to access database objects in PostgreSQL. Tables in PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are available in Tajo via this storage handles. Its main advantages is to allow federated query processing among tables in stored HDFS and PostgreSQL.
          +
          +Configuration
          +=============
          +
          +PostgreSQL storage handler is a builtin storage handler. So, you can eaisly register PostgreSQL databases to a Tajo cluster if you just add the following line to ``conf/storage-site.json`` file. If you want to know more information about ``storage-site.json``, please refer to :doc:`/table_management/tablespaces`.
          +
          +.. code-block:: json
          +
          + {
          + "spaces": {
          + "pgsql_db1": {
          + "uri": "jdbc:postgresql://hostname:port/db1"
          +
          + "configs":

          { + "mapped_database": "tajo_db1" + }

          + }
          + }
          + }
          +
          +Its configuration change add the tablespace named ``pgsql_db1`` mapped to the database ``db1`` in PostgreSQL. ``uri`` should be just a JDBC connection url. ``mapped_database`` is a database name shown in Tajo for the database ``db1`` in PostgreSQL. The tablespace will be available after you restart a tajo cluster.
          — End diff –

          I'd like to show as follows:

          • table space name: pgsql_db1
          • postgresql db: db1
          • tajo db: tajo_db1
          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38721509 — Diff: tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst — @@ -0,0 +1,29 @@ +************************************* +PostgreSQL Storage Handler +************************************* + +Overview +======== + +PostgreSQL storage handler available by default in Tajo. It enables users' queries to access database objects in PostgreSQL. Tables in PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are available in Tajo via this storage handles. Its main advantages is to allow federated query processing among tables in stored HDFS and PostgreSQL. + +Configuration +============= + +PostgreSQL storage handler is a builtin storage handler. So, you can eaisly register PostgreSQL databases to a Tajo cluster if you just add the following line to ``conf/storage-site.json`` file. If you want to know more information about ``storage-site.json``, please refer to :doc:`/table_management/tablespaces`. + +.. code-block:: json + + { + "spaces": { + "pgsql_db1": { + "uri": "jdbc:postgresql://hostname:port/db1" + + "configs": { + "mapped_database": "tajo_db1" + } + } + } + } + +Its configuration change add the tablespace named ``pgsql_db1`` mapped to the database ``db1`` in PostgreSQL. ``uri`` should be just a JDBC connection url. ``mapped_database`` is a database name shown in Tajo for the database ``db1`` in PostgreSQL. The tablespace will be available after you restart a tajo cluster. — End diff – I'd like to show as follows: table space name: pgsql_db1 postgresql db: db1 tajo db: tajo_db1
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38721459

          — Diff: tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst —
          @@ -0,0 +1,29 @@
          +*************************************
          +PostgreSQL Storage Handler
          +*************************************
          +
          +Overview
          +========
          +
          +PostgreSQL storage handler available by default in Tajo. It enables users' queries to access database objects in PostgreSQL. Tables in PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are available in Tajo via this storage handles. Its main advantages is to allow federated query processing among tables in stored HDFS and PostgreSQL.
          — End diff –

          Thanks for the comments.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38721459 — Diff: tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst — @@ -0,0 +1,29 @@ +************************************* +PostgreSQL Storage Handler +************************************* + +Overview +======== + +PostgreSQL storage handler available by default in Tajo. It enables users' queries to access database objects in PostgreSQL. Tables in PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are available in Tajo via this storage handles. Its main advantages is to allow federated query processing among tables in stored HDFS and PostgreSQL. — End diff – Thanks for the comments.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38721444

          — Diff: tajo-docs/src/main/sphinx/storage_plugins/overview.rst —
          @@ -1,5 +1,5 @@
          *************************************
          -Storage Plugin
          +Storage Plugin Overview
          — End diff –

          I agreed. I will improve the documentation.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38721444 — Diff: tajo-docs/src/main/sphinx/storage_plugins/overview.rst — @@ -1,5 +1,5 @@ ************************************* -Storage Plugin +Storage Plugin Overview — End diff – I agreed. I will improve the documentation.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38721432

          — Diff: tajo-core/src/main/java/org/apache/tajo/master/exec/QueryExecutor.java —
          @@ -260,6 +260,7 @@ public void execSimpleQuery(QueryContext queryContext, Session session, String q
          if (plan.getRootBlock().hasNode(NodeType.LIMIT)) {
          LimitNode limitNode = plan.getRootBlock().getNode(NodeType.LIMIT);
          maxRow = (int) limitNode.getFetchFirstNum();
          + scanNode.setLimit(maxRow);
          — End diff –

          The main purpose of setLimit is a hint described in the comment. It does not guarantee the number of retrieved rows. So, we must use both.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38721432 — Diff: tajo-core/src/main/java/org/apache/tajo/master/exec/QueryExecutor.java — @@ -260,6 +260,7 @@ public void execSimpleQuery(QueryContext queryContext, Session session, String q if (plan.getRootBlock().hasNode(NodeType.LIMIT)) { LimitNode limitNode = plan.getRootBlock().getNode(NodeType.LIMIT); maxRow = (int) limitNode.getFetchFirstNum(); + scanNode.setLimit(maxRow); — End diff – The main purpose of setLimit is a hint described in the comment. It does not guarantee the number of retrieved rows. So, we must use both.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38721393

          — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/LinkedMetadataManager.java —
          @@ -217,7 +205,7 @@ public void dropTable(String databaseName, String schemaName, String tableName)
          */
          public boolean existsTable(String dbName, String schemaName, String tableName) {
          if (providerMap.containsKey(dbName)) {

          • return providerMap.get(dbName).getTables(schemaName).contains(tableName);
            + return providerMap.get(dbName).getTables(schemaName, tableName).contains(tableName);
              • End diff –

          getTables() takes a pattern, and it results in a collection of table names.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38721393 — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/LinkedMetadataManager.java — @@ -217,7 +205,7 @@ public void dropTable(String databaseName, String schemaName, String tableName) */ public boolean existsTable(String dbName, String schemaName, String tableName) { if (providerMap.containsKey(dbName)) { return providerMap.get(dbName).getTables(schemaName).contains(tableName); + return providerMap.get(dbName).getTables(schemaName, tableName).contains(tableName); End diff – getTables() takes a pattern, and it results in a collection of table names.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38719467

          — Diff: tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Tablespace.java —
          @@ -149,7 +150,7 @@ public URI getRootUri() {

          • It is called by a Repartitioner for range shuffling when the SortRangeType of SortNode is USING_STORAGE_MANAGER.
          • In general Repartitioner determines the partition range using previous output statistics data.
          • In the special cases, such as HBase Repartitioner uses the result of this method.
          • *
            + *ㅂ ㅂ
              • End diff –

          Please remove this line.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38719467 — Diff: tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Tablespace.java — @@ -149,7 +150,7 @@ public URI getRootUri() { It is called by a Repartitioner for range shuffling when the SortRangeType of SortNode is USING_STORAGE_MANAGER. In general Repartitioner determines the partition range using previous output statistics data. In the special cases, such as HBase Repartitioner uses the result of this method. * + *ㅂ ㅂ End diff – Please remove this line.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38719289

          — Diff: tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java —
          @@ -173,6 +179,11 @@ public void setFilter(EvalNode filter) {
          }

          @Override
          + public void setLimit(long num) {
          + throw new TajoRuntimeException(new UnsupportedException());
          — End diff –

          Would you share some backgrounds why ```MergeScanner``` cannot support this feature?

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38719289 — Diff: tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java — @@ -173,6 +179,11 @@ public void setFilter(EvalNode filter) { } @Override + public void setLimit(long num) { + throw new TajoRuntimeException(new UnsupportedException()); — End diff – Would you share some backgrounds why ```MergeScanner``` cannot support this feature?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38719104

          — Diff: tajo-jdbc/pom.xml —
          @@ -120,7 +120,7 @@
          <dependency>
          <groupId>org.apache.hadoop</groupId>
          <artifactId>hadoop-common</artifactId>

          • <scope>test</scope>
            + <scope>provided</scope>
              • End diff –

          This change looks not necessary.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38719104 — Diff: tajo-jdbc/pom.xml — @@ -120,7 +120,7 @@ <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <scope>test</scope> + <scope>provided</scope> End diff – This change looks not necessary.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38718904

          — Diff: tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst —
          @@ -0,0 +1,29 @@
          +*************************************
          +PostgreSQL Storage Handler
          +*************************************
          +
          +Overview
          +========
          +
          +PostgreSQL storage handler available by default in Tajo. It enables users' queries to access database objects in PostgreSQL. Tables in PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are available in Tajo via this storage handles. Its main advantages is to allow federated query processing among tables in stored HDFS and PostgreSQL.
          +
          +Configuration
          +=============
          +
          +PostgreSQL storage handler is a builtin storage handler. So, you can eaisly register PostgreSQL databases to a Tajo cluster if you just add the following line to ``conf/storage-site.json`` file. If you want to know more information about ``storage-site.json``, please refer to :doc:`/table_management/tablespaces`.
          +
          +.. code-block:: json
          +
          + {
          + "spaces": {
          + "pgsql_db1": {
          + "uri": "jdbc:postgresql://hostname:port/db1"
          +
          + "configs":

          { + "mapped_database": "tajo_db1" + }

          + }
          + }
          + }
          +
          +Its configuration change add the tablespace named ``pgsql_db1`` mapped to the database ``db1`` in PostgreSQL. ``uri`` should be just a JDBC connection url. ``mapped_database`` is a database name shown in Tajo for the database ``db1`` in PostgreSQL. The tablespace will be available after you restart a tajo cluster.
          — End diff –

          Maybe ```tajo_db1``` instead of ```db1```.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38718904 — Diff: tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst — @@ -0,0 +1,29 @@ +************************************* +PostgreSQL Storage Handler +************************************* + +Overview +======== + +PostgreSQL storage handler available by default in Tajo. It enables users' queries to access database objects in PostgreSQL. Tables in PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are available in Tajo via this storage handles. Its main advantages is to allow federated query processing among tables in stored HDFS and PostgreSQL. + +Configuration +============= + +PostgreSQL storage handler is a builtin storage handler. So, you can eaisly register PostgreSQL databases to a Tajo cluster if you just add the following line to ``conf/storage-site.json`` file. If you want to know more information about ``storage-site.json``, please refer to :doc:`/table_management/tablespaces`. + +.. code-block:: json + + { + "spaces": { + "pgsql_db1": { + "uri": "jdbc:postgresql://hostname:port/db1" + + "configs": { + "mapped_database": "tajo_db1" + } + } + } + } + +Its configuration change add the tablespace named ``pgsql_db1`` mapped to the database ``db1`` in PostgreSQL. ``uri`` should be just a JDBC connection url. ``mapped_database`` is a database name shown in Tajo for the database ``db1`` in PostgreSQL. The tablespace will be available after you restart a tajo cluster. — End diff – Maybe ```tajo_db1``` instead of ```db1```.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38718867

          — Diff: tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst —
          @@ -0,0 +1,29 @@
          +*************************************
          +PostgreSQL Storage Handler
          +*************************************
          +
          +Overview
          +========
          +
          +PostgreSQL storage handler available by default in Tajo. It enables users' queries to access database objects in PostgreSQL. Tables in PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are available in Tajo via this storage handles. Its main advantages is to allow federated query processing among tables in stored HDFS and PostgreSQL.
          +
          +Configuration
          +=============
          +
          +PostgreSQL storage handler is a builtin storage handler. So, you can eaisly register PostgreSQL databases to a Tajo cluster if you just add the following line to ``conf/storage-site.json`` file. If you want to know more information about ``storage-site.json``, please refer to :doc:`/table_management/tablespaces`.
          +
          +.. code-block:: json
          +
          + {
          + "spaces": {
          + "pgsql_db1": {
          + "uri": "jdbc:postgresql://hostname:port/db1"
          +
          + "configs": {
          + "mapped_database": "tajo_db1"
          — End diff –

          Sorry, it way my misunderstanding.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38718867 — Diff: tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst — @@ -0,0 +1,29 @@ +************************************* +PostgreSQL Storage Handler +************************************* + +Overview +======== + +PostgreSQL storage handler available by default in Tajo. It enables users' queries to access database objects in PostgreSQL. Tables in PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are available in Tajo via this storage handles. Its main advantages is to allow federated query processing among tables in stored HDFS and PostgreSQL. + +Configuration +============= + +PostgreSQL storage handler is a builtin storage handler. So, you can eaisly register PostgreSQL databases to a Tajo cluster if you just add the following line to ``conf/storage-site.json`` file. If you want to know more information about ``storage-site.json``, please refer to :doc:`/table_management/tablespaces`. + +.. code-block:: json + + { + "spaces": { + "pgsql_db1": { + "uri": "jdbc:postgresql://hostname:port/db1" + + "configs": { + "mapped_database": "tajo_db1" — End diff – Sorry, it way my misunderstanding.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38718857

          — Diff: tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst —
          @@ -0,0 +1,29 @@
          +*************************************
          +PostgreSQL Storage Handler
          +*************************************
          +
          +Overview
          +========
          +
          +PostgreSQL storage handler available by default in Tajo. It enables users' queries to access database objects in PostgreSQL. Tables in PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are available in Tajo via this storage handles. Its main advantages is to allow federated query processing among tables in stored HDFS and PostgreSQL.
          +
          +Configuration
          +=============
          +
          +PostgreSQL storage handler is a builtin storage handler. So, you can eaisly register PostgreSQL databases to a Tajo cluster if you just add the following line to ``conf/storage-site.json`` file. If you want to know more information about ``storage-site.json``, please refer to :doc:`/table_management/tablespaces`.
          +
          +.. code-block:: json
          +
          + {
          + "spaces": {
          + "pgsql_db1": {
          + "uri": "jdbc:postgresql://hostname:port/db1"
          +
          + "configs": {
          + "mapped_database": "tajo_db1"
          — End diff –

          Maybe ```pgsql_db1``` instead of ```tajo_db1```.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38718857 — Diff: tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst — @@ -0,0 +1,29 @@ +************************************* +PostgreSQL Storage Handler +************************************* + +Overview +======== + +PostgreSQL storage handler available by default in Tajo. It enables users' queries to access database objects in PostgreSQL. Tables in PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are available in Tajo via this storage handles. Its main advantages is to allow federated query processing among tables in stored HDFS and PostgreSQL. + +Configuration +============= + +PostgreSQL storage handler is a builtin storage handler. So, you can eaisly register PostgreSQL databases to a Tajo cluster if you just add the following line to ``conf/storage-site.json`` file. If you want to know more information about ``storage-site.json``, please refer to :doc:`/table_management/tablespaces`. + +.. code-block:: json + + { + "spaces": { + "pgsql_db1": { + "uri": "jdbc:postgresql://hostname:port/db1" + + "configs": { + "mapped_database": "tajo_db1" — End diff – Maybe ```pgsql_db1``` instead of ```tajo_db1```.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38718817

          — Diff: tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst —
          @@ -0,0 +1,29 @@
          +*************************************
          +PostgreSQL Storage Handler
          +*************************************
          +
          +Overview
          +========
          +
          +PostgreSQL storage handler available by default in Tajo. It enables users' queries to access database objects in PostgreSQL. Tables in PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are available in Tajo via this storage handles. Its main advantages is to allow federated query processing among tables in stored HDFS and PostgreSQL.
          — End diff –

          ```is``` is missing in ```PostgreSQL storage handler available by default in Tajo.```. Also, ```handles``` should be fixed as ```handlers```.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38718817 — Diff: tajo-docs/src/main/sphinx/storage_plugins/postgresql.rst — @@ -0,0 +1,29 @@ +************************************* +PostgreSQL Storage Handler +************************************* + +Overview +======== + +PostgreSQL storage handler available by default in Tajo. It enables users' queries to access database objects in PostgreSQL. Tables in PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are available in Tajo via this storage handles. Its main advantages is to allow federated query processing among tables in stored HDFS and PostgreSQL. — End diff – ```is``` is missing in ```PostgreSQL storage handler available by default in Tajo.```. Also, ```handles``` should be fixed as ```handlers```.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38718765

          — Diff: tajo-docs/src/main/sphinx/storage_plugins/overview.rst —
          @@ -1,5 +1,5 @@
          *************************************
          -Storage Plugin
          +Storage Plugin Overview
          — End diff –

          You've added a handler for PostgreSQL. Please update the expression ```we have a plan to support RDBMS storages like Oracle, MySQL, PostgreSQL.``` in this document.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38718765 — Diff: tajo-docs/src/main/sphinx/storage_plugins/overview.rst — @@ -1,5 +1,5 @@ ************************************* -Storage Plugin +Storage Plugin Overview — End diff – You've added a handler for PostgreSQL. Please update the expression ```we have a plan to support RDBMS storages like Oracle, MySQL, PostgreSQL.``` in this document.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38718391

          — Diff: tajo-core/src/main/java/org/apache/tajo/master/exec/QueryExecutor.java —
          @@ -260,6 +260,7 @@ public void execSimpleQuery(QueryContext queryContext, Session session, String q
          if (plan.getRootBlock().hasNode(NodeType.LIMIT)) {
          LimitNode limitNode = plan.getRootBlock().getNode(NodeType.LIMIT);
          maxRow = (int) limitNode.getFetchFirstNum();
          + scanNode.setLimit(maxRow);
          — End diff –

          It seems that this change makes NonForwardQueryResultFileScanner.maxRow unnecessary.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38718391 — Diff: tajo-core/src/main/java/org/apache/tajo/master/exec/QueryExecutor.java — @@ -260,6 +260,7 @@ public void execSimpleQuery(QueryContext queryContext, Session session, String q if (plan.getRootBlock().hasNode(NodeType.LIMIT)) { LimitNode limitNode = plan.getRootBlock().getNode(NodeType.LIMIT); maxRow = (int) limitNode.getFetchFirstNum(); + scanNode.setLimit(maxRow); — End diff – It seems that this change makes NonForwardQueryResultFileScanner.maxRow unnecessary.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38717883

          — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/LinkedMetadataManager.java —
          @@ -217,7 +205,7 @@ public void dropTable(String databaseName, String schemaName, String tableName)
          */
          public boolean existsTable(String dbName, String schemaName, String tableName) {
          if (providerMap.containsKey(dbName)) {

          • return providerMap.get(dbName).getTables(schemaName).contains(tableName);
            + return providerMap.get(dbName).getTables(schemaName, tableName).contains(tableName);
              • End diff –

          ```contains()``` looks redundant.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38717883 — Diff: tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/LinkedMetadataManager.java — @@ -217,7 +205,7 @@ public void dropTable(String databaseName, String schemaName, String tableName) */ public boolean existsTable(String dbName, String schemaName, String tableName) { if (providerMap.containsKey(dbName)) { return providerMap.get(dbName).getTables(schemaName).contains(tableName); + return providerMap.get(dbName).getTables(schemaName, tableName).contains(tableName); End diff – ```contains()``` looks redundant.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38717641

          — Diff: tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java —
          @@ -31,9 +31,9 @@

          String getDatabaseName();

          • Collection<String> getCatalogs();
            + Collection<String> getSchemas();
              • End diff –

          Would you add some descriptions for this class?

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38717641 — Diff: tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java — @@ -31,9 +31,9 @@ String getDatabaseName(); Collection<String> getCatalogs(); + Collection<String> getSchemas(); End diff – Would you add some descriptions for this class?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/719#discussion_r38717625

          — Diff: tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java —
          @@ -31,9 +31,9 @@

          String getDatabaseName();

          • Collection<String> getCatalogs();
            + Collection<String> getSchemas();
          • Collection<String> getTables(@Nullable String catalog);
            + Collection<String> getTables(@Nullable String schemaPattern, @Nullable String tablePattern);
          • TableDesc getTableDescriptor(String catalogName, String tableName) throws UndefinedTablespaceException;
            + TableDesc getTableDescriptor(String schemaName, String tableName) throws UndefinedTablespaceException;
              • End diff –

          We have both ```TableDesc``` and ```TableDescriptor```. This naming convention is very weird, but I think we have to distinguish them now.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/719#discussion_r38717625 — Diff: tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/MetadataProvider.java — @@ -31,9 +31,9 @@ String getDatabaseName(); Collection<String> getCatalogs(); + Collection<String> getSchemas(); Collection<String> getTables(@Nullable String catalog); + Collection<String> getTables(@Nullable String schemaPattern, @Nullable String tablePattern); TableDesc getTableDescriptor(String catalogName, String tableName) throws UndefinedTablespaceException; + TableDesc getTableDescriptor(String schemaName, String tableName) throws UndefinedTablespaceException; End diff – We have both ```TableDesc``` and ```TableDescriptor```. This naming convention is very weird, but I think we have to distinguish them now.
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12753967/TAJO-1730_3.patch
          against master revision release-0.9.0-rc0-437-g22ab1cf.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 54 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to cause Findbugs (version 2.0.3) to fail.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-server tajo-client tajo-cluster-tests tajo-common tajo-core tajo-core-tests tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs tajo-storage/tajo-storage-jdbc tajo-storage/tajo-storage-pgsql:
          org.apache.tajo.storage.pgsql.TestPgSQLEndPointTests
          org.apache.tajo.storage.pgsql.TestPgSQLMetadataProvider
          org.apache.tajo.storage.pgsql.TestPgSQLSimpleQueryTests
          org.apache.tajo.storage.pgsql.TestPgSQLQueryTests
          org.apache.tajo.storage.pgsql.TestPgSQLJdbcTableSpace

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/845//testReport/
          Findbugs results: https://builds.apache.org/job/PreCommit-TAJO-Build/845//findbugsResult
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/845//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12753967/TAJO-1730_3.patch against master revision release-0.9.0-rc0-437-g22ab1cf. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 54 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to cause Findbugs (version 2.0.3) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-server tajo-client tajo-cluster-tests tajo-common tajo-core tajo-core-tests tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs tajo-storage/tajo-storage-jdbc tajo-storage/tajo-storage-pgsql: org.apache.tajo.storage.pgsql.TestPgSQLEndPointTests org.apache.tajo.storage.pgsql.TestPgSQLMetadataProvider org.apache.tajo.storage.pgsql.TestPgSQLSimpleQueryTests org.apache.tajo.storage.pgsql.TestPgSQLQueryTests org.apache.tajo.storage.pgsql.TestPgSQLJdbcTableSpace Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/845//testReport/ Findbugs results: https://builds.apache.org/job/PreCommit-TAJO-Build/845//findbugsResult Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/845//console This message is automatically generated.
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12753884/TAJO-1730.patch
          against master revision release-0.9.0-rc0-437-g22ab1cf.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 54 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to cause Findbugs (version 2.0.3) to fail.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-server tajo-client tajo-cluster-tests tajo-common tajo-core tajo-core-tests tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs tajo-storage/tajo-storage-jdbc tajo-storage/tajo-storage-pgsql:
          org.apache.tajo.storage.pgsql.TestPgSQLMetadataProvider
          org.apache.tajo.storage.pgsql.TestPgSQLSimpleQueryTests
          org.apache.tajo.storage.pgsql.TestPgSQLEndPointTests
          org.apache.tajo.storage.pgsql.TestPgSQLJdbcTableSpace
          org.apache.tajo.storage.pgsql.TestPgSQLQueryTests

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/841//testReport/
          Findbugs results: https://builds.apache.org/job/PreCommit-TAJO-Build/841//findbugsResult
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/841//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12753884/TAJO-1730.patch against master revision release-0.9.0-rc0-437-g22ab1cf. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 54 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to cause Findbugs (version 2.0.3) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-server tajo-client tajo-cluster-tests tajo-common tajo-core tajo-core-tests tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs tajo-storage/tajo-storage-jdbc tajo-storage/tajo-storage-pgsql: org.apache.tajo.storage.pgsql.TestPgSQLMetadataProvider org.apache.tajo.storage.pgsql.TestPgSQLSimpleQueryTests org.apache.tajo.storage.pgsql.TestPgSQLEndPointTests org.apache.tajo.storage.pgsql.TestPgSQLJdbcTableSpace org.apache.tajo.storage.pgsql.TestPgSQLQueryTests Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/841//testReport/ Findbugs results: https://builds.apache.org/job/PreCommit-TAJO-Build/841//findbugsResult Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/841//console This message is automatically generated.
          Hide
          githubbot ASF GitHub Bot added a comment -
          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/719#issuecomment-137048437 Here is a documentation. http://people.apache.org/~hyunsik/TAJO-1730/storage_plugins.html
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/719#issuecomment-137034402

          I'm going to add documentation soon.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/719#issuecomment-137034402 I'm going to add documentation soon.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user hyunsik opened a pull request:

          https://github.com/apache/tajo/pull/719

          TAJO-1730: JDBC Tablespace support.

          This patch includes jdbc table storage support and PostgreSQL database tablespace.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/hyunsik/tajo TAJO-1730

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/tajo/pull/719.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #719


          commit 45479947332fef46d34a023751b8656a24495539
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2015-07-31T00:01:45Z

          initial work.

          commit 5ca1a4abcc85b0a5742ff98c282b62e5d39f3436
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2015-08-09T06:31:29Z

          Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1730

          commit 920e8a42689e3f38d1ccac7eff3b8a60dd0a8a46
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2015-08-09T14:36:24Z

          Implement the base jdbc tablespace and its unit test base.

          commit 9b26ccc0478162ef2c723bab299296d5c6ce618c
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2015-08-10T22:17:28Z

          Add base implementation for mysql and pgsql storages.

          commit 39e7bd8e3f98565eae47eb8eae10888d655c11b9
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2015-08-13T14:16:06Z

          Implement the base.

          commit fe0b8a769a80322538517c3a5385d1bca042bd60
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2015-08-13T14:16:42Z

          Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1730


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user hyunsik opened a pull request: https://github.com/apache/tajo/pull/719 TAJO-1730 : JDBC Tablespace support. This patch includes jdbc table storage support and PostgreSQL database tablespace. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hyunsik/tajo TAJO-1730 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/719.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #719 commit 45479947332fef46d34a023751b8656a24495539 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-07-31T00:01:45Z initial work. commit 5ca1a4abcc85b0a5742ff98c282b62e5d39f3436 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-08-09T06:31:29Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1730 commit 920e8a42689e3f38d1ccac7eff3b8a60dd0a8a46 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-08-09T14:36:24Z Implement the base jdbc tablespace and its unit test base. commit 9b26ccc0478162ef2c723bab299296d5c6ce618c Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-08-10T22:17:28Z Add base implementation for mysql and pgsql storages. commit 39e7bd8e3f98565eae47eb8eae10888d655c11b9 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-08-13T14:16:06Z Implement the base. commit fe0b8a769a80322538517c3a5385d1bca042bd60 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-08-13T14:16:42Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1730

            People

            • Assignee:
              hyunsik Hyunsik Choi
              Reporter:
              hyunsik Hyunsik Choi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development