Sqoop
  1. Sqoop
  2. SQOOP-601

Support custom schemas in PostgreSQL Connector

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.2
    • Fix Version/s: 1.4.3
    • Component/s: None
    • Labels:
      None

      Description

      We should add support for custom schemas in PostgreSQL Connector.

      1. SQOOP-601.patch
        48 kB
        Jarek Jarcec Cecho

        Issue Links

          Activity

          Hide
          Cheolsoo Park added a comment -

          Jarcec identified that the failures were unrelated with this commit.

          • hadoop20: hbase test timeout. for this, I am going to a new jira to increase timeout.
          • hadoop200: test environment issue.

          Thanks!

          Show
          Cheolsoo Park added a comment - Jarcec identified that the failures were unrelated with this commit. hadoop20: hbase test timeout. for this, I am going to a new jira to increase timeout. hadoop200: test environment issue. Thanks!
          Hide
          Cheolsoo Park added a comment -

          Looks lime my commit broke the build. I will look into it.

          Show
          Cheolsoo Park added a comment - Looks lime my commit broke the build. I will look into it.
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop20 #56 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/56/)
          SQOOP-601 Support custom schemas in PostgreSQL Connector (Revision f11c3091c2ef3e52717acbc40c88def238b1cc3a)

          Result = FAILURE
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f11c3091c2ef3e52717acbc40c88def238b1cc3a
          Files :

          • src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java
          • src/test/com/cloudera/sqoop/manager/PostgresqlExportTest.java
          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java
          • src/java/org/apache/sqoop/manager/PostgresqlManager.java
          • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java
          • src/test/com/cloudera/sqoop/manager/PostgresqlImportTest.java
          • src/java/org/apache/sqoop/mapreduce/JdbcUpsertExportJob.java
          • src/docs/user/connectors.txt
          • src/java/com/cloudera/sqoop/manager/PostgresqlManager.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/test/com/cloudera/sqoop/manager/PostgresqlTest.java
          • src/java/org/apache/sqoop/manager/PGBulkloadManager.java
          • src/java/org/apache/sqoop/manager/CatalogQueryManager.java
          • src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #56 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/56/ ) SQOOP-601 Support custom schemas in PostgreSQL Connector (Revision f11c3091c2ef3e52717acbc40c88def238b1cc3a) Result = FAILURE cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f11c3091c2ef3e52717acbc40c88def238b1cc3a Files : src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java src/test/com/cloudera/sqoop/manager/PostgresqlExportTest.java src/test/com/cloudera/sqoop/ThirdPartyTests.java src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java src/java/org/apache/sqoop/manager/PostgresqlManager.java src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java src/test/com/cloudera/sqoop/manager/PostgresqlImportTest.java src/java/org/apache/sqoop/mapreduce/JdbcUpsertExportJob.java src/docs/user/connectors.txt src/java/com/cloudera/sqoop/manager/PostgresqlManager.java src/java/org/apache/sqoop/manager/SqlManager.java src/test/com/cloudera/sqoop/manager/PostgresqlTest.java src/java/org/apache/sqoop/manager/PGBulkloadManager.java src/java/org/apache/sqoop/manager/CatalogQueryManager.java src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop23 #191 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/191/)
          SQOOP-601 Support custom schemas in PostgreSQL Connector (Revision f11c3091c2ef3e52717acbc40c88def238b1cc3a)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f11c3091c2ef3e52717acbc40c88def238b1cc3a
          Files :

          • src/test/com/cloudera/sqoop/manager/PostgresqlTest.java
          • src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java
          • src/test/com/cloudera/sqoop/manager/PostgresqlExportTest.java
          • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java
          • src/docs/user/connectors.txt
          • src/java/org/apache/sqoop/manager/PostgresqlManager.java
          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/java/org/apache/sqoop/manager/PGBulkloadManager.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java
          • src/java/org/apache/sqoop/mapreduce/JdbcUpsertExportJob.java
          • src/java/com/cloudera/sqoop/manager/PostgresqlManager.java
          • src/java/org/apache/sqoop/manager/CatalogQueryManager.java
          • src/test/com/cloudera/sqoop/manager/PostgresqlImportTest.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #191 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/191/ ) SQOOP-601 Support custom schemas in PostgreSQL Connector (Revision f11c3091c2ef3e52717acbc40c88def238b1cc3a) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f11c3091c2ef3e52717acbc40c88def238b1cc3a Files : src/test/com/cloudera/sqoop/manager/PostgresqlTest.java src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java src/test/com/cloudera/sqoop/manager/PostgresqlExportTest.java src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java src/docs/user/connectors.txt src/java/org/apache/sqoop/manager/PostgresqlManager.java src/test/com/cloudera/sqoop/ThirdPartyTests.java src/java/org/apache/sqoop/manager/PGBulkloadManager.java src/java/org/apache/sqoop/manager/SqlManager.java src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java src/java/org/apache/sqoop/mapreduce/JdbcUpsertExportJob.java src/java/com/cloudera/sqoop/manager/PostgresqlManager.java src/java/org/apache/sqoop/manager/CatalogQueryManager.java src/test/com/cloudera/sqoop/manager/PostgresqlImportTest.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop100 #62 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/62/)
          SQOOP-601 Support custom schemas in PostgreSQL Connector (Revision f11c3091c2ef3e52717acbc40c88def238b1cc3a)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f11c3091c2ef3e52717acbc40c88def238b1cc3a
          Files :

          • src/test/com/cloudera/sqoop/manager/PostgresqlImportTest.java
          • src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java
          • src/java/org/apache/sqoop/manager/PGBulkloadManager.java
          • src/docs/user/connectors.txt
          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/java/org/apache/sqoop/manager/CatalogQueryManager.java
          • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java
          • src/test/com/cloudera/sqoop/manager/PostgresqlTest.java
          • src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java
          • src/test/com/cloudera/sqoop/manager/PostgresqlExportTest.java
          • src/java/org/apache/sqoop/mapreduce/JdbcUpsertExportJob.java
          • src/java/org/apache/sqoop/manager/PostgresqlManager.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/java/com/cloudera/sqoop/manager/PostgresqlManager.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #62 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/62/ ) SQOOP-601 Support custom schemas in PostgreSQL Connector (Revision f11c3091c2ef3e52717acbc40c88def238b1cc3a) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f11c3091c2ef3e52717acbc40c88def238b1cc3a Files : src/test/com/cloudera/sqoop/manager/PostgresqlImportTest.java src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java src/java/org/apache/sqoop/manager/PGBulkloadManager.java src/docs/user/connectors.txt src/test/com/cloudera/sqoop/ThirdPartyTests.java src/java/org/apache/sqoop/manager/CatalogQueryManager.java src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java src/test/com/cloudera/sqoop/manager/PostgresqlTest.java src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java src/test/com/cloudera/sqoop/manager/PostgresqlExportTest.java src/java/org/apache/sqoop/mapreduce/JdbcUpsertExportJob.java src/java/org/apache/sqoop/manager/PostgresqlManager.java src/java/org/apache/sqoop/manager/SqlManager.java src/java/com/cloudera/sqoop/manager/PostgresqlManager.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop200 #41 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/41/)
          SQOOP-601 Support custom schemas in PostgreSQL Connector (Revision f11c3091c2ef3e52717acbc40c88def238b1cc3a)

          Result = FAILURE
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f11c3091c2ef3e52717acbc40c88def238b1cc3a
          Files :

          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java
          • src/test/com/cloudera/sqoop/manager/PostgresqlTest.java
          • src/java/org/apache/sqoop/manager/PGBulkloadManager.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/docs/user/connectors.txt
          • src/java/org/apache/sqoop/mapreduce/JdbcUpsertExportJob.java
          • src/java/com/cloudera/sqoop/manager/PostgresqlManager.java
          • src/test/com/cloudera/sqoop/manager/PostgresqlImportTest.java
          • src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java
          • src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java
          • src/test/com/cloudera/sqoop/manager/PostgresqlExportTest.java
          • src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java
          • src/java/org/apache/sqoop/manager/CatalogQueryManager.java
          • src/java/org/apache/sqoop/manager/PostgresqlManager.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #41 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/41/ ) SQOOP-601 Support custom schemas in PostgreSQL Connector (Revision f11c3091c2ef3e52717acbc40c88def238b1cc3a) Result = FAILURE cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f11c3091c2ef3e52717acbc40c88def238b1cc3a Files : src/test/com/cloudera/sqoop/ThirdPartyTests.java src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java src/test/com/cloudera/sqoop/manager/PostgresqlTest.java src/java/org/apache/sqoop/manager/PGBulkloadManager.java src/java/org/apache/sqoop/manager/SqlManager.java src/docs/user/connectors.txt src/java/org/apache/sqoop/mapreduce/JdbcUpsertExportJob.java src/java/com/cloudera/sqoop/manager/PostgresqlManager.java src/test/com/cloudera/sqoop/manager/PostgresqlImportTest.java src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java src/test/com/cloudera/sqoop/manager/PostgresqlExportTest.java src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java src/java/org/apache/sqoop/manager/CatalogQueryManager.java src/java/org/apache/sqoop/manager/PostgresqlManager.java
          Hide
          Cheolsoo Park added a comment -

          Committed to trunk. Thanks Jarcec!

          Show
          Cheolsoo Park added a comment - Committed to trunk. Thanks Jarcec!
          Hide
          Kathleen Ting added a comment -

          +1 on Jarcec's patch and +1 on having Cheolsoo commit.

          Show
          Kathleen Ting added a comment - +1 on Jarcec's patch and +1 on having Cheolsoo commit.
          Hide
          Jarek Jarcec Cecho added a comment -

          Thanks Cheolsoo for your review. I'm not in hurry for this, so I let this patch ready for you to commit so that you can verify your commit access.

          Jarcec

          Show
          Jarek Jarcec Cecho added a comment - Thanks Cheolsoo for your review. I'm not in hurry for this, so I let this patch ready for you to commit so that you can verify your commit access. Jarcec
          Hide
          Cheolsoo Park added a comment -

          +1

          @Jarcec
          All tests pass for me. I don't have commit permission yet, so you can go ahead commit if you want to.

          Show
          Cheolsoo Park added a comment - +1 @Jarcec All tests pass for me. I don't have commit permission yet, so you can go ahead commit if you want to.

            People

            • Assignee:
              Jarek Jarcec Cecho
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development