Sqoop
  1. Sqoop
  2. SQOOP-1309

Expand the Sqoop to support CUBRID database.

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.5
    • Fix Version/s: 1.4.6
    • Component/s: None
    • Labels:
      None

      Description

      We often need to get the data from the Apache Hadoop migrated to a relational database, or from
      relational database migration to Hadoop. Apache Sqoop provides convenient and efficient for us. But It could not support CUBRID, I want expand the Sqoop tools to support CUBRID for migration.

      Improvement:
      Through CUBRID JDBC to support sqoop-import and sqoop-export operation
      Support various database data types of CUBRID conversion
      Support to migrate data between HDFS/Hive/HBase and CUBRID.

      Appendix
      CUBRID: http://www.cubrid.org/

        Issue Links

          Activity

          Hide
          Jarek Jarcec Cecho added a comment -

          Good idea Chenqiang, that might be indeed helpful! Are you planning to work on this one?

          Show
          Jarek Jarcec Cecho added a comment - Good idea Chenqiang , that might be indeed helpful! Are you planning to work on this one?
          Hide
          Chenqiang added a comment -

          Thanks for your support, I am very happy to complete this function.

          Show
          Chenqiang added a comment - Thanks for your support, I am very happy to complete this function.
          Hide
          Jarek Jarcec Cecho added a comment -

          I'm assigning this ticket you you then Chenqiang.

          Show
          Jarek Jarcec Cecho added a comment - I'm assigning this ticket you you then Chenqiang .
          Hide
          Chenqiang added a comment -

          OK, I have seen that this ticket is assigned to me.

          Now, I want to know which branch I will begin from source.

          Current branch list:
          origin/SQOOP-1082
          origin/branch-1.4.0
          origin/branch-1.4.1
          origin/branch-1.4.2
          origin/branch-1.4.3
          origin/branch-1.4.4
          origin/branch-1.99.1
          origin/branch-1.99.2
          origin/branch-1.99.3
          origin/sqoop2
          origin/trunk

          Shall I change to the branch origin/branch-1.4.4? And I will add new feature on this branch and submit my new codes.

          Thanks

          Show
          Chenqiang added a comment - OK, I have seen that this ticket is assigned to me. Now, I want to know which branch I will begin from source. Current branch list: origin/ SQOOP-1082 origin/branch-1.4.0 origin/branch-1.4.1 origin/branch-1.4.2 origin/branch-1.4.3 origin/branch-1.4.4 origin/branch-1.99.1 origin/branch-1.99.2 origin/branch-1.99.3 origin/sqoop2 origin/trunk Shall I change to the branch origin/branch-1.4.4? And I will add new feature on this branch and submit my new codes. Thanks
          Hide
          Jarek Jarcec Cecho added a comment -

          The change should be definitely developed against trunk branch Chenqiang.

          Show
          Jarek Jarcec Cecho added a comment - The change should be definitely developed against trunk branch Chenqiang .
          Hide
          Chenqiang added a comment -

          OK, Thank you.
          I am going to develop this task now. When I prepare to submit the code, Maybe have some problems also.

          Show
          Chenqiang added a comment - OK, Thank you. I am going to develop this task now. When I prepare to submit the code, Maybe have some problems also.
          Hide
          Jarek Jarcec Cecho added a comment -

          Sure, let us know if you will have any difficulties Chenqiang!

          Show
          Jarek Jarcec Cecho added a comment - Sure, let us know if you will have any difficulties Chenqiang !
          Hide
          Chenqiang added a comment -

          Now, I want to commit the new code.
          Could you tell me what the submission process is?

          Show
          Chenqiang added a comment - Now, I want to commit the new code. Could you tell me what the submission process is?
          Hide
          Jarek Jarcec Cecho added a comment -

          Check out How to Contribute wiki page, Chenqiang. If you have patch ready, please upload it to Review board.

          Show
          Jarek Jarcec Cecho added a comment - Check out How to Contribute wiki page, Chenqiang . If you have patch ready, please upload it to Review board .
          Hide
          Chenqiang added a comment -

          SQOOP-1309 submit patch.

          Show
          Chenqiang added a comment - SQOOP-1309 submit patch.
          Hide
          Chenqiang added a comment - - edited

          I have submited the review patch on Review board.
          https://reviews.apache.org/r/21100/

          Thanks

          Show
          Chenqiang added a comment - - edited I have submited the review patch on Review board. https://reviews.apache.org/r/21100/ Thanks
          Hide
          Jarek Jarcec Cecho added a comment -

          I left couple comments on review board Chenqiang.

          Show
          Jarek Jarcec Cecho added a comment - I left couple comments on review board Chenqiang .
          Hide
          Jarek Jarcec Cecho added a comment -

          Canceling patch as we're waiting on incorporating feedback from the review.

          Show
          Jarek Jarcec Cecho added a comment - Canceling patch as we're waiting on incorporating feedback from the review.
          Hide
          Jarek Jarcec Cecho added a comment -

          Putting back to patch available state.

          Show
          Jarek Jarcec Cecho added a comment - Putting back to patch available state.
          Hide
          Chenqiang added a comment -

          Upload the patch.
          Thanks

          Show
          Chenqiang added a comment - Upload the patch. Thanks
          Hide
          ASF subversion and git services added a comment -

          Commit e0fc46e949d878cf52f93b0206facf2db73f4d5c in sqoop's branch refs/heads/trunk from Jarek Jarcec Cecho
          [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=e0fc46e ]

          SQOOP-1309: Expand the Sqoop to support CUBRID database.

          Show
          ASF subversion and git services added a comment - Commit e0fc46e949d878cf52f93b0206facf2db73f4d5c in sqoop's branch refs/heads/trunk from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=e0fc46e ] SQOOP-1309 : Expand the Sqoop to support CUBRID database.
          Hide
          Jarek Jarcec Cecho added a comment -

          Committed to trunk, thank you Chenqiang!

          Show
          Jarek Jarcec Cecho added a comment - Committed to trunk, thank you Chenqiang !
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #887 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/887/)
          SQOOP-1309: Expand the Sqoop to support CUBRID database. (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e0fc46e949d878cf52f93b0206facf2db73f4d5c)

          • COMPILING.txt
          • src/test/org/apache/sqoop/manager/cubrid/CubridAuthTest.java
          • src/test/com/cloudera/sqoop/manager/CubridManagerExportTest.java
          • src/java/org/apache/sqoop/manager/DefaultManagerFactory.java
          • src/test/org/apache/sqoop/manager/cubrid/CubridCompatTest.java
          • src/test/com/cloudera/sqoop/manager/CubridManagerImportTest.java
          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/java/org/apache/sqoop/mapreduce/cubrid/CubridUpsertOutputFormat.java
          • src/java/org/apache/sqoop/manager/CubridManager.java
          • src/test/org/apache/sqoop/manager/cubrid/CubridTestUtils.java
          • src/docs/user/compatibility.txt
          • build.xml
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #887 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/887/ ) SQOOP-1309 : Expand the Sqoop to support CUBRID database. (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e0fc46e949d878cf52f93b0206facf2db73f4d5c ) COMPILING.txt src/test/org/apache/sqoop/manager/cubrid/CubridAuthTest.java src/test/com/cloudera/sqoop/manager/CubridManagerExportTest.java src/java/org/apache/sqoop/manager/DefaultManagerFactory.java src/test/org/apache/sqoop/manager/cubrid/CubridCompatTest.java src/test/com/cloudera/sqoop/manager/CubridManagerImportTest.java src/test/com/cloudera/sqoop/ThirdPartyTests.java src/java/org/apache/sqoop/mapreduce/cubrid/CubridUpsertOutputFormat.java src/java/org/apache/sqoop/manager/CubridManager.java src/test/org/apache/sqoop/manager/cubrid/CubridTestUtils.java src/docs/user/compatibility.txt build.xml
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #923 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/923/)
          SQOOP-1309: Expand the Sqoop to support CUBRID database. (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e0fc46e949d878cf52f93b0206facf2db73f4d5c)

          • src/java/org/apache/sqoop/manager/CubridManager.java
          • src/test/org/apache/sqoop/manager/cubrid/CubridAuthTest.java
          • COMPILING.txt
          • src/test/com/cloudera/sqoop/manager/CubridManagerExportTest.java
          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/test/org/apache/sqoop/manager/cubrid/CubridTestUtils.java
          • src/java/org/apache/sqoop/manager/DefaultManagerFactory.java
          • src/docs/user/compatibility.txt
          • src/test/org/apache/sqoop/manager/cubrid/CubridCompatTest.java
          • src/test/com/cloudera/sqoop/manager/CubridManagerImportTest.java
          • src/java/org/apache/sqoop/mapreduce/cubrid/CubridUpsertOutputFormat.java
          • build.xml
          Show
          Hudson added a comment - FAILURE: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #923 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/923/ ) SQOOP-1309 : Expand the Sqoop to support CUBRID database. (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e0fc46e949d878cf52f93b0206facf2db73f4d5c ) src/java/org/apache/sqoop/manager/CubridManager.java src/test/org/apache/sqoop/manager/cubrid/CubridAuthTest.java COMPILING.txt src/test/com/cloudera/sqoop/manager/CubridManagerExportTest.java src/test/com/cloudera/sqoop/ThirdPartyTests.java src/test/org/apache/sqoop/manager/cubrid/CubridTestUtils.java src/java/org/apache/sqoop/manager/DefaultManagerFactory.java src/docs/user/compatibility.txt src/test/org/apache/sqoop/manager/cubrid/CubridCompatTest.java src/test/com/cloudera/sqoop/manager/CubridManagerImportTest.java src/java/org/apache/sqoop/mapreduce/cubrid/CubridUpsertOutputFormat.java build.xml
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #929 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/929/)
          SQOOP-1309: Expand the Sqoop to support CUBRID database. (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e0fc46e949d878cf52f93b0206facf2db73f4d5c)

          • src/test/org/apache/sqoop/manager/cubrid/CubridTestUtils.java
          • src/test/com/cloudera/sqoop/manager/CubridManagerExportTest.java
          • src/java/org/apache/sqoop/mapreduce/cubrid/CubridUpsertOutputFormat.java
          • src/java/org/apache/sqoop/manager/DefaultManagerFactory.java
          • src/docs/user/compatibility.txt
          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • build.xml
          • src/test/org/apache/sqoop/manager/cubrid/CubridAuthTest.java
          • src/java/org/apache/sqoop/manager/CubridManager.java
          • COMPILING.txt
          • src/test/com/cloudera/sqoop/manager/CubridManagerImportTest.java
          • src/test/org/apache/sqoop/manager/cubrid/CubridCompatTest.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #929 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/929/ ) SQOOP-1309 : Expand the Sqoop to support CUBRID database. (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e0fc46e949d878cf52f93b0206facf2db73f4d5c ) src/test/org/apache/sqoop/manager/cubrid/CubridTestUtils.java src/test/com/cloudera/sqoop/manager/CubridManagerExportTest.java src/java/org/apache/sqoop/mapreduce/cubrid/CubridUpsertOutputFormat.java src/java/org/apache/sqoop/manager/DefaultManagerFactory.java src/docs/user/compatibility.txt src/test/com/cloudera/sqoop/ThirdPartyTests.java build.xml src/test/org/apache/sqoop/manager/cubrid/CubridAuthTest.java src/java/org/apache/sqoop/manager/CubridManager.java COMPILING.txt src/test/com/cloudera/sqoop/manager/CubridManagerImportTest.java src/test/org/apache/sqoop/manager/cubrid/CubridCompatTest.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1126 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1126/)
          SQOOP-1309: Expand the Sqoop to support CUBRID database. (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e0fc46e949d878cf52f93b0206facf2db73f4d5c)

          • COMPILING.txt
          • src/test/com/cloudera/sqoop/ThirdPartyTests.java
          • src/docs/user/compatibility.txt
          • src/test/org/apache/sqoop/manager/cubrid/CubridCompatTest.java
          • src/test/com/cloudera/sqoop/manager/CubridManagerExportTest.java
          • build.xml
          • src/test/com/cloudera/sqoop/manager/CubridManagerImportTest.java
          • src/test/org/apache/sqoop/manager/cubrid/CubridTestUtils.java
          • src/test/org/apache/sqoop/manager/cubrid/CubridAuthTest.java
          • src/java/org/apache/sqoop/mapreduce/cubrid/CubridUpsertOutputFormat.java
          • src/java/org/apache/sqoop/manager/CubridManager.java
          • src/java/org/apache/sqoop/manager/DefaultManagerFactory.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1126 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1126/ ) SQOOP-1309 : Expand the Sqoop to support CUBRID database. (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e0fc46e949d878cf52f93b0206facf2db73f4d5c ) COMPILING.txt src/test/com/cloudera/sqoop/ThirdPartyTests.java src/docs/user/compatibility.txt src/test/org/apache/sqoop/manager/cubrid/CubridCompatTest.java src/test/com/cloudera/sqoop/manager/CubridManagerExportTest.java build.xml src/test/com/cloudera/sqoop/manager/CubridManagerImportTest.java src/test/org/apache/sqoop/manager/cubrid/CubridTestUtils.java src/test/org/apache/sqoop/manager/cubrid/CubridAuthTest.java src/java/org/apache/sqoop/mapreduce/cubrid/CubridUpsertOutputFormat.java src/java/org/apache/sqoop/manager/CubridManager.java src/java/org/apache/sqoop/manager/DefaultManagerFactory.java

            People

            • Assignee:
              Chenqiang
              Reporter:
              Chenqiang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 336h
                336h
                Remaining:
                Remaining Estimate - 336h
                336h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development