Sqoop
  1. Sqoop
  2. SQOOP-682

Sqoop2: Use templating in job.etl classes

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 1.99.2
    • Component/s: None
    • Labels:
      None

      Description

      Connector is specifying configuration objects and therefore we're passing generic objects to callbacks. Would be great if callback objects (Extractor, Loader, ...) would use java template mechanism to specify which configuration objects they are expecting to avoid unnecessary object casting.

      1. bugSQOOP-682.patch
        18 kB
        Jarek Jarcec Cecho

        Issue Links

          Activity

          Hide
          Cheolsoo Park added a comment -

          +1. will commit it after running unit tests.

          Show
          Cheolsoo Park added a comment - +1. will commit it after running unit tests.
          Hide
          Cheolsoo Park added a comment -

          Committed to sqoop2. Thanks Jarcec!

          Show
          Cheolsoo Park added a comment - Committed to sqoop2. Thanks Jarcec!
          Hide
          Hudson added a comment -

          Integrated in Sqoop2-hadoop200 #91 (See https://builds.apache.org/job/Sqoop2-hadoop200/91/)
          SQOOP-682: Use templating in job.etl classes (Revision 53688b54b53ced0b7e56774969ad14f0e80f23d8)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=53688b54b53ced0b7e56774969ad14f0e80f23d8
          Files :

          • spi/src/main/java/org/apache/sqoop/job/etl/Destroyer.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportDestroyer.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java
          • spi/src/main/java/org/apache/sqoop/job/etl/Initializer.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportInitializer.java
          • spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportDestroyer.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportInitializer.java
          • spi/src/main/java/org/apache/sqoop/job/etl/Partitioner.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java
          • spi/src/main/java/org/apache/sqoop/job/etl/Loader.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java
          Show
          Hudson added a comment - Integrated in Sqoop2-hadoop200 #91 (See https://builds.apache.org/job/Sqoop2-hadoop200/91/ ) SQOOP-682 : Use templating in job.etl classes (Revision 53688b54b53ced0b7e56774969ad14f0e80f23d8) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=53688b54b53ced0b7e56774969ad14f0e80f23d8 Files : spi/src/main/java/org/apache/sqoop/job/etl/Destroyer.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportDestroyer.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java spi/src/main/java/org/apache/sqoop/job/etl/Initializer.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportInitializer.java spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportDestroyer.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportInitializer.java spi/src/main/java/org/apache/sqoop/job/etl/Partitioner.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java spi/src/main/java/org/apache/sqoop/job/etl/Loader.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop2-hadoop100 #91 (See https://builds.apache.org/job/Sqoop2-hadoop100/91/)
          SQOOP-682: Use templating in job.etl classes (Revision 53688b54b53ced0b7e56774969ad14f0e80f23d8)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=53688b54b53ced0b7e56774969ad14f0e80f23d8
          Files :

          • spi/src/main/java/org/apache/sqoop/job/etl/Destroyer.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportInitializer.java
          • spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java
          • spi/src/main/java/org/apache/sqoop/job/etl/Partitioner.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportDestroyer.java
          • spi/src/main/java/org/apache/sqoop/job/etl/Initializer.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportDestroyer.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportInitializer.java
          • spi/src/main/java/org/apache/sqoop/job/etl/Loader.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java
          Show
          Hudson added a comment - Integrated in Sqoop2-hadoop100 #91 (See https://builds.apache.org/job/Sqoop2-hadoop100/91/ ) SQOOP-682 : Use templating in job.etl classes (Revision 53688b54b53ced0b7e56774969ad14f0e80f23d8) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=53688b54b53ced0b7e56774969ad14f0e80f23d8 Files : spi/src/main/java/org/apache/sqoop/job/etl/Destroyer.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportInitializer.java spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java spi/src/main/java/org/apache/sqoop/job/etl/Partitioner.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportDestroyer.java spi/src/main/java/org/apache/sqoop/job/etl/Initializer.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportDestroyer.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportInitializer.java spi/src/main/java/org/apache/sqoop/job/etl/Loader.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java

            People

            • Assignee:
              Jarek Jarcec Cecho
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development