Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-582

Create a template method for job submission in Export/Import JobBase

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.1-incubating
    • Fix Version/s: 1.4.3
    • Component/s: connectors
    • Labels:
      None

      Description

      Currently the job submission is synchronous and waits for job to complete. There are use cases when a connector needs to submit job and do other things while the job is in progress retaining the code to extract counters.

      1. SQOOP-582.patch
        2 kB
        Venkatesh Seetharam

        Activity

        Hide
        svenkat Venkatesh Seetharam added a comment -

        Attaching a patch to address the issue.

        Show
        svenkat Venkatesh Seetharam added a comment - Attaching a patch to address the issue.
        Hide
        abhijeet_gaikwad Abhijeet Gaikwad added a comment -

        I see redundant code. Can we move doSubmitJob() method to JobBase class. Also change JobBase.runJob() accordingly.
        Let me know.

        Thanks.

        Show
        abhijeet_gaikwad Abhijeet Gaikwad added a comment - I see redundant code. Can we move doSubmitJob() method to JobBase class. Also change JobBase.runJob() accordingly. Let me know. Thanks.
        Hide
        svenkat Venkatesh Seetharam added a comment -

        Well, you need to look at the implementations carefully to observe that it is necessary to duplicate the code. The runJob in itself is a template method that is overridden by Import and Export and the counter read by each of these are different.

        I could copy the doSubmitJob method to the base class but using it in runJob does not make sense IMO. Do you prefer the former?

        Show
        svenkat Venkatesh Seetharam added a comment - Well, you need to look at the implementations carefully to observe that it is necessary to duplicate the code. The runJob in itself is a template method that is overridden by Import and Export and the counter read by each of these are different. I could copy the doSubmitJob method to the base class but using it in runJob does not make sense IMO. Do you prefer the former?
        Hide
        jarcec Jarek Jarcec Cecho added a comment -

        Thanks Venkatesh for your comment,
        I believe that current patch make sense. It's pretty simple patch, so I'm bypassing review board and giving my +1 here.

        Jarcec

        Show
        jarcec Jarek Jarcec Cecho added a comment - Thanks Venkatesh for your comment, I believe that current patch make sense. It's pretty simple patch, so I'm bypassing review board and giving my +1 here. Jarcec
        Hide
        jarcec Jarek Jarcec Cecho added a comment -

        Committed revision 1375395.

        Thank you for your contribution Venkatesh!

        Jarcec

        Show
        jarcec Jarek Jarcec Cecho added a comment - Committed revision 1375395. Thank you for your contribution Venkatesh! Jarcec
        Hide
        hudson Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop20 #12 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/12/)
        SQOOP-582. Create a template method for job submission in Export/Import JobBase.

        (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375395)

        Result = SUCCESS
        jarcec :
        Files :

        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Show
        hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #12 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/12/ ) SQOOP-582 . Create a template method for job submission in Export/Import JobBase. (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375395) Result = SUCCESS jarcec : Files : /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Hide
        hudson Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop200 #3 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/3/)
        SQOOP-582. Create a template method for job submission in Export/Import JobBase.

        (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375395)

        Result = SUCCESS
        jarcec :
        Files :

        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Show
        hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #3 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/3/ ) SQOOP-582 . Create a template method for job submission in Export/Import JobBase. (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375395) Result = SUCCESS jarcec : Files : /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Hide
        hudson Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop23 #142 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/142/)
        SQOOP-582. Create a template method for job submission in Export/Import JobBase.

        (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375395)

        Result = SUCCESS
        jarcec :
        Files :

        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Show
        hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #142 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/142/ ) SQOOP-582 . Create a template method for job submission in Export/Import JobBase. (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375395) Result = SUCCESS jarcec : Files : /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Hide
        hudson Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop100 #12 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/12/)
        SQOOP-582. Create a template method for job submission in Export/Import JobBase.

        (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375395)

        Result = SUCCESS
        jarcec :
        Files :

        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Show
        hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #12 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/12/ ) SQOOP-582 . Create a template method for job submission in Export/Import JobBase. (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375395) Result = SUCCESS jarcec : Files : /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java

          People

          • Assignee:
            svenkat Venkatesh Seetharam
            Reporter:
            svenkat Venkatesh Seetharam
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development