Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-580

Add Open-ended "teardown" routine that is called after the job execution for cleanup

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.1-incubating
    • Fix Version/s: 1.4.3
    • Component/s: connectors
    • Labels:
      None

      Description

      org.apache.sqoop.mapreduce.ImportJobBase.runImport -
      There is a jobSetup that is called after the job is configured but just before it is submitted to MapReduce. I'm adding a jobTeardown method in the finally block.

      I think the same applies to org.apache.sqoop.mapreduce.ExportJobBase.runExport. But the setup isnt called there.

      1. SQOOP-580.patch
        3 kB
        Venkatesh Seetharam

        Activity

        Hide
        svenkat Venkatesh Seetharam added a comment -

        Trivial addition of a method.

        Show
        svenkat Venkatesh Seetharam added a comment - Trivial addition of a method.
        Hide
        abhijeet_gaikwad Abhijeet Gaikwad added a comment -

        Can we move these methods up in JobBase Class. Let me know.

        Thanks.

        Show
        abhijeet_gaikwad Abhijeet Gaikwad added a comment - Can we move these methods up in JobBase Class. Let me know. Thanks.
        Hide
        jarcec Jarek Jarcec Cecho added a comment -

        Moving those methods to JobBase make sense to me as it seems as unnecessary code repetition.

        Jarcec

        Show
        jarcec Jarek Jarcec Cecho added a comment - Moving those methods to JobBase make sense to me as it seems as unnecessary code repetition. Jarcec
        Hide
        svenkat Venkatesh Seetharam added a comment -

        Moving those methods to JobBase make sense to me as it seems as unnecessary code repetition.

        I thought of this as well but if you look at the exceptions, they are different.
        ImportJobBase throws ImportException while ExportJobBase throws ExportException apart from IOException. I could very well wrap these into IOException but there is an implementation in org.apache.sqoop.mapreduce.HBaseImportJob that uses the specific ImportException and wanted to leave it in there.

        Makes sense?

        Any thoughts on eliminating repetition are welcome.

        Show
        svenkat Venkatesh Seetharam added a comment - Moving those methods to JobBase make sense to me as it seems as unnecessary code repetition. I thought of this as well but if you look at the exceptions, they are different. ImportJobBase throws ImportException while ExportJobBase throws ExportException apart from IOException. I could very well wrap these into IOException but there is an implementation in org.apache.sqoop.mapreduce.HBaseImportJob that uses the specific ImportException and wanted to leave it in there. Makes sense? Any thoughts on eliminating repetition are welcome.
        Hide
        jarcec Jarek Jarcec Cecho added a comment -

        I see, I've missed different exceptions. Let's keep it as it for now. It's pretty simple patch, so I'm skipping review board and giving my +1 here.

        Jarcec

        Show
        jarcec Jarek Jarcec Cecho added a comment - I see, I've missed different exceptions. Let's keep it as it for now. It's pretty simple patch, so I'm skipping review board and giving my +1 here. Jarcec
        Hide
        jarcec Jarek Jarcec Cecho added a comment -

        Committed revision 1375399.

        Thank you for your contribution Venkatesh!

        Jarcec

        Show
        jarcec Jarek Jarcec Cecho added a comment - Committed revision 1375399. Thank you for your contribution Venkatesh! Jarcec
        Hide
        hudson Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop20 #12 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/12/)
        SQOOP-580. Add Open-ended "teardown" routine that is called after the job execution for cleanup.

        (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375399)

        Result = SUCCESS
        jarcec :
        Files :

        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Show
        hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #12 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/12/ ) SQOOP-580 . Add Open-ended "teardown" routine that is called after the job execution for cleanup. (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375399) Result = SUCCESS jarcec : Files : /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Hide
        hudson Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop200 #3 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/3/)
        SQOOP-580. Add Open-ended "teardown" routine that is called after the job execution for cleanup.

        (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375399)

        Result = SUCCESS
        jarcec :
        Files :

        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Show
        hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #3 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/3/ ) SQOOP-580 . Add Open-ended "teardown" routine that is called after the job execution for cleanup. (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375399) Result = SUCCESS jarcec : Files : /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Hide
        hudson Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop23 #142 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/142/)
        SQOOP-580. Add Open-ended "teardown" routine that is called after the job execution for cleanup.

        (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375399)

        Result = SUCCESS
        jarcec :
        Files :

        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Show
        hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #142 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/142/ ) SQOOP-580 . Add Open-ended "teardown" routine that is called after the job execution for cleanup. (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375399) Result = SUCCESS jarcec : Files : /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Hide
        hudson Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop100 #12 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/12/)
        SQOOP-580. Add Open-ended "teardown" routine that is called after the job execution for cleanup.

        (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375399)

        Result = SUCCESS
        jarcec :
        Files :

        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
        • /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
        Show
        hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #12 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/12/ ) SQOOP-580 . Add Open-ended "teardown" routine that is called after the job execution for cleanup. (Seetharam Venkatesh via Jarek Jarcec Cecho) (Revision 1375399) Result = SUCCESS jarcec : Files : /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ExportJobBase.java /sqoop/trunk/src/java/org/apache/sqoop/mapreduce/ImportJobBase.java

          People

          • Assignee:
            svenkat Venkatesh Seetharam
            Reporter:
            svenkat Venkatesh Seetharam
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development