Sqoop
  1. Sqoop
  2. SQOOP-650

Add ability to name mapreduce job generated by Sqoop

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.2
    • Fix Version/s: 1.4.3
    • Component/s: None
    • Labels:
      None

      Description

      Names of generated mapreduce job is currently the same as jar name (for example table_name.jar). It would be nice if Sqoop would have command line parameter to override that so that user could modify that arbitrarily.

      Please note that this can be partially achieved even now, by passing Dmapred.job.name. However this approach do not work reliably when executing from Oozie or other tools that are directly executing hadoop jar.

      1. SQOOP-650.patch
        6 kB
        Jarek Jarcec Cecho
      2. bugSQOOP-650.patch
        6 kB
        Jarek Jarcec Cecho

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop23 #426 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/426/)
          SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3
          Files :

          • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
          • src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
          • src/java/org/apache/sqoop/SqoopOptions.java
          • src/java/org/apache/sqoop/tool/ExportTool.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #426 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/426/ ) SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3 Files : src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/mapreduce/ImportJobBase.java src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/tool/ExportTool.java src/java/org/apache/sqoop/tool/ImportTool.java src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop200 #279 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/279/)
          SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3
          Files :

          • src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
          • src/java/org/apache/sqoop/tool/ExportTool.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/java/org/apache/sqoop/SqoopOptions.java
          • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
          • src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #279 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/279/ ) SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3 Files : src/java/org/apache/sqoop/mapreduce/ImportJobBase.java src/java/org/apache/sqoop/tool/ExportTool.java src/java/org/apache/sqoop/tool/ImportTool.java src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop20 #273 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/273/)
          SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3
          Files :

          • src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
          • src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
          • src/java/org/apache/sqoop/SqoopOptions.java
          • src/java/org/apache/sqoop/tool/ExportTool.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #273 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/273/ ) SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3 Files : src/java/org/apache/sqoop/mapreduce/ImportJobBase.java src/java/org/apache/sqoop/mapreduce/ExportJobBase.java src/java/org/apache/sqoop/tool/ImportTool.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/tool/ExportTool.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop100 #267 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/267/)
          SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3
          Files :

          • src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/java/org/apache/sqoop/SqoopOptions.java
          • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
          • src/java/org/apache/sqoop/tool/ExportTool.java
          • src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #267 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/267/ ) SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3 Files : src/java/org/apache/sqoop/mapreduce/ExportJobBase.java src/java/org/apache/sqoop/tool/ImportTool.java src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/tool/ExportTool.java src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
          Hide
          Cheolsoo Park added a comment -

          Committed to trunk. Thanks Jarcec!

          Show
          Cheolsoo Park added a comment - Committed to trunk. Thanks Jarcec!
          Hide
          Jarek Jarcec Cecho added a comment -

          Pingning active committers - would it be possible to review my patch?

          Show
          Jarek Jarcec Cecho added a comment - Pingning active committers - would it be possible to review my patch?

            People

            • Assignee:
              Jarek Jarcec Cecho
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development