Sqoop
  1. Sqoop
  2. SQOOP-650

Add ability to name mapreduce job generated by Sqoop

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.2
    • Fix Version/s: 1.4.3
    • Component/s: None
    • Labels:
      None

      Description

      Names of generated mapreduce job is currently the same as jar name (for example table_name.jar). It would be nice if Sqoop would have command line parameter to override that so that user could modify that arbitrarily.

      Please note that this can be partially achieved even now, by passing Dmapred.job.name. However this approach do not work reliably when executing from Oozie or other tools that are directly executing hadoop jar.

      1. bugSQOOP-650.patch
        6 kB
        Jarek Jarcec Cecho
      2. SQOOP-650.patch
        6 kB
        Jarek Jarcec Cecho

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          7m 34s 1 Jarek Jarcec Cecho 23/Oct/12 02:40
          Patch Available Patch Available Resolved Resolved
          14d 21h 50m 1 Cheolsoo Park 07/Nov/12 00:30
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop23 #426 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/426/)
          SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3
          Files :

          • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
          • src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
          • src/java/org/apache/sqoop/SqoopOptions.java
          • src/java/org/apache/sqoop/tool/ExportTool.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #426 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/426/ ) SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3 Files : src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/mapreduce/ImportJobBase.java src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/tool/ExportTool.java src/java/org/apache/sqoop/tool/ImportTool.java src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop200 #279 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/279/)
          SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3
          Files :

          • src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
          • src/java/org/apache/sqoop/tool/ExportTool.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/java/org/apache/sqoop/SqoopOptions.java
          • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
          • src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #279 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/279/ ) SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3 Files : src/java/org/apache/sqoop/mapreduce/ImportJobBase.java src/java/org/apache/sqoop/tool/ExportTool.java src/java/org/apache/sqoop/tool/ImportTool.java src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop20 #273 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/273/)
          SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3
          Files :

          • src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
          • src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
          • src/java/org/apache/sqoop/SqoopOptions.java
          • src/java/org/apache/sqoop/tool/ExportTool.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #273 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/273/ ) SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3 Files : src/java/org/apache/sqoop/mapreduce/ImportJobBase.java src/java/org/apache/sqoop/mapreduce/ExportJobBase.java src/java/org/apache/sqoop/tool/ImportTool.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/tool/ExportTool.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop100 #267 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/267/)
          SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3
          Files :

          • src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/java/org/apache/sqoop/SqoopOptions.java
          • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
          • src/java/org/apache/sqoop/tool/ExportTool.java
          • src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #267 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/267/ ) SQOOP-650 Add ability to name mapreduce job generated by Sqoop (Revision ebeb93351335cc73d6d77ef1fd0d7316655defa3) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ebeb93351335cc73d6d77ef1fd0d7316655defa3 Files : src/java/org/apache/sqoop/mapreduce/ExportJobBase.java src/java/org/apache/sqoop/tool/ImportTool.java src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/tool/ExportTool.java src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
          Cheolsoo Park made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Cheolsoo Park added a comment -

          Committed to trunk. Thanks Jarcec!

          Show
          Cheolsoo Park added a comment - Committed to trunk. Thanks Jarcec!
          Jarek Jarcec Cecho made changes -
          Attachment bugSQOOP-650.patch [ 12552370 ]
          Hide
          Jarek Jarcec Cecho added a comment -

          Pingning active committers - would it be possible to review my patch?

          Show
          Jarek Jarcec Cecho added a comment - Pingning active committers - would it be possible to review my patch?
          Jarek Jarcec Cecho made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Jarek Jarcec Cecho made changes -
          Attachment SQOOP-650.patch [ 12550404 ]
          Jarek Jarcec Cecho made changes -
          Remote Link This issue links to "Review board (Web Link)" [ 11401 ]
          Jarek Jarcec Cecho made changes -
          Field Original Value New Value
          Description Names of generated mapreduce job is currently the same as jar name (for example table_name.jar). It would be nice if Sqoop would have command line parameter to override that so that user could modify that arbitrarily.

          Please note that thi-s can be partially achieved even now, by passing Dmapred.job.name. However this approach do not work reliably when executing from Oozie or other tools that are directly executing hadoop jar.
          Names of generated mapreduce job is currently the same as jar name (for example table_name.jar). It would be nice if Sqoop would have command line parameter to override that so that user could modify that arbitrarily.

          Please note that this can be partially achieved even now, by passing Dmapred.job.name. However this approach do not work reliably when executing from Oozie or other tools that are directly executing hadoop jar.
          Jarek Jarcec Cecho created issue -

            People

            • Assignee:
              Jarek Jarcec Cecho
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development