Sqoop
  1. Sqoop
  2. SQOOP-986

Sqoop2: Add detection of local mode to mapreduce submission engine

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.99.1
    • Fix Version/s: 1.99.2
    • Component/s: None
    • Labels:
      None

      Description

      During setting up integration tests with various databases, I've noticed that LocalJobRunner is not behaving correctly as normal mapreduce. For example JobClient API getJob seems to be returning always null which is breaking the integration tests.

      I would suggest to add detection of local mode and make the job submission synchronous when running in with LocalJobRunner.

      1. bugSQOOP-986.patch
        2 kB
        Jarek Jarcec Cecho

        Issue Links

          Activity

          Hide
          Kathleen Ting added a comment -
          Show
          Kathleen Ting added a comment - Thanks Jarcec for your contribution. The patch is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=75ae6e3ea0931f7f677d3e0bfd5109c8fe85c369
          Hide
          Hudson added a comment -

          Integrated in Sqoop2-hadoop200 #265 (See https://builds.apache.org/job/Sqoop2-hadoop200/265/)
          SQOOP-986: Sqoop2: Add detection of local mode to mapreduce submission engine (Revision 75ae6e3ea0931f7f677d3e0bfd5109c8fe85c369)

          Result = SUCCESS
          kathleen : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=75ae6e3ea0931f7f677d3e0bfd5109c8fe85c369
          Files :

          • submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java
          Show
          Hudson added a comment - Integrated in Sqoop2-hadoop200 #265 (See https://builds.apache.org/job/Sqoop2-hadoop200/265/ ) SQOOP-986 : Sqoop2: Add detection of local mode to mapreduce submission engine (Revision 75ae6e3ea0931f7f677d3e0bfd5109c8fe85c369) Result = SUCCESS kathleen : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=75ae6e3ea0931f7f677d3e0bfd5109c8fe85c369 Files : submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop2-hadoop100 #266 (See https://builds.apache.org/job/Sqoop2-hadoop100/266/)
          SQOOP-986: Sqoop2: Add detection of local mode to mapreduce submission engine (Revision 75ae6e3ea0931f7f677d3e0bfd5109c8fe85c369)

          Result = SUCCESS
          kathleen : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=75ae6e3ea0931f7f677d3e0bfd5109c8fe85c369
          Files :

          • submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java
          Show
          Hudson added a comment - Integrated in Sqoop2-hadoop100 #266 (See https://builds.apache.org/job/Sqoop2-hadoop100/266/ ) SQOOP-986 : Sqoop2: Add detection of local mode to mapreduce submission engine (Revision 75ae6e3ea0931f7f677d3e0bfd5109c8fe85c369) Result = SUCCESS kathleen : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=75ae6e3ea0931f7f677d3e0bfd5109c8fe85c369 Files : submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java

            People

            • Assignee:
              Jarek Jarcec Cecho
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development