Sqoop
  1. Sqoop
  2. SQOOP-723

Add possibility to have synchronous submissions in Sqoop 2

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 1.99.2
    • Component/s: None
    • Labels:
      None

      Description

      It would be nice if Sqoop 2 would have ability to emulate synchronous submissions from client perspective.

        Issue Links

          Activity

          Hide
          Abraham Elmahrek added a comment - - edited

          Here are two solutions:

          1. Add option to submission start that tells the client to poll until finished. IE: submission start, followed by "synchronous" display of some kind.
          2. Add a new command 'submission submit' that starts a job and polls until finished.

          Also, it seems we should add another option to set how often sqoop should be polling when waiting for submission to finish.

          Show
          Abraham Elmahrek added a comment - - edited Here are two solutions: Add option to submission start that tells the client to poll until finished. IE: submission start, followed by "synchronous" display of some kind. Add a new command 'submission submit' that starts a job and polls until finished. Also, it seems we should add another option to set how often sqoop should be polling when waiting for submission to finish.
          Hide
          Jarek Jarcec Cecho added a comment -

          Hi Abraham Elmahrek,
          thank you very much for looking into this ticket. I personally like the first proposed solution, but without strong preference. It would be also great to do this functionality in SqoopClient API and base the shell implementation on it.

          Jarcec

          Show
          Jarek Jarcec Cecho added a comment - Hi Abraham Elmahrek , thank you very much for looking into this ticket. I personally like the first proposed solution, but without strong preference. It would be also great to do this functionality in SqoopClient API and base the shell implementation on it. Jarcec
          Hide
          Jarek Jarcec Cecho added a comment -

          Changing status to "patch available" so that it will show up in our review queue.

          Show
          Jarek Jarcec Cecho added a comment - Changing status to "patch available" so that it will show up in our review queue.
          Hide
          Jarek Jarcec Cecho added a comment -
          Show
          Jarek Jarcec Cecho added a comment - The patch is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=6ceb22497c0161880a37a7bee51f1de6093e6dcc Thank you Abe for your contribution! Jarcec
          Hide
          Hudson added a comment -

          Integrated in Sqoop2-hadoop100 #264 (See https://builds.apache.org/job/Sqoop2-hadoop100/264/)
          SQOOP-723: Add possibility to have synchronous submissions in Sqoop 2 (Revision 6ceb22497c0161880a37a7bee51f1de6093e6dcc)

          Result = FAILURE
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=6ceb22497c0161880a37a7bee51f1de6093e6dcc
          Files :

          • client/src/main/resources/client-resource.properties
          • client/src/main/java/org/apache/sqoop/client/core/Constants.java
          • client/src/main/java/org/apache/sqoop/client/shell/SubmissionStartFunction.java
          Show
          Hudson added a comment - Integrated in Sqoop2-hadoop100 #264 (See https://builds.apache.org/job/Sqoop2-hadoop100/264/ ) SQOOP-723 : Add possibility to have synchronous submissions in Sqoop 2 (Revision 6ceb22497c0161880a37a7bee51f1de6093e6dcc) Result = FAILURE jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=6ceb22497c0161880a37a7bee51f1de6093e6dcc Files : client/src/main/resources/client-resource.properties client/src/main/java/org/apache/sqoop/client/core/Constants.java client/src/main/java/org/apache/sqoop/client/shell/SubmissionStartFunction.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop2-hadoop200 #264 (See https://builds.apache.org/job/Sqoop2-hadoop200/264/)
          SQOOP-723: Add possibility to have synchronous submissions in Sqoop 2 (Revision 6ceb22497c0161880a37a7bee51f1de6093e6dcc)

          Result = SUCCESS
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=6ceb22497c0161880a37a7bee51f1de6093e6dcc
          Files :

          • client/src/main/java/org/apache/sqoop/client/shell/SubmissionStartFunction.java
          • client/src/main/java/org/apache/sqoop/client/core/Constants.java
          • client/src/main/resources/client-resource.properties
          Show
          Hudson added a comment - Integrated in Sqoop2-hadoop200 #264 (See https://builds.apache.org/job/Sqoop2-hadoop200/264/ ) SQOOP-723 : Add possibility to have synchronous submissions in Sqoop 2 (Revision 6ceb22497c0161880a37a7bee51f1de6093e6dcc) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=6ceb22497c0161880a37a7bee51f1de6093e6dcc Files : client/src/main/java/org/apache/sqoop/client/shell/SubmissionStartFunction.java client/src/main/java/org/apache/sqoop/client/core/Constants.java client/src/main/resources/client-resource.properties
          Hide
          Jarek Jarcec Cecho added a comment -

          The failure seems to be more an issue on the jenkins slave rather than a real issue.

          Show
          Jarek Jarcec Cecho added a comment - The failure seems to be more an issue on the jenkins slave rather than a real issue.

            People

            • Assignee:
              Abraham Elmahrek
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development