Sqoop
  1. Sqoop
  2. SQOOP-723

Add possibility to have synchronous submissions in Sqoop 2

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 1.99.2
    • Component/s: None
    • Labels:
      None

      Description

      It would be nice if Sqoop 2 would have ability to emulate synchronous submissions from client perspective.

        Issue Links

          Activity

          Jarek Jarcec Cecho created issue -
          Jarek Jarcec Cecho made changes -
          Field Original Value New Value
          Link This issue depends on SQOOP-918 [ SQOOP-918 ]
          Abraham Elmahrek made changes -
          Assignee Abraham Elmahrek [ abec ]
          Hide
          Abraham Elmahrek added a comment - - edited

          Here are two solutions:

          1. Add option to submission start that tells the client to poll until finished. IE: submission start, followed by "synchronous" display of some kind.
          2. Add a new command 'submission submit' that starts a job and polls until finished.

          Also, it seems we should add another option to set how often sqoop should be polling when waiting for submission to finish.

          Show
          Abraham Elmahrek added a comment - - edited Here are two solutions: Add option to submission start that tells the client to poll until finished. IE: submission start, followed by "synchronous" display of some kind. Add a new command 'submission submit' that starts a job and polls until finished. Also, it seems we should add another option to set how often sqoop should be polling when waiting for submission to finish.
          Hide
          Jarek Jarcec Cecho added a comment -

          Hi Abraham Elmahrek,
          thank you very much for looking into this ticket. I personally like the first proposed solution, but without strong preference. It would be also great to do this functionality in SqoopClient API and base the shell implementation on it.

          Jarcec

          Show
          Jarek Jarcec Cecho added a comment - Hi Abraham Elmahrek , thank you very much for looking into this ticket. I personally like the first proposed solution, but without strong preference. It would be also great to do this functionality in SqoopClient API and base the shell implementation on it. Jarcec
          Jarek Jarcec Cecho made changes -
          Remote Link This issue links to "Review board (Web Link)" [ 12144 ]
          Hide
          Jarek Jarcec Cecho added a comment -

          Changing status to "patch available" so that it will show up in our review queue.

          Show
          Jarek Jarcec Cecho added a comment - Changing status to "patch available" so that it will show up in our review queue.
          Jarek Jarcec Cecho made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Jarek Jarcec Cecho added a comment -
          Show
          Jarek Jarcec Cecho added a comment - The patch is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=6ceb22497c0161880a37a7bee51f1de6093e6dcc Thank you Abe for your contribution! Jarcec
          Jarek Jarcec Cecho made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 1.99.2 [ 12323640 ]
          Fix Version/s 2.0.0 [ 12319272 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Sqoop2-hadoop100 #264 (See https://builds.apache.org/job/Sqoop2-hadoop100/264/)
          SQOOP-723: Add possibility to have synchronous submissions in Sqoop 2 (Revision 6ceb22497c0161880a37a7bee51f1de6093e6dcc)

          Result = FAILURE
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=6ceb22497c0161880a37a7bee51f1de6093e6dcc
          Files :

          • client/src/main/resources/client-resource.properties
          • client/src/main/java/org/apache/sqoop/client/core/Constants.java
          • client/src/main/java/org/apache/sqoop/client/shell/SubmissionStartFunction.java
          Show
          Hudson added a comment - Integrated in Sqoop2-hadoop100 #264 (See https://builds.apache.org/job/Sqoop2-hadoop100/264/ ) SQOOP-723 : Add possibility to have synchronous submissions in Sqoop 2 (Revision 6ceb22497c0161880a37a7bee51f1de6093e6dcc) Result = FAILURE jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=6ceb22497c0161880a37a7bee51f1de6093e6dcc Files : client/src/main/resources/client-resource.properties client/src/main/java/org/apache/sqoop/client/core/Constants.java client/src/main/java/org/apache/sqoop/client/shell/SubmissionStartFunction.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop2-hadoop200 #264 (See https://builds.apache.org/job/Sqoop2-hadoop200/264/)
          SQOOP-723: Add possibility to have synchronous submissions in Sqoop 2 (Revision 6ceb22497c0161880a37a7bee51f1de6093e6dcc)

          Result = SUCCESS
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=6ceb22497c0161880a37a7bee51f1de6093e6dcc
          Files :

          • client/src/main/java/org/apache/sqoop/client/shell/SubmissionStartFunction.java
          • client/src/main/java/org/apache/sqoop/client/core/Constants.java
          • client/src/main/resources/client-resource.properties
          Show
          Hudson added a comment - Integrated in Sqoop2-hadoop200 #264 (See https://builds.apache.org/job/Sqoop2-hadoop200/264/ ) SQOOP-723 : Add possibility to have synchronous submissions in Sqoop 2 (Revision 6ceb22497c0161880a37a7bee51f1de6093e6dcc) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=6ceb22497c0161880a37a7bee51f1de6093e6dcc Files : client/src/main/java/org/apache/sqoop/client/shell/SubmissionStartFunction.java client/src/main/java/org/apache/sqoop/client/core/Constants.java client/src/main/resources/client-resource.properties
          Hide
          Jarek Jarcec Cecho added a comment -

          The failure seems to be more an issue on the jenkins slave rather than a real issue.

          Show
          Jarek Jarcec Cecho added a comment - The failure seems to be more an issue on the jenkins slave rather than a real issue.
          Gavin made changes -
          Link This issue depends on SQOOP-918 [ SQOOP-918 ]
          Gavin made changes -
          Link This issue depends upon SQOOP-918 [ SQOOP-918 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          140d 23h 41m 1 Jarek Jarcec Cecho 13/Apr/13 20:27
          Patch Available Patch Available Resolved Resolved
          18m 31s 1 Jarek Jarcec Cecho 13/Apr/13 20:45

            People

            • Assignee:
              Abraham Elmahrek
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development