Sqoop
  1. Sqoop
  2. SQOOP-494

Create client infrastructure to allow interactive selection of options of a command.

    Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: 1.99.1
    • Component/s: None
    • Labels:
      None

      Description

      For example, if a command does not have full options specified, instead of displaying full help, the shell could get into an interactive session to gather the options one by one from the user.

      1. SQOOP-494.patch
        25 kB
        Cheolsoo Park
      2. SQOOP-494.patch
        19 kB
        Cheolsoo Park
      3. SQOOP-494.patch
        14 kB
        Cheolsoo Park
      4. SQOOP-494.patch
        16 kB
        Cheolsoo Park

        Activity

        Hide
        Cheolsoo Park added a comment -

        I have been thinking about how to design this, and this is what I have in mind. I'd like to know what others think.

        1) OptionGroup

        I think that there are 3 types of questions that we can ask via an interactive session:

        • Y/N question
        • M/C question
        • Short answer question

        For example, the "show server" function has 4 options - all (a), host (h), port (p), webapp (w). When the user types "show server" and hits enter, the question should be like this:

        Q1. Select one [a/h/p/w]: _ <enter>
        

        On the other hand, the user types "set server" and hits enter, the question should be like this:

        Q1. Hostname: ___ <enter>
        Q2. Port number: ___ <enter>
        Q3. Web app name: ___ <enter>
        

        Lastly, there is Y/N question such as:

        Q1. Hive import [y/n]: _ <enter>
        

        The challenge is that sometimes multiple options form a question (M/C question), and sometimes one option forms a question (short answer question, Y/N question). Granted, I could always present an option as one question. For example,

        Q1. all [y/n]: _ <enter>
        Q2. host [y/n]: _ <enter>
        Q3. port [y/n]: _ <enter>
        Q4. webapp [y/n]: _ <enter>
        

        But this seems rather strange to me.

        To handle this, I organize options into sub-groups using OptionGroup class. Logically, each OptionGroup represents a group of options that form a question. In my example, options for "show server" (a/h/p/w) are placed in one group while options for "set server" (hostname, port number, web app name) are placed in multiple groups. Therefore, they can be asked as one question and multiple questions respectively.

        2) getOptionsViaInteractiveSession() method

        Interactive session is implemented in a method with the following signature:

        CommandLine getOptionsViaInteractiveSession(IO io, List<Question> questions) // about Question class, see below.
        

        The idea is that each function (e.g. show server, set server, etc) passes a list of Question instances, and this method iterates through them, prints them to, and read input from stdout. At the end, I call parseOptions() with collected input.

        3) Question classes

        I define a hierarchy of classes as follows:

        Question extends OptionGroup
        YesNoQuestion extends Question
        MultiChoiceQuestion extends Question
        ShortAnswerQuestion extends Question
        

        These classes are essentially all OptionGroup, but additionally, they define some question-type-specific behaviors. In particular, Question class defines a method print() that each sub-class overrides. Then, getOptionsViaInteractiveSession() can simply go over a list of Question instances and invokes the print() method to print different types of questions.

        4) From MForm to OptionGroup

        One thing that is not very clear to me now is how to map MInput (Option) to Question (OptionGroup). In the current implementation of MInput, the concept of OptionGroup does not exist. I might have to add additional fields to indicate which group options belong to.

        Please let me know if you have any concerns and/or questions. If everyone is fine with my design, I will go ahead commit the implementation.

        Thanks!

        Show
        Cheolsoo Park added a comment - I have been thinking about how to design this, and this is what I have in mind. I'd like to know what others think. 1) OptionGroup I think that there are 3 types of questions that we can ask via an interactive session: Y/N question M/C question Short answer question For example, the "show server" function has 4 options - all (a), host (h), port (p), webapp (w). When the user types "show server" and hits enter, the question should be like this: Q1. Select one [a/h/p/w]: _ <enter> On the other hand, the user types "set server" and hits enter, the question should be like this: Q1. Hostname: ___ <enter> Q2. Port number: ___ <enter> Q3. Web app name: ___ <enter> Lastly, there is Y/N question such as: Q1. Hive import [y/n]: _ <enter> The challenge is that sometimes multiple options form a question (M/C question), and sometimes one option forms a question (short answer question, Y/N question). Granted, I could always present an option as one question. For example, Q1. all [y/n]: _ <enter> Q2. host [y/n]: _ <enter> Q3. port [y/n]: _ <enter> Q4. webapp [y/n]: _ <enter> But this seems rather strange to me. To handle this, I organize options into sub-groups using OptionGroup class. Logically, each OptionGroup represents a group of options that form a question. In my example, options for "show server" (a/h/p/w) are placed in one group while options for "set server" (hostname, port number, web app name) are placed in multiple groups. Therefore, they can be asked as one question and multiple questions respectively. 2) getOptionsViaInteractiveSession() method Interactive session is implemented in a method with the following signature: CommandLine getOptionsViaInteractiveSession(IO io, List<Question> questions) // about Question class, see below. The idea is that each function (e.g. show server, set server, etc) passes a list of Question instances, and this method iterates through them, prints them to, and read input from stdout. At the end, I call parseOptions() with collected input. 3) Question classes I define a hierarchy of classes as follows: Question extends OptionGroup YesNoQuestion extends Question MultiChoiceQuestion extends Question ShortAnswerQuestion extends Question These classes are essentially all OptionGroup, but additionally, they define some question-type-specific behaviors. In particular, Question class defines a method print() that each sub-class overrides. Then, getOptionsViaInteractiveSession() can simply go over a list of Question instances and invokes the print() method to print different types of questions. 4) From MForm to OptionGroup One thing that is not very clear to me now is how to map MInput (Option) to Question (OptionGroup). In the current implementation of MInput, the concept of OptionGroup does not exist. I might have to add additional fields to indicate which group options belong to. Please let me know if you have any concerns and/or questions. If everyone is fine with my design, I will go ahead commit the implementation. Thanks!
        Hide
        Cheolsoo Park added a comment -

        Attached is the patch that implements my design. I haven't figured out how to upload it to review board.

        Thanks!

        Show
        Cheolsoo Park added a comment - Attached is the patch that implements my design. I haven't figured out how to upload it to review board. Thanks!
        Hide
        Cheolsoo Park added a comment -

        Here is what it looks like:

        1) show server:

        sqoop:000> show server
        Usage: show server
        -a,--all       Display all information
        -h,--host      Display server host name
        -p,--port      Display server port number
        -w,--webapp    Display server web app name
        
        
        Select one [w/p/a/h]: h
        

        2) show connector

        sqoop:000> show connector 
        Usage: show connector
        -a,--all          Display all connectors
        -c,--cid <cid>    Display the connector with cid
        
        Display all connectors [y/n]: y
        

        Note that no further question is asked since we're going to display all connectors. But if the answer is no, we ask another question for the connector id as below:

        sqoop:000> show connector 
        Usage: show connector
        -a,--all          Display all connectors
        -c,--cid <cid>    Display the connector with cid
        
        Display all connectors [y/n]: n
        Display the connector with cid: 12345
        

        3) show version

        sqoop:000> show version 
        Usage: show version
        -a,--all         Display all versions
        -c,--client      Display client version
        -p,--protocol    Display protocol version
        -s,--server      Display server version
        
        
        Select one [s/c/p/a]: a
        

        4) set server

        sqoop:000> set server 
        Usage: set server
        -h,--host <host>         Host name to invoke server resources
        -p,--port <port>         Port number to invoke server resources
        -w,--webapp <webbapp>    Web app to invoke server resources
        
        Host name to invoke server resources: localhost
        Port number to invoke server resources: 8080
        Web app to invoke server resources: sqoop
        
        Show
        Cheolsoo Park added a comment - Here is what it looks like: 1) show server: sqoop:000> show server Usage: show server -a,--all Display all information -h,--host Display server host name -p,--port Display server port number -w,--webapp Display server web app name Select one [w/p/a/h]: h 2) show connector sqoop:000> show connector Usage: show connector -a,--all Display all connectors -c,--cid <cid> Display the connector with cid Display all connectors [y/n]: y Note that no further question is asked since we're going to display all connectors. But if the answer is no, we ask another question for the connector id as below: sqoop:000> show connector Usage: show connector -a,--all Display all connectors -c,--cid <cid> Display the connector with cid Display all connectors [y/n]: n Display the connector with cid: 12345 3) show version sqoop:000> show version Usage: show version -a,--all Display all versions -c,--client Display client version -p,--protocol Display protocol version -s,--server Display server version Select one [s/c/p/a]: a 4) set server sqoop:000> set server Usage: set server -h,--host <host> Host name to invoke server resources -p,--port <port> Port number to invoke server resources -w,--webapp <webbapp> Web app to invoke server resources Host name to invoke server resources: localhost Port number to invoke server resources: 8080 Web app to invoke server resources: sqoop
        Hide
        Vasanth kumar RJ added a comment -

        I would like to suggest few things

        For Show command, shall we have selection more than one options?

        sqoop:000> show server
        Usage: show server
        -a,--all       Display all information
        -h,--host      Display server host name
        -p,--port      Display server port number
        -w,--webapp    Display server web app name
        
        Select [w/p/a/h]: hp
        

        Set server

        sqoop:000> set server
        Usage: set server
        -h,--host <host>         Host name to invoke server resources
        -p,--port <port>         Port number to invoke server resources
        -w,--webapp <webbapp>    Web app to invoke server resources
        -q                       quit interactive
        
        Select [w/p/h/q]: h
        Host name to invoke server resources: localhost
        Select [w/p/h/q]: p
        Port number to invoke server resources: 8080
        Select [w/p/h/q]: wq
        Web app to invoke server resources: sqoop
        

        Then, user should able to turn on/off interactive mode.

        Show
        Vasanth kumar RJ added a comment - I would like to suggest few things For Show command, shall we have selection more than one options? sqoop:000> show server Usage: show server -a,--all Display all information -h,--host Display server host name -p,--port Display server port number -w,--webapp Display server web app name Select [w/p/a/h]: hp Set server sqoop:000> set server Usage: set server -h,--host <host> Host name to invoke server resources -p,--port <port> Port number to invoke server resources -w,--webapp <webbapp> Web app to invoke server resources -q quit interactive Select [w/p/h/q]: h Host name to invoke server resources: localhost Select [w/p/h/q]: p Port number to invoke server resources: 8080 Select [w/p/h/q]: wq Web app to invoke server resources: sqoop Then, user should able to turn on/off interactive mode.
        Hide
        Cheolsoo Park added a comment -

        Hi Vasanth, thank you very much for your input!

        1) Allowing multiple answers is certainly a good idea. I believe that it only requires minor changes to the current implementation.

        2) I also definitely like your idea regarding the "set" command. That's certainly a smarter way to ask questions.

        Let me incorporate your comments in a new patch. Again, thanks a lot!

        Show
        Cheolsoo Park added a comment - Hi Vasanth, thank you very much for your input! 1) Allowing multiple answers is certainly a good idea. I believe that it only requires minor changes to the current implementation. 2) I also definitely like your idea regarding the "set" command. That's certainly a smarter way to ask questions. Let me incorporate your comments in a new patch. Again, thanks a lot!
        Hide
        Cheolsoo Park added a comment - - edited

        I incorporated Vasanth's comments.

        Beside the changes, I also added two junit tests for "show server" and "set server". I used expectj to simulate interactive session, and you can run these tests with maven.

        I didn't add tests for other commands such as "show version" and "show connector" because they require the server as well as the client. As of now, my junit tests only start the client shell via expectj.

        In the long term, I'd like to use expectj in the front-end for all end-to-end tests. I am going to do that work as part of SQOOP-506.

        For now, I see two more minor things that I'd like to improve if time permits:

        1) The order of choices in a m/c question is not sorted. This is because OptionGroup stores options in an unsorted hashmap.
        2) The quit choice 'q' in a m/c question does not stand out. The user might be confused what 'q' is for.

        Please let me know what you think.

        Thanks!

        Show
        Cheolsoo Park added a comment - - edited I incorporated Vasanth's comments. Beside the changes, I also added two junit tests for "show server" and "set server". I used expectj to simulate interactive session, and you can run these tests with maven. I didn't add tests for other commands such as "show version" and "show connector" because they require the server as well as the client. As of now, my junit tests only start the client shell via expectj. In the long term, I'd like to use expectj in the front-end for all end-to-end tests. I am going to do that work as part of SQOOP-506 . For now, I see two more minor things that I'd like to improve if time permits: 1) The order of choices in a m/c question is not sorted. This is because OptionGroup stores options in an unsorted hashmap. 2) The quit choice 'q' in a m/c question does not stand out. The user might be confused what 'q' is for. Please let me know what you think. Thanks!
        Hide
        Cheolsoo Park added a comment -
        Show
        Cheolsoo Park added a comment - Review board: https://reviews.apache.org/r/5626/
        Hide
        Jarek Jarcec Cecho added a comment -

        Since this JIRA has diverged from current sqoop2 development, might I suggest closing it? We can always reopen or file a new one if we will need to address the interactive command selection again.

        Jarcec

        Show
        Jarek Jarcec Cecho added a comment - Since this JIRA has diverged from current sqoop2 development, might I suggest closing it? We can always reopen or file a new one if we will need to address the interactive command selection again. Jarcec
        Hide
        Cheolsoo Park added a comment -

        Sure. I am closing.

        Thanks Jarcec for your comment.

        Show
        Cheolsoo Park added a comment - Sure. I am closing. Thanks Jarcec for your comment.

          People

          • Assignee:
            Cheolsoo Park
            Reporter:
            Arvind Prabhakar
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development