Sqoop
  1. Sqoop
  2. SQOOP-495

Support for parameter substitution in the client shell.

    Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.99.1
    • Component/s: None
    • Labels:
      None

      Description

      For instance the user could specify set foo = bar, following which every reference to $foo will be replaced with bar.

      1. SQOOP-495.patch
        3 kB
        Vasanth kumar RJ

        Activity

        Hide
        Vasanth kumar RJ added a comment -

        Thanks a lot Jarcec

        Vasanth

        Show
        Vasanth kumar RJ added a comment - Thanks a lot Jarcec Vasanth
        Hide
        Jarek Jarcec Cecho added a comment -

        Committed revision 1362238.

        Thank you Vasanth for your contribution!

        Jarcec

        Show
        Jarek Jarcec Cecho added a comment - Committed revision 1362238. Thank you Vasanth for your contribution! Jarcec
        Hide
        Bilung Lee added a comment -

        Some suggestions:

        • How about have a method (eg. "List resolveVariables(List args)") in SqoopCommand to return a list of arguments with variables resolved? Then you can invoke this method at the beginning of the execute method within each subclass of SqoopCommand before the resolved arguments are passed down to each function of the command subclass.
        • The code only considers the variable in the form of "$variable". It would be better to allow the variable to be specified as "$ {variable}

          ". The variable may also appear as a substring, such as "sqoop_$version" and "sqoop_$

          {version}

          ".

        • You may want to check out what and how the build-in "println" supports.
        Show
        Bilung Lee added a comment - Some suggestions: How about have a method (eg. "List resolveVariables(List args)") in SqoopCommand to return a list of arguments with variables resolved? Then you can invoke this method at the beginning of the execute method within each subclass of SqoopCommand before the resolved arguments are passed down to each function of the command subclass. The code only considers the variable in the form of "$variable". It would be better to allow the variable to be specified as "$ {variable} ". The variable may also appear as a substring, such as "sqoop_$version" and "sqoop_$ {version} ". You may want to check out what and how the build-in "println" supports.
        Hide
        Vasanth kumar RJ added a comment -

        Attached patch for this issue. Kindly review.

        I am not able to attach patch in review board for Sqoop 2 version.

        Thanks

        Show
        Vasanth kumar RJ added a comment - Attached patch for this issue. Kindly review. I am not able to attach patch in review board for Sqoop 2 version. Thanks
        Hide
        Bilung Lee added a comment -

        I was suggesting neither ("set" nor "export"). As I mentioned, you already can do the following in Sqoop shell:

        sqoop:000> webapp = "sqoop-server"
        sqoop:000> println "$webapp"
        sqoop-server

        It will be better to leverage this support, i.e. simply webapp = "sqoop-server" (without "set" or "export").

        Show
        Bilung Lee added a comment - I was suggesting neither ("set" nor "export"). As I mentioned, you already can do the following in Sqoop shell: sqoop:000> webapp = "sqoop-server" sqoop:000> println "$webapp" sqoop-server It will be better to leverage this support, i.e. simply webapp = "sqoop-server" (without "set" or "export").
        Hide
        Vasanth kumar RJ added a comment -

        Hi Bilung Lee,
        Thanks for reply. Parameter substitution is going implement on sqoop shell.
        I asked like whether can use "set" command itself for setting parameter
        Example:
        Sqoop-shell>>set webapp="sqoop-server"

        "set" command is already implemented for server and connection properties.
        Sqoop-shell>>set server --host hostname

        So I suggested the new "export" command for parameter substitution
        Sqoop-shell>>export webapp="sqoop-server"

        Kindly suggest if I am wrong.

        Show
        Vasanth kumar RJ added a comment - Hi Bilung Lee, Thanks for reply. Parameter substitution is going implement on sqoop shell. I asked like whether can use "set" command itself for setting parameter Example: Sqoop-shell>>set webapp="sqoop-server" "set" command is already implemented for server and connection properties. Sqoop-shell>>set server --host hostname So I suggested the new "export" command for parameter substitution Sqoop-shell>>export webapp="sqoop-server" Kindly suggest if I am wrong.
        Hide
        Bilung Lee added a comment -

        Groovy shell already provides a way to support variables, for example,
        webapp = "sqoop-server"
        printf "$

        {webapp}"

        Sqoop shell is built on top of Groovy shell, and the above commands also work in Sqoop shell. Thus, it would be better to leverage the variable support in Groovy shell (if possible) and to make Sqoop commands work in a similar way, for example,
        webapp = "sqoop-server"
        set server --webapp ${webapp}
        Show
        Bilung Lee added a comment - Groovy shell already provides a way to support variables, for example, webapp = "sqoop-server" printf "$ {webapp}" Sqoop shell is built on top of Groovy shell, and the above commands also work in Sqoop shell. Thus, it would be better to leverage the variable support in Groovy shell (if possible) and to make Sqoop commands work in a similar way, for example, webapp = "sqoop-server" set server --webapp ${webapp}
        Hide
        Vasanth kumar RJ added a comment -

        Hi,
        Set command implemented already for setting server and connector properties.
        Using same set command, we can set as sqoop environment parameter if it is not server and connector.
        Like:
        set server --host <host> ===> Setting server properties
        set connector --something ===> Setting connector properties
        set username=USER ===> Parameter substitution

        other way, Can we use "export" instead of "set" for parameter substitution ?
        suggest

        Show
        Vasanth kumar RJ added a comment - Hi, Set command implemented already for setting server and connector properties. Using same set command, we can set as sqoop environment parameter if it is not server and connector. Like: set server --host <host> ===> Setting server properties set connector --something ===> Setting connector properties set username=USER ===> Parameter substitution other way, Can we use "export" instead of "set" for parameter substitution ? suggest

          People

          • Assignee:
            Vasanth kumar RJ
            Reporter:
            Arvind Prabhakar
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development