Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.99.3
    • Fix Version/s: 1.99.3
    • Component/s: sqoop2-client
    • Labels:
      None

      Description

      Problem in sqoop.sh scripts.

      $./sqoop.sh server start
      Sqoop home directory: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100...
      ./sqoop.sh: line 78: /home/user/git/sqoop2/dist/target/bin/sqoop-sys.sh: No such file or directory
      Using CATALINA_OPTS:
      Adding to CATALINA_OPTS: -Dsqoop.http.port= -Dsqoop.admin.port=
      Using CATALINA_BASE: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server

      1. SQOOP-1003.patch
        2 kB
        Vasanth kumar RJ
      2. SQOOP-1003.patch
        0.8 kB
        Jarek Jarcec Cecho
      3. SQOOP-1003.patch
        1 kB
        Vasanth kumar RJ

        Issue Links

          Activity

          Hide
          Jarek Jarcec Cecho added a comment -

          The build failures are more Jenkins issues than a real problem, JIRA SQOOP-1026 was created to keep track of that.

          Show
          Jarek Jarcec Cecho added a comment - The build failures are more Jenkins issues than a real problem, JIRA SQOOP-1026 was created to keep track of that.
          Hide
          Hudson added a comment -

          Integrated in Sqoop2-hadoop100 #276 (See https://builds.apache.org/job/Sqoop2-hadoop100/276/)
          SQOOP-1003: Sqoop2: sqoop.sh execution error (Revision 01ffb2f4282b6fac7fa05d6a8f8b4ec0feb76082)

          Result = FAILURE
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=01ffb2f4282b6fac7fa05d6a8f8b4ec0feb76082
          Files :

          • dist/src/main/bin/sqoop.sh
          Show
          Hudson added a comment - Integrated in Sqoop2-hadoop100 #276 (See https://builds.apache.org/job/Sqoop2-hadoop100/276/ ) SQOOP-1003 : Sqoop2: sqoop.sh execution error (Revision 01ffb2f4282b6fac7fa05d6a8f8b4ec0feb76082) Result = FAILURE jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=01ffb2f4282b6fac7fa05d6a8f8b4ec0feb76082 Files : dist/src/main/bin/sqoop.sh
          Hide
          Hudson added a comment -

          Integrated in Sqoop2-hadoop200 #275 (See https://builds.apache.org/job/Sqoop2-hadoop200/275/)
          SQOOP-1003: Sqoop2: sqoop.sh execution error (Revision 01ffb2f4282b6fac7fa05d6a8f8b4ec0feb76082)

          Result = FAILURE
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=01ffb2f4282b6fac7fa05d6a8f8b4ec0feb76082
          Files :

          • dist/src/main/bin/sqoop.sh
          Show
          Hudson added a comment - Integrated in Sqoop2-hadoop200 #275 (See https://builds.apache.org/job/Sqoop2-hadoop200/275/ ) SQOOP-1003 : Sqoop2: sqoop.sh execution error (Revision 01ffb2f4282b6fac7fa05d6a8f8b4ec0feb76082) Result = FAILURE jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=01ffb2f4282b6fac7fa05d6a8f8b4ec0feb76082 Files : dist/src/main/bin/sqoop.sh
          Hide
          Jarek Jarcec Cecho added a comment -

          The patch is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=01ffb2f4282b6fac7fa05d6a8f8b4ec0feb76082

          Thank you Vasanth for all your effort on this ticket!

          Jarcec

          Show
          Jarek Jarcec Cecho added a comment - The patch is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=01ffb2f4282b6fac7fa05d6a8f8b4ec0feb76082 Thank you Vasanth for all your effort on this ticket! Jarcec
          Hide
          Jarek Jarcec Cecho added a comment -

          Vasanth kumar RJ, yeah sure. I left few nits on the review board.

          Show
          Jarek Jarcec Cecho added a comment - Vasanth kumar RJ , yeah sure. I left few nits on the review board.
          Hide
          Vasanth kumar RJ added a comment -

          Sorry. Yup I am fine with your patch. My previously attached patch fix the same with softlink. Can we have this softlink?

          Show
          Vasanth kumar RJ added a comment - Sorry. Yup I am fine with your patch. My previously attached patch fix the same with softlink. Can we have this softlink?
          Hide
          Jarek Jarcec Cecho added a comment -

          Interesting, I was working for me yesterday.

          Anyway, Vasanth kumar RJ do you see any issues with accepting the patch? I think that the portion of code in question was pulled from Ooozie when we were introducing the parametrized ports and it's not needed in fact in our case as we're already doing path resolution.

          Show
          Jarek Jarcec Cecho added a comment - Interesting, I was working for me yesterday. Anyway, Vasanth kumar RJ do you see any issues with accepting the patch? I think that the portion of code in question was pulled from Ooozie when we were introducing the parametrized ports and it's not needed in fact in our case as we're already doing path resolution.
          Hide
          Vasanth kumar RJ added a comment -

          Your new patch working fine but symbolic link script will help.
          I tried fresh clone in my CentOS.

          [vasanthkumar@vasanthkumar bin]$ echo $SHELL
          /bin/bash
          [vasanthkumar@vasanthkumar bin]$ uname -a
          Linux vasanthkumar 2.6.32-358.el6.x86_64 #1 SMP Fri Feb 22 00:31:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
          [vasanthkumar@vasanthkumar bin]$ cat /etc/redhat-release
          CentOS release 6.4 (Final)
          [vasanthkumar@vasanthkumar bin]$ pwd
          /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/bin
          [vasanthkumar@vasanthkumar bin]$ ./sqoop.sh server start
          Sqoop home directory: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100...
          ./sqoop.sh: line 78: /home/vasanthkumar/git/sqoop2/dist/target/bin/sqoop-sys.sh: No such file or directory
          Using CATALINA_OPTS:
          Adding to CATALINA_OPTS: -Dsqoop.http.port= -Dsqoop.admin.port=
          Using CATALINA_BASE: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server
          Using CATALINA_HOME: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server
          Using CATALINA_TMPDIR: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server/temp
          Using JRE_HOME: /usr/java/jdk1.6.0_43
          Using CLASSPATH: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server/bin/bootstrap.jar
          [vasanthkumar@vasanthkumar bin]$ cd ..
          [vasanthkumar@vasanthkumar sqoop-2.0.0-SNAPSHOT-bin-hadoop100]$ ./bin/sqoop.sh server start
          Sqoop home directory: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100...
          Setting SQOOP_HTTP_PORT: 12000
          Setting SQOOP_ADMIN_PORT: 12001
          Using CATALINA_OPTS:
          Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
          Using CATALINA_BASE: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server
          Using CATALINA_HOME: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server
          Using CATALINA_TMPDIR: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server/temp
          Using JRE_HOME: /usr/java/jdk1.6.0_43
          Using CLASSPATH: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server/bin/bootstrap.jar
          [vasanthkumar@vasanthkumar sqoop-2.0.0-SNAPSHOT-bin-hadoop100]$

          [vasanthkumar@vasanthkumar sqoop2]$ git status

          1. On branch sqoop2
            nothing to commit (working directory clean)

          Using Bash shell and CentOS. Above given series of commands, command works only if executed from Sqoop home.

          Show
          Vasanth kumar RJ added a comment - Your new patch working fine but symbolic link script will help. I tried fresh clone in my CentOS. [vasanthkumar@vasanthkumar bin] $ echo $SHELL /bin/bash [vasanthkumar@vasanthkumar bin] $ uname -a Linux vasanthkumar 2.6.32-358.el6.x86_64 #1 SMP Fri Feb 22 00:31:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux [vasanthkumar@vasanthkumar bin] $ cat /etc/redhat-release CentOS release 6.4 (Final) [vasanthkumar@vasanthkumar bin] $ pwd /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/bin [vasanthkumar@vasanthkumar bin] $ ./sqoop.sh server start Sqoop home directory: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100... ./sqoop.sh: line 78: /home/vasanthkumar/git/sqoop2/dist/target/bin/sqoop-sys.sh: No such file or directory Using CATALINA_OPTS: Adding to CATALINA_OPTS: -Dsqoop.http.port= -Dsqoop.admin.port= Using CATALINA_BASE: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server Using CATALINA_HOME: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server Using CATALINA_TMPDIR: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server/temp Using JRE_HOME: /usr/java/jdk1.6.0_43 Using CLASSPATH: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server/bin/bootstrap.jar [vasanthkumar@vasanthkumar bin] $ cd .. [vasanthkumar@vasanthkumar sqoop-2.0.0-SNAPSHOT-bin-hadoop100] $ ./bin/sqoop.sh server start Sqoop home directory: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100... Setting SQOOP_HTTP_PORT: 12000 Setting SQOOP_ADMIN_PORT: 12001 Using CATALINA_OPTS: Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001 Using CATALINA_BASE: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server Using CATALINA_HOME: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server Using CATALINA_TMPDIR: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server/temp Using JRE_HOME: /usr/java/jdk1.6.0_43 Using CLASSPATH: /home/vasanthkumar/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server/bin/bootstrap.jar [vasanthkumar@vasanthkumar sqoop-2.0.0-SNAPSHOT-bin-hadoop100] $ [vasanthkumar@vasanthkumar sqoop2] $ git status On branch sqoop2 nothing to commit (working directory clean) Using Bash shell and CentOS. Above given series of commands, command works only if executed from Sqoop home.
          Hide
          Mark Grover added a comment -

          +1 (non-committer)

          Show
          Mark Grover added a comment - +1 (non-committer)
          Hide
          Jarek Jarcec Cecho added a comment -

          I was actually using fresh clone on Centos and I did not see any issue there yesterday. I was able to replicate that on Ubuntu though.

          I'm attaching patch that fixed the issue for me. Vasanth kumar RJ, would you mind taking a look to see if it fixes the problem for you as well?

          Show
          Jarek Jarcec Cecho added a comment - I was actually using fresh clone on Centos and I did not see any issue there yesterday. I was able to replicate that on Ubuntu though. I'm attaching patch that fixed the issue for me. Vasanth kumar RJ , would you mind taking a look to see if it fixes the problem for you as well?
          Hide
          Vasanth kumar RJ added a comment -

          Hi Jarek,
          I think you are using 1.99.2 release. Defect is in patch SQOOP-990 which is committed in current 1.99.3 and update latest sqoop.sh.
          I am using ubuntu and bash shell.

          Show
          Vasanth kumar RJ added a comment - Hi Jarek, I think you are using 1.99.2 release. Defect is in patch SQOOP-990 which is committed in current 1.99.3 and update latest sqoop.sh. I am using ubuntu and bash shell.
          Hide
          Jarek Jarcec Cecho added a comment -

          Vasanth kumar RJ, I've tried several scenarios (including your examples) and everything seems to be working for me. What operating system and shell are you using?

          Show
          Jarek Jarcec Cecho added a comment - Vasanth kumar RJ , I've tried several scenarios (including your examples) and everything seems to be working for me. What operating system and shell are you using?
          Hide
          Jarek Jarcec Cecho added a comment -

          Hi Vasanth kumar RJ,
          no apologies necessary, I think that you've risen very goot point. I'll take a deeper look.

          Jarcec

          Show
          Jarek Jarcec Cecho added a comment - Hi Vasanth kumar RJ , no apologies necessary, I think that you've risen very goot point. I'll take a deeper look. Jarcec
          Hide
          Vasanth kumar RJ added a comment -

          Hi Jarek,

          After more investigation, below given executed output

          Test 1:
          :~/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/bin$ ./sqoop.sh server start
          Sqoop home directory: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100...
          ./sqoop.sh: line 78: /home/user/git/sqoop2/dist/target/bin/sqoop-sys.sh: No such file or directory
          Using CATALINA_OPTS:
          Adding to CATALINA_OPTS: -Dsqoop.http.port= -Dsqoop.admin.port=
          Using CATALINA_BASE: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server

          Above Test:1 I executed sqoop.sh from bin directory and getting error in line 78. Then sqoop-sys.sh path is incorrect.

          Test 2:
          :~/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100$ ./bin/sqoop.sh server start
          Sqoop home directory: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100...
          Setting SQOOP_HTTP_PORT: 12000
          Setting SQOOP_ADMIN_PORT: 12001
          Using CATALINA_OPTS:
          Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
          Using CATALINA_BASE: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server

          Above Test2: Successfully executed when sqoop.sh executed from home directory.

          Test3:
          :~/git/sqoop2/dist/target$ ./sqoop-2.0.0-SNAPSHOT-bin-hadoop100/bin/sqoop.sh server start
          Sqoop home directory: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100...
          ./sqoop-2.0.0-SNAPSHOT-bin-hadoop100/bin/sqoop.sh: line 76: cd: ./sqoop-2.0.0-SNAPSHOT-bin-hadoop100/bin/..: No such file or directory
          Setting SQOOP_HTTP_PORT: 12000
          Setting SQOOP_ADMIN_PORT: 12001
          Using CATALINA_OPTS:
          Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
          Using CATALINA_BASE: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server

          Above Test3: Again got execution error in line 76. Executed sqoop.sh from outside of sqoop home directory.

          sqoop.sh execute successfully only if executed from sqoop home directory. Please have a try.

          Apologize me for giving such big test result and troubling you.

          Show
          Vasanth kumar RJ added a comment - Hi Jarek, After more investigation, below given executed output Test 1: :~/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/bin$ ./sqoop.sh server start Sqoop home directory: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100... ./sqoop.sh: line 78: /home/user/git/sqoop2/dist/target/bin/sqoop-sys.sh: No such file or directory Using CATALINA_OPTS: Adding to CATALINA_OPTS: -Dsqoop.http.port= -Dsqoop.admin.port= Using CATALINA_BASE: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server Above Test:1 I executed sqoop.sh from bin directory and getting error in line 78. Then sqoop-sys.sh path is incorrect. Test 2: :~/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100$ ./bin/sqoop.sh server start Sqoop home directory: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100... Setting SQOOP_HTTP_PORT: 12000 Setting SQOOP_ADMIN_PORT: 12001 Using CATALINA_OPTS: Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001 Using CATALINA_BASE: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server Above Test2: Successfully executed when sqoop.sh executed from home directory. Test3: :~/git/sqoop2/dist/target$ ./sqoop-2.0.0-SNAPSHOT-bin-hadoop100/bin/sqoop.sh server start Sqoop home directory: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100... ./sqoop-2.0.0-SNAPSHOT-bin-hadoop100/bin/sqoop.sh: line 76: cd: ./sqoop-2.0.0-SNAPSHOT-bin-hadoop100/bin/..: No such file or directory Setting SQOOP_HTTP_PORT: 12000 Setting SQOOP_ADMIN_PORT: 12001 Using CATALINA_OPTS: Adding to CATALINA_OPTS: -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001 Using CATALINA_BASE: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100/server Above Test3: Again got execution error in line 76. Executed sqoop.sh from outside of sqoop home directory. sqoop.sh execute successfully only if executed from sqoop home directory. Please have a try. Apologize me for giving such big test result and troubling you.
          Hide
          Vasanth kumar RJ added a comment -

          Hi Jarek,

          $./sqoop.sh server start
          Sqoop home directory: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100...
          ./sqoop.sh: line 78: /home/user/git/sqoop2/dist/target/bin/sqoop-sys.sh: No such file or directory

          Here, sqoop-sys.sh path is incorrect. Missing "sqoop-2.0.0-SNAPSHOT-bin-hadoop100" directory in path.

          Just now verified that. Already I got sqoop-sys.sh in bin directory. Applied my patch to sqoop.sh alone and now works fine.

          Thanks,
          Vasanth kumar

          Show
          Vasanth kumar RJ added a comment - Hi Jarek, $./sqoop.sh server start Sqoop home directory: /home/user/git/sqoop2/dist/target/sqoop-2.0.0-SNAPSHOT-bin-hadoop100... ./sqoop.sh: line 78: /home/user/git/sqoop2/dist/target/bin/sqoop-sys.sh: No such file or directory Here, sqoop-sys.sh path is incorrect. Missing "sqoop-2.0.0-SNAPSHOT-bin-hadoop100" directory in path. Just now verified that. Already I got sqoop-sys.sh in bin directory. Applied my patch to sqoop.sh alone and now works fine. Thanks, Vasanth kumar
          Hide
          Jarek Jarcec Cecho added a comment -

          Hi Vasanth kumar RJ,
          thank you very much for reporting this issue. Based on the error exception it seems that your installation is missing file sqoop-sys.sh, but that file seems to be correctly available in the repository. Would you mind verifying your checkout?

          Show
          Jarek Jarcec Cecho added a comment - Hi Vasanth kumar RJ , thank you very much for reporting this issue. Based on the error exception it seems that your installation is missing file sqoop-sys.sh , but that file seems to be correctly available in the repository . Would you mind verifying your checkout?
          Hide
          Vasanth kumar RJ added a comment -

          support softlink for both server and client command.

          Show
          Vasanth kumar RJ added a comment - support softlink for both server and client command.

            People

            • Assignee:
              Vasanth kumar RJ
              Reporter:
              Vasanth kumar RJ
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development