HBase
  1. HBase
  2. HBASE-11441

[hadoopqa] jenkinsEnv paths are stale/wrong; JAVA_HOME in particular

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Cannot Reproduce
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: build
    • Labels:
      None

      Description

      jenkinsEnv paths are off.

      We moved to jdk1.7 for master branch. Though we'd set hadoopqa to run with java 1.7, the jenkinsEnv overrides what jenkins sets for JAVA_HOME so we've been running 1.6 java over this last couple of months rather than 1.7.

      Trying to work around jenkinsEnv settings reveals that there is no 1.7 at the location jenkins goes to find it at ("Latest JDK 7" option):

      ls: cannot access /home/hudson/tools/java/latest1.7/..: No such file or directory
      

      Messing w/ the options in the jenkins UI and then putting script in text box to expose paths, almost none of the advertised options are available.

      Something changed.

      I got a 1.7 working by setting jdk7-u51 explicitly and then setting PATH and JAVA_HOME just before our test script runs (it downloaded it)

      ...
      ls $JAVA_HOME/..
      which java
      java -version
      echo $JAVA_HOME
      saveJavaHome=$JAVA_HOME
      $JAVA_HOME/bin/java -version
      ulimit -a
      set +x
      source ${WORKSPACE}/dev-support/jenkinsEnv.sh
      export JAVA_HOME="$saveJavaHome"
      export PATH=$JAVA_HOME/bin:$PATH
      which java
      java -version
      echo "Resetting java_home -- FIX jenkinsEnv.sh!!! $JAVA_HOME"
      ....
      

      HBase builds on hadoop1||hadoop2||hadoop3||hadoop8

      I ain't sure how this stuff is supposed to be set. Asking Andrew Bayer on our end (He said 'credits' needed refresh last night but that doesn't seem to have helped here).

        Issue Links

          Activity

          Hide
          stack added a comment -

          Talking to Andrew, he is afraid to mess on these boxes. Says the jdk1.7 trick above should work for now.

          Seems like this jenkinsEnv stuff needs revamp but ain't sure what to revamp it too... just yet.

          Show
          stack added a comment - Talking to Andrew, he is afraid to mess on these boxes. Says the jdk1.7 trick above should work for now. Seems like this jenkinsEnv stuff needs revamp but ain't sure what to revamp it too... just yet.
          Hide
          Enis Soztutar added a comment -

          The trunk build is using that path from https://builds.apache.org/view/All/job/HBase-TRUNK/5249/consoleFull:

          + which java
          /home/hudson/tools/java/latest1.7/bin/java
          + java -version
          java version "1.7.0_25"
          Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
          Java HotSpot(TM) Server VM (build 23.25-b01, mixed mode)
          
          Show
          Enis Soztutar added a comment - The trunk build is using that path from https://builds.apache.org/view/All/job/HBase-TRUNK/5249/consoleFull: + which java /home/hudson/tools/java/latest1.7/bin/java + java -version java version "1.7.0_25" Java(TM) SE Runtime Environment (build 1.7.0_25-b15) Java HotSpot(TM) Server VM (build 23.25-b01, mixed mode)
          Hide
          Enis Soztutar added a comment -

          Should we try to set the precommit to use 1.7 latest, then set JAVA_HOME in jenkinsEnv.sh to /home/hudson/tools/java/latest1.7/ ?
          It seems that setting the jdk works for hbase-trunk build.

          Show
          Enis Soztutar added a comment - Should we try to set the precommit to use 1.7 latest, then set JAVA_HOME in jenkinsEnv.sh to /home/hudson/tools/java/latest1.7/ ? It seems that setting the jdk works for hbase-trunk build.
          Hide
          stack added a comment -

          Are they running on the same machines; hadoopqa vs master build? I've not done the research.

          Show
          stack added a comment - Are they running on the same machines; hadoopqa vs master build? I've not done the research.
          Hide
          Enis Soztutar added a comment -

          I think not. Precommit runs in hadoop1||hadoop2||hadoop3||hadoop8 and hbase-trunk runs in ubuntu && !ubuntu4. Although, I though jenkins will provide the JVM if you select it in the job's configuration.

          Show
          Enis Soztutar added a comment - I think not. Precommit runs in hadoop1||hadoop2||hadoop3||hadoop8 and hbase-trunk runs in ubuntu && !ubuntu4 . Although, I though jenkins will provide the JVM if you select it in the job's configuration.
          Hide
          stack added a comment -

          Although, I though jenkins will provide the JVM if you select it in the job's configuration.

          That works sometimes, for some of the options. Other times it fails for want of a login and password against oracle I presume (Sometimes it caches the jdk but I've also seen it do a redownload on each build).

          I don't have the full picture on build infrastructure. Learning it by running jobs just to list directories and their content takes a bunch of time.

          Show
          stack added a comment - Although, I though jenkins will provide the JVM if you select it in the job's configuration. That works sometimes, for some of the options. Other times it fails for want of a login and password against oracle I presume (Sometimes it caches the jdk but I've also seen it do a redownload on each build). I don't have the full picture on build infrastructure. Learning it by running jobs just to list directories and their content takes a bunch of time.
          Hide
          Enis Soztutar added a comment -

          I see. It seems that precommit builds are working now with jdk7-u51. keep this setup, and move on?

          Show
          Enis Soztutar added a comment - I see. It seems that precommit builds are working now with jdk7-u51. keep this setup, and move on?
          Hide
          stack added a comment -

          Example of failed download of JDK:

          Installing JDK jdk-7u51-oth-JPR
          Downloading JDK from http://download.oracle.com/otn/java/jdk/7u51-b13/jdk-7u51-linux-i586.tar.gz
          
          Your Oracle account doesn't appear valid. Please specify a valid username/password
          ERROR: Unable to install JDK unless a valid username/password is provided.
          Retrying after 10 seconds
          
          Installing JDK jdk-7u51-oth-JPR
          Downloading JDK from http://download.oracle.com/otn/java/jdk/7u51-b13/jdk-7u51-linux-i586.tar.gz
          
          Your Oracle account doesn't appear valid. Please specify a valid username/password
          ERROR: Unable to install JDK unless a valid username/password is provided.
          Retrying after 10 seconds
          
          Show
          stack added a comment - Example of failed download of JDK: Installing JDK jdk-7u51-oth-JPR Downloading JDK from http: //download.oracle.com/otn/java/jdk/7u51-b13/jdk-7u51-linux-i586.tar.gz Your Oracle account doesn't appear valid. Please specify a valid username/password ERROR: Unable to install JDK unless a valid username/password is provided. Retrying after 10 seconds Installing JDK jdk-7u51-oth-JPR Downloading JDK from http: //download.oracle.com/otn/java/jdk/7u51-b13/jdk-7u51-linux-i586.tar.gz Your Oracle account doesn't appear valid. Please specify a valid username/password ERROR: Unable to install JDK unless a valid username/password is provided. Retrying after 10 seconds
          Hide
          Sean Busbey added a comment -

          closing as obviated by our current JDK woes and the docker work in HBASE-16591

          Show
          Sean Busbey added a comment - closing as obviated by our current JDK woes and the docker work in HBASE-16591

            People

            • Assignee:
              Unassigned
              Reporter:
              stack
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development