HBase
  1. HBase
  2. HBASE-11441

[hadoopqa] jenkinsEnv paths are stale/wrong; JAVA_HOME in particular

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: build
    • Labels:
      None

      Description

      jenkinsEnv paths are off.

      We moved to jdk1.7 for master branch. Though we'd set hadoopqa to run with java 1.7, the jenkinsEnv overrides what jenkins sets for JAVA_HOME so we've been running 1.6 java over this last couple of months rather than 1.7.

      Trying to work around jenkinsEnv settings reveals that there is no 1.7 at the location jenkins goes to find it at ("Latest JDK 7" option):

      ls: cannot access /home/hudson/tools/java/latest1.7/..: No such file or directory
      

      Messing w/ the options in the jenkins UI and then putting script in text box to expose paths, almost none of the advertised options are available.

      Something changed.

      I got a 1.7 working by setting jdk7-u51 explicitly and then setting PATH and JAVA_HOME just before our test script runs (it downloaded it)

      ...
      ls $JAVA_HOME/..
      which java
      java -version
      echo $JAVA_HOME
      saveJavaHome=$JAVA_HOME
      $JAVA_HOME/bin/java -version
      ulimit -a
      set +x
      source ${WORKSPACE}/dev-support/jenkinsEnv.sh
      export JAVA_HOME="$saveJavaHome"
      export PATH=$JAVA_HOME/bin:$PATH
      which java
      java -version
      echo "Resetting java_home -- FIX jenkinsEnv.sh!!! $JAVA_HOME"
      ....
      

      HBase builds on hadoop1||hadoop2||hadoop3||hadoop8

      I ain't sure how this stuff is supposed to be set. Asking Andrew Bayer on our end (He said 'credits' needed refresh last night but that doesn't seem to have helped here).

        Activity

        Hide
        stack added a comment -

        Talking to Andrew, he is afraid to mess on these boxes. Says the jdk1.7 trick above should work for now.

        Seems like this jenkinsEnv stuff needs revamp but ain't sure what to revamp it too... just yet.

        Show
        stack added a comment - Talking to Andrew, he is afraid to mess on these boxes. Says the jdk1.7 trick above should work for now. Seems like this jenkinsEnv stuff needs revamp but ain't sure what to revamp it too... just yet.
        Hide
        Enis Soztutar added a comment -

        The trunk build is using that path from https://builds.apache.org/view/All/job/HBase-TRUNK/5249/consoleFull:

        + which java
        /home/hudson/tools/java/latest1.7/bin/java
        + java -version
        java version "1.7.0_25"
        Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
        Java HotSpot(TM) Server VM (build 23.25-b01, mixed mode)
        
        Show
        Enis Soztutar added a comment - The trunk build is using that path from https://builds.apache.org/view/All/job/HBase-TRUNK/5249/consoleFull: + which java /home/hudson/tools/java/latest1.7/bin/java + java -version java version "1.7.0_25" Java(TM) SE Runtime Environment (build 1.7.0_25-b15) Java HotSpot(TM) Server VM (build 23.25-b01, mixed mode)
        Hide
        Enis Soztutar added a comment -

        Should we try to set the precommit to use 1.7 latest, then set JAVA_HOME in jenkinsEnv.sh to /home/hudson/tools/java/latest1.7/ ?
        It seems that setting the jdk works for hbase-trunk build.

        Show
        Enis Soztutar added a comment - Should we try to set the precommit to use 1.7 latest, then set JAVA_HOME in jenkinsEnv.sh to /home/hudson/tools/java/latest1.7/ ? It seems that setting the jdk works for hbase-trunk build.
        Hide
        stack added a comment -

        Are they running on the same machines; hadoopqa vs master build? I've not done the research.

        Show
        stack added a comment - Are they running on the same machines; hadoopqa vs master build? I've not done the research.
        Hide
        Enis Soztutar added a comment -

        I think not. Precommit runs in hadoop1||hadoop2||hadoop3||hadoop8 and hbase-trunk runs in ubuntu && !ubuntu4. Although, I though jenkins will provide the JVM if you select it in the job's configuration.

        Show
        Enis Soztutar added a comment - I think not. Precommit runs in hadoop1||hadoop2||hadoop3||hadoop8 and hbase-trunk runs in ubuntu && !ubuntu4 . Although, I though jenkins will provide the JVM if you select it in the job's configuration.
        Hide
        stack added a comment -

        Although, I though jenkins will provide the JVM if you select it in the job's configuration.

        That works sometimes, for some of the options. Other times it fails for want of a login and password against oracle I presume (Sometimes it caches the jdk but I've also seen it do a redownload on each build).

        I don't have the full picture on build infrastructure. Learning it by running jobs just to list directories and their content takes a bunch of time.

        Show
        stack added a comment - Although, I though jenkins will provide the JVM if you select it in the job's configuration. That works sometimes, for some of the options. Other times it fails for want of a login and password against oracle I presume (Sometimes it caches the jdk but I've also seen it do a redownload on each build). I don't have the full picture on build infrastructure. Learning it by running jobs just to list directories and their content takes a bunch of time.
        Hide
        Enis Soztutar added a comment -

        I see. It seems that precommit builds are working now with jdk7-u51. keep this setup, and move on?

        Show
        Enis Soztutar added a comment - I see. It seems that precommit builds are working now with jdk7-u51. keep this setup, and move on?
        Hide
        stack added a comment -

        Example of failed download of JDK:

        Installing JDK jdk-7u51-oth-JPR
        Downloading JDK from http://download.oracle.com/otn/java/jdk/7u51-b13/jdk-7u51-linux-i586.tar.gz
        
        Your Oracle account doesn't appear valid. Please specify a valid username/password
        ERROR: Unable to install JDK unless a valid username/password is provided.
        Retrying after 10 seconds
        
        Installing JDK jdk-7u51-oth-JPR
        Downloading JDK from http://download.oracle.com/otn/java/jdk/7u51-b13/jdk-7u51-linux-i586.tar.gz
        
        Your Oracle account doesn't appear valid. Please specify a valid username/password
        ERROR: Unable to install JDK unless a valid username/password is provided.
        Retrying after 10 seconds
        
        Show
        stack added a comment - Example of failed download of JDK: Installing JDK jdk-7u51-oth-JPR Downloading JDK from http: //download.oracle.com/otn/java/jdk/7u51-b13/jdk-7u51-linux-i586.tar.gz Your Oracle account doesn't appear valid. Please specify a valid username/password ERROR: Unable to install JDK unless a valid username/password is provided. Retrying after 10 seconds Installing JDK jdk-7u51-oth-JPR Downloading JDK from http: //download.oracle.com/otn/java/jdk/7u51-b13/jdk-7u51-linux-i586.tar.gz Your Oracle account doesn't appear valid. Please specify a valid username/password ERROR: Unable to install JDK unless a valid username/password is provided. Retrying after 10 seconds

          People

          • Assignee:
            Unassigned
            Reporter:
            stack
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development