Bigtop
  1. Bigtop
  2. BIGTOP-1011

bigtop-detect-javahome has a quirky search order

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: None
    • Labels:
      None

      Description

      If JAVA_HOME is not set, bigtop-detect-javahome has a very quirky search order that can have surprising behavior.

      Some problems:

      • because the "defaults" (like /usr/java/default, /usr/lib/jvm/default-java, /Library/Java/Home) are in the middle of the search list, bigtop will pick up other random JDKs instead of the 'default' the user already has set up on the system.
      • due to use of wildcards, earlier minor versions can be preferred over later minor versions, e.g. /usr/java/jdk1.6* will find 1.6.0_31 before 1.6.0_43, which is probably not what the user wants.
      • 1.6 and 1.7 preference order is not consistent (e.g. generally 1.6 is preferred over 1.7 but not for openJDK)

      This is related to previous issue: https://issues.apache.org/jira/browse/BIGTOP-843

      1. attempt to find java
        if [ -z "$JAVA_HOME" ]; then
        for candidate in \
        /usr/lib/j2sdk1.6-sun \
        /usr/lib/jvm/java-6-sun \
        /usr/lib/jvm/java-1.6.0-sun-1.6.0.* \
        /usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/ \
        /usr/lib/jvm/j2sdk1.6-oracle \
        /usr/lib/jvm/j2sdk1.6-oracle/jre \
        /usr/java/jdk1.6* \
        /usr/java/jre1.6* \
        /usr/java/jdk1.7* \
        /usr/java/jre1.7* \
        /usr/lib/jvm/j2sdk1.7-oracle \
        /usr/lib/jvm/j2sdk1.7-oracle/jre \
        /Library/Java/Home \
        /usr/java/default \
        /usr/lib/jvm/default-java \
        /usr/lib/jvm/java-openjdk \
        /usr/lib/jvm/jre-openjdk \
        /usr/lib/jvm/java-1.7.0-openjdk* \
        /usr/lib/jvm/java-7-openjdk* \
        /usr/lib/jvm/java-7-oracle* \
        /usr/lib/jvm/java-1.6.0-openjdk \
        /usr/lib/jvm/java-1.6.0-openjdk-* \
        /usr/lib/jvm/jre-1.6.0-openjdk* ; do

        Activity

        Hide
        Sean Mackrory added a comment -

        I'll need to do some experimenting to see if I can verify how much of my speculation is true, but here's my speculation anyway:

        • Regarding defaults: Some systems will, by necessity, have a JDK we don't support installed by the host OS, and links like /usr/java/default may be maintained by those packages. In such instances, it is probably correct to default to a version we explicitly support first, then to the generic links, then to other common links that we explicitly don't support.
        • I agree this is a problem, and I'll see if I can tweak our algorithm to use a reverse alphanumeric sort...
        • AFAIK when it comes to Hadoop, Oracle Java 6 > Oracle Java 7, OpenJDK 6 < OpenJDK 7, and Oracle JDK * > OpenJDK *. I may be wrong
        Show
        Sean Mackrory added a comment - I'll need to do some experimenting to see if I can verify how much of my speculation is true, but here's my speculation anyway: Regarding defaults: Some systems will, by necessity, have a JDK we don't support installed by the host OS, and links like /usr/java/default may be maintained by those packages. In such instances, it is probably correct to default to a version we explicitly support first, then to the generic links, then to other common links that we explicitly don't support. I agree this is a problem, and I'll see if I can tweak our algorithm to use a reverse alphanumeric sort... AFAIK when it comes to Hadoop, Oracle Java 6 > Oracle Java 7, OpenJDK 6 < OpenJDK 7, and Oracle JDK * > OpenJDK *. I may be wrong
        Hide
        Sean Mackrory added a comment -

        This patch lets higher versions take precedence over lower versions (unless the order in our list is more specific).

        I found out that /usr/lib/jvm/default-java is actually a symlinked maintained by some Ubuntu packages, and /Library/Java/Home appears to be maintained by some older OS X versions. In these cases, I believe my original statement is correct that it's not safe to let these take precedence over the recommended Oracle JDK's because they're more than likely OpenJDK.

        Regarding the relative order or Oracle / Open JDK 6 and 7 - I think that warrants a closer look still.

        Show
        Sean Mackrory added a comment - This patch lets higher versions take precedence over lower versions (unless the order in our list is more specific). I found out that /usr/lib/jvm/default-java is actually a symlinked maintained by some Ubuntu packages, and /Library/Java/Home appears to be maintained by some older OS X versions. In these cases, I believe my original statement is correct that it's not safe to let these take precedence over the recommended Oracle JDK's because they're more than likely OpenJDK. Regarding the relative order or Oracle / Open JDK 6 and 7 - I think that warrants a closer look still.
        Hide
        Mark Grover added a comment -

        Took a brief look, I don't think you meant to do "echo $candidate"
        Also, you probably want to break out of the for loop or do something similar.

        This JIRA raises some important questions. While I haven't really gotten a chance to think fully think about the pros and cons, I am tending to agree that prioritizing defaults may not be the best thing to do.

        And, of course, we would need add some more entries to the list (related to Java7 and Open JDK like Jolly suggested in the JIRA description.

        Show
        Mark Grover added a comment - Took a brief look, I don't think you meant to do "echo $candidate" Also, you probably want to break out of the for loop or do something similar. This JIRA raises some important questions. While I haven't really gotten a chance to think fully think about the pros and cons, I am tending to agree that prioritizing defaults may not be the best thing to do. And, of course, we would need add some more entries to the list (related to Java7 and Open JDK like Jolly suggested in the JIRA description.
        Hide
        Sean Mackrory added a comment -

        Took a brief look, I don't think you meant to do "echo $candidate"
        Also, you probably want to break out of the for loop or do something similar.

        Thanks for reviewing, Mark. The echo's presence and the break's absence were for testing which JDKs get detected, but both have been fixed.

        This JIRA raises some important questions. While I haven't really gotten a chance to think fully think about the pros and cons, I am tending to agree that prioritizing defaults may not be the best thing to do.

        Correctly prioritized defaults are better than incorrectly prioritized defaults, but if you're suggesting that we should get people to explicitly set JAVA_HOME, then I absolutely agree. Since other packages can drag in JDK dependencies, a routine upgrade or installation can slip a different JDK in front of the one intended for Bigtop. It bothers me when serious users do not have this set, and I think we should do something to make that process a more obvious step.

        Show
        Sean Mackrory added a comment - Took a brief look, I don't think you meant to do "echo $candidate" Also, you probably want to break out of the for loop or do something similar. Thanks for reviewing, Mark. The echo's presence and the break's absence were for testing which JDKs get detected, but both have been fixed. This JIRA raises some important questions. While I haven't really gotten a chance to think fully think about the pros and cons, I am tending to agree that prioritizing defaults may not be the best thing to do. Correctly prioritized defaults are better than incorrectly prioritized defaults, but if you're suggesting that we should get people to explicitly set JAVA_HOME, then I absolutely agree. Since other packages can drag in JDK dependencies, a routine upgrade or installation can slip a different JDK in front of the one intended for Bigtop. It bothers me when serious users do not have this set, and I think we should do something to make that process a more obvious step.
        Hide
        Mark Grover added a comment -

        Sean, to your latter point: I agree with you and would vouch for our users to set JAVA_HOME (in the /etc/defaults file) explicitly whenever and wherever possible.

        Show
        Mark Grover added a comment - Sean, to your latter point: I agree with you and would vouch for our users to set JAVA_HOME (in the /etc/defaults file) explicitly whenever and wherever possible.
        Hide
        Sean Mackrory added a comment -

        So are there any remaining concerns with this? I've fixed the sorting issue, but I believe the number versions and the generic symlinks are all ordered correctly for the reasons cited above.

        Show
        Sean Mackrory added a comment - So are there any remaining concerns with this? I've fixed the sorting issue, but I believe the number versions and the generic symlinks are all ordered correctly for the reasons cited above.
        Hide
        Mark Grover added a comment -

        +1

        Sean, can you commit this please?

        Show
        Mark Grover added a comment - +1 Sean, can you commit this please?
        Hide
        Sean Mackrory added a comment -

        Committed.

        Show
        Sean Mackrory added a comment - Committed.

          People

          • Assignee:
            Sean Mackrory
            Reporter:
            Jolly Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development