Apache Whirr (retired)
  1. Apache Whirr (retired)
  2. WHIRR-649

Make install_cdh_hadoop.sh idempotent, fast failing if already run

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.1
    • Fix Version/s: 0.8.1
    • Component/s: None
    • Labels:
      None

      Description

      Currently, install_cdh_hadoop.sh does not check it has already run and return, unlike install_hadoop.sh.

      This is inefficient for services that would like to install CDH components which have often already been installed (eg Cloudera Manager).

      1. WHIRR-649.patch
        0.6 kB
        Graham Gear

        Activity

        Hide
        Graham Gear added a comment - - edited

        Actually, I can see now from the code that the install pipeline is protected from running a script multiple times so this is not an issue.

        This does however optimise install_cdh* scripts that require base CDH installations (eg HBase/YARN etc), once updated to also check INSTALL_HADOOP_DONE, so is still a worthwhile patch.

        Show
        Graham Gear added a comment - - edited Actually, I can see now from the code that the install pipeline is protected from running a script multiple times so this is not an issue. This does however optimise install_cdh* scripts that require base CDH installations (eg HBase/YARN etc), once updated to also check INSTALL_HADOOP_DONE, so is still a worthwhile patch.
        Hide
        Graham Gear added a comment -

        I also note, that since a install_hadoop invoked install_cdh_hadoop script includes a tarball paramater, it qualifies as not equal to a parameterless install_cdh_hadoop statement and therefore results in 2 invocations of this function, Eg:

        bootstrap-*.sh
        install_oab_java || exit 1
        
        install_cdh_hadoop -u http://apache.osuosl.org/hadoop/common/hadoop-0.20.205.0/hadoop-0.20.205.0.tar.gz || exit 1
        
        install_cdh_hadoop || exit 1
        
        install_cm || exit 1
        

        This patch protects against running this function twice, although there is a bit of code smell around this.

        Show
        Graham Gear added a comment - I also note, that since a install_hadoop invoked install_cdh_hadoop script includes a tarball paramater, it qualifies as not equal to a parameterless install_cdh_hadoop statement and therefore results in 2 invocations of this function, Eg: bootstrap-*.sh install_oab_java || exit 1 install_cdh_hadoop -u http: //apache.osuosl.org/hadoop/common/hadoop-0.20.205.0/hadoop-0.20.205.0.tar.gz || exit 1 install_cdh_hadoop || exit 1 install_cm || exit 1 This patch protects against running this function twice, although there is a bit of code smell around this.
        Hide
        Tom White added a comment -

        +1 The patch is an improvement.

        Show
        Tom White added a comment - +1 The patch is an improvement.
        Hide
        Tom White added a comment -

        I just committed this. Thanks Graham!

        Show
        Tom White added a comment - I just committed this. Thanks Graham!

          People

          • Assignee:
            Graham Gear
            Reporter:
            Graham Gear
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development