Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1567

puppet cannot install components whatever given to it

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 1.0.0
    • Component/s: deployment
    • Labels:
      None

      Description

      The fix in BIGTOP-1553 introduce an issue that is variable $components will always be an one item array.
      This can be identified by adding a quick evaluation snippet in bigtop/bigtop-deploy/puppet/manifests/cluster.pp:

      fail("print component[0]: ${$components[0]}")
      

      the output:

      Error: print component[0]: hadoopyarnmapred-app at /bigtop-home/bigtop-deploy/puppet/manifests/cluster.pp:83 on node bigtop1.vagrant
      

      Consequently, the following "if" statement for component installation will not be true because it can not match the specific string.

        Activity

        Hide
        jayunit100 jay vyas added a comment -

        Oh so it's mingling them all into one string, huh?
        Ok I will fix it immediately ....I guess since I tested yarn maybe the transitive dependencies hid this error ....

        Show
        jayunit100 jay vyas added a comment - Oh so it's mingling them all into one string, huh? Ok I will fix it immediately ....I guess since I tested yarn maybe the transitive dependencies hid this error ....
        Hide
        jayunit100 jay vyas added a comment -

        Nvm evans has the fix to recast it to a proper array for multi component. I'll review it when it get attached.

        Show
        jayunit100 jay vyas added a comment - Nvm evans has the fix to recast it to a proper array for multi component. I'll review it when it get attached.
        Hide
        evans_ye Evans Ye added a comment -

        uploaded the patch.
        I've tested it by deploying cluster with single component(hadoop) as well as multiple components(hadoop,yarn,hbase).
        The components are all well installed now

        Show
        evans_ye Evans Ye added a comment - uploaded the patch. I've tested it by deploying cluster with single component(hadoop) as well as multiple components(hadoop,yarn,hbase). The components are all well installed now
        Hide
        jayunit100 jay vyas added a comment -

        thanks again evans for saving the day !
        +1 & commited.

        Show
        jayunit100 jay vyas added a comment - thanks again evans for saving the day ! +1 & commited.
        Hide
        vishnu vishnu gajendran added a comment -

        so, I do not understand the part where you added split($components, ",") in $components_tmp = extlookup("components", split($components, ",")). can you please explain?

        Show
        vishnu vishnu gajendran added a comment - so, I do not understand the part where you added split($components, ",") in $components_tmp = extlookup("components", split($components, ",")). can you please explain?
        Hide
        evans_ye Evans Ye added a comment -

        IIRC, that was for determining whether to transform components_tmp to an array

        if is_array($components_tmp) {
        

        Since originally, any2array will not automatically make an array for us from a , split-ed string,
        thus we still need to split it by ourselves.

        Show
        evans_ye Evans Ye added a comment - IIRC, that was for determining whether to transform components_tmp to an array if is_array($components_tmp) { Since originally, any2array will not automatically make an array for us from a , split-ed string, thus we still need to split it by ourselves.
        Hide
        vishnu vishnu gajendran added a comment -

        okay, that make sense. But, what I do not understand is how extlookup method works in this case. in the puppet docs, I see extlookup("variable name to lookup", "some default value") as the usage. But, in this patch, you try to split $components on comma for the default value but i do not see a $components variable defined anywhere in cluster.pp. please correct me if I am missing something here.

        Show
        vishnu vishnu gajendran added a comment - okay, that make sense. But, what I do not understand is how extlookup method works in this case. in the puppet docs, I see extlookup("variable name to lookup", "some default value") as the usage. But, in this patch, you try to split $components on comma for the default value but i do not see a $components variable defined anywhere in cluster.pp. please correct me if I am missing something here.
        Hide
        evans_ye Evans Ye added a comment -

        I got your point now vishnu gajendran. I'm pretty sure that to set default components is a bug, since if the components has not being set, we'll turn $all to true and install all the components (WTH I was doing?).
        Usually it requires a patch to fix it. But now there is a work to shift extlookup to hiera, which I found the guy might already fixed the bug for me (checkout BIGTOP-1634)
        It would be great if you can help to discover issues on top of that or send us patches to improve it. Anyhow, thanks for catching this!

        Show
        evans_ye Evans Ye added a comment - I got your point now vishnu gajendran . I'm pretty sure that to set default components is a bug, since if the components has not being set, we'll turn $all to true and install all the components (WTH I was doing?). Usually it requires a patch to fix it. But now there is a work to shift extlookup to hiera, which I found the guy might already fixed the bug for me (checkout BIGTOP-1634 ) It would be great if you can help to discover issues on top of that or send us patches to improve it. Anyhow, thanks for catching this!

          People

          • Assignee:
            evans_ye Evans Ye
            Reporter:
            evans_ye Evans Ye
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development