Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 1.0.0
    • Component/s: deployment
    • Labels:
      None

      Description

      Currently bigtop-deploy/puppet module supports Puppet 2.x only. In particular, this is because of using dynamic scoping which was deprecated since 2.5 and not supported in 3.x. This task is to get rid of dynamic scoping to make bigtop-deploy/puppet working with Puppet 3.x.

      More on dynamic scoping in Puppet: http://docs.puppetlabs.com/guides/scope_and_puppet.html

      1. 0001-BIGTOP-1047.-Support-Puppet-3.x.patch
        46 kB
        Konstantin Boudnik
      2. BIGTOP-1047.3.patch
        46 kB
        Evans Ye
      3. BIGTOP-1047--n2.patch
        44 kB
        Andrey Klochkov
      4. BIGTOP-1047.patch
        40 kB
        Andrey Klochkov

        Issue Links

        1.
        Fix future dynamic lookups Sub-task Resolved Unassigned
         

          Activity

          Andrey Klochkov created issue -
          Andrey Klochkov made changes -
          Field Original Value New Value
          Description Currently bigtop-deploy/puppet module supports Puppet 2.x only. In particular, this is because of using dynamic scoping which was deprecated since 2.5 and not supported in 3.x. This task is to get rid of dynamics scoping to make bigtop-deploy/puppet working with Puppet 3.x.

          More on dynamics scoping in Puppet: http://docs.puppetlabs.com/guides/scope_and_puppet.html
          Currently bigtop-deploy/puppet module supports Puppet 2.x only. In particular, this is because of using dynamic scoping which was deprecated since 2.5 and not supported in 3.x. This task is to get rid of dynamic scoping to make bigtop-deploy/puppet working with Puppet 3.x.

          More on dynamics scoping in Puppet: http://docs.puppetlabs.com/guides/scope_and_puppet.html
          Andrey Klochkov made changes -
          Description Currently bigtop-deploy/puppet module supports Puppet 2.x only. In particular, this is because of using dynamic scoping which was deprecated since 2.5 and not supported in 3.x. This task is to get rid of dynamic scoping to make bigtop-deploy/puppet working with Puppet 3.x.

          More on dynamics scoping in Puppet: http://docs.puppetlabs.com/guides/scope_and_puppet.html
          Currently bigtop-deploy/puppet module supports Puppet 2.x only. In particular, this is because of using dynamic scoping which was deprecated since 2.5 and not supported in 3.x. This task is to get rid of dynamic scoping to make bigtop-deploy/puppet working with Puppet 3.x.

          More on dynamic scoping in Puppet: http://docs.puppetlabs.com/guides/scope_and_puppet.html
          Andrey Klochkov made changes -
          Attachment BIGTOP-1047.patch [ 12597206 ]
          Andrey Klochkov made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Andrey Klochkov added a comment -

          Attached the patch. I'm not experienced with Puppet, so the patch needs to be reviewed by someone more familiar with it. The patch is tested in my environment (centos63, puppet 3.2.3, kerberos support is not enabled).

          Show
          Andrey Klochkov added a comment - Attached the patch. I'm not experienced with Puppet, so the patch needs to be reviewed by someone more familiar with it. The patch is tested in my environment (centos63, puppet 3.2.3, kerberos support is not enabled).
          Hide
          Andrey Klochkov added a comment -

          Updating the patch with additional fixes

          Show
          Andrey Klochkov added a comment - Updating the patch with additional fixes
          Andrey Klochkov made changes -
          Attachment BIGTOP-1047--n2.patch [ 12598552 ]
          Hide
          Konstantin Boudnik added a comment -

          Anyone with Puppet 3.x knowledge can look at this?

          Show
          Konstantin Boudnik added a comment - Anyone with Puppet 3.x knowledge can look at this?
          Hide
          Evans Ye added a comment -

          Hi folks,
          I tried the patch out and ran through some tests against puppet-3.6.1.
          The patch is good except it's a little bit out of date, for example, spark didn't covered in the patch, which is reasonable since spark didn't integrated into bigtop at the time patch was made.
          We can fix it by update those dynamic lookup variables to puppet 3 syntax.
          Andrey Klochkov can you upload a new patch?
          I can do that only if you don't have time to deal with it.

          Show
          Evans Ye added a comment - Hi folks, I tried the patch out and ran through some tests against puppet-3.6.1. The patch is good except it's a little bit out of date, for example, spark didn't covered in the patch, which is reasonable since spark didn't integrated into bigtop at the time patch was made. We can fix it by update those dynamic lookup variables to puppet 3 syntax. Andrey Klochkov can you upload a new patch? I can do that only if you don't have time to deal with it.
          Konstantin Boudnik made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Nate DAmico added a comment -

          Looks like this issue is most up to date with 3.x dev efforts

          In discussing with Roman Shaposhnik our org can take on the lead going forward with puppet updating. We have been using parts of the bigtop modules for sometime in our service engagements and have begun to productize around it.

          We probably want to create some sub-tasks to incrementally start getting things in check. First would be picking initial service to upgrade and harmonize design patter for 3.x as a couple approaches. Combined with this would be for group to decide on the "canonical" configuration parameters and defaults to be shipped with.

          From there we can use as initial as template and roll through the other services. We will also aim to get some latest efforts we have underway with Accumulo and Spark/Shark

          Show
          Nate DAmico added a comment - Looks like this issue is most up to date with 3.x dev efforts In discussing with Roman Shaposhnik our org can take on the lead going forward with puppet updating. We have been using parts of the bigtop modules for sometime in our service engagements and have begun to productize around it. We probably want to create some sub-tasks to incrementally start getting things in check. First would be picking initial service to upgrade and harmonize design patter for 3.x as a couple approaches. Combined with this would be for group to decide on the "canonical" configuration parameters and defaults to be shipped with. From there we can use as initial as template and roll through the other services. We will also aim to get some latest efforts we have underway with Accumulo and Spark/Shark
          Hide
          Konstantin Boudnik added a comment -

          That'd be great guys! How do you see this going forward? First, bringing existing recipes to puppet 3.x, so we can immediately switch over to the newer system in the trunk? And then gradually improve them?

          Or you perhaps want to start doing a more massive improvements immediately? In which case perhaps doing this on a branch would be less disruptive? What's the plan?

          Show
          Konstantin Boudnik added a comment - That'd be great guys! How do you see this going forward? First, bringing existing recipes to puppet 3.x, so we can immediately switch over to the newer system in the trunk? And then gradually improve them? Or you perhaps want to start doing a more massive improvements immediately? In which case perhaps doing this on a branch would be less disruptive? What's the plan?
          Hide
          Richard Pelavin added a comment -

          Suggested plan would we do this first on a branch and start with basic building blocks: hdfs, yarn and zookeeper and then we can get a sense about any backward compatibility issues.

          Show
          Richard Pelavin added a comment - Suggested plan would we do this first on a branch and start with basic building blocks: hdfs, yarn and zookeeper and then we can get a sense about any backward compatibility issues.
          Hide
          Konstantin Boudnik added a comment -

          Sounds good to me!

          Show
          Konstantin Boudnik added a comment - Sounds good to me!
          Hide
          Roman Shaposhnik added a comment -

          Richard Pelavin Nate DAmico would love to see this come into a branch!

          Show
          Roman Shaposhnik added a comment - Richard Pelavin Nate DAmico would love to see this come into a branch!
          Hide
          Evans Ye added a comment -

          Just want to confirm aoubt the direction.
          It sounds awesome to have such energize development on supporting puppet 3.x.
          But should we temporary focus on improving other parts instead of the puppet recipes like BIGTOP-1336?
          I mean, if we still try to improve current puppet like bugfixing, supporting journal based ha, etc, this might cause duplicate effort. Such development also may not fit in a re-architected puppet recipes.

          Show
          Evans Ye added a comment - Just want to confirm aoubt the direction. It sounds awesome to have such energize development on supporting puppet 3.x. But should we temporary focus on improving other parts instead of the puppet recipes like BIGTOP-1336 ? I mean, if we still try to improve current puppet like bugfixing, supporting journal based ha, etc, this might cause duplicate effort. Such development also may not fit in a re-architected puppet recipes.
          Hide
          Konstantin Boudnik added a comment -

          Perhaps, a mix of both is possible? Improving the current design can go on the master, while 3.x development can be done on a branch. It might be to much to carry on though...

          Show
          Konstantin Boudnik added a comment - Perhaps, a mix of both is possible? Improving the current design can go on the master, while 3.x development can be done on a branch. It might be to much to carry on though...
          Hide
          Richard Pelavin added a comment -

          Good points. How about I focus mainly on 3.x, but can also coordinate on the updates to the current puppet modules. Evans, should I coordinate with you on this? If you want I can hop on a call early next week to work out more detail.

          Show
          Richard Pelavin added a comment - Good points. How about I focus mainly on 3.x, but can also coordinate on the updates to the current puppet modules. Evans, should I coordinate with you on this? If you want I can hop on a call early next week to work out more detail.
          Hide
          Evans Ye added a comment -

          Thanks Richard Pelavin, the proposed strategy sounds good to me. I think I can ping you if I get any improvement on the master. Then we can work together and have some discussion to better get things sorted out. Since the iteration here probably may not be rolling faster than yours, ideally it won't be a frequent change that distract your major work on puppet 3.x branch. Sorry to be negative at the beginning. Let's look positive first and see how things going.

          Show
          Evans Ye added a comment - Thanks Richard Pelavin , the proposed strategy sounds good to me. I think I can ping you if I get any improvement on the master. Then we can work together and have some discussion to better get things sorted out. Since the iteration here probably may not be rolling faster than yours, ideally it won't be a frequent change that distract your major work on puppet 3.x branch. Sorry to be negative at the beginning. Let's look positive first and see how things going .
          Hide
          Konstantin Boudnik added a comment -

          I think the important part is to make sure that individual schedules do not block other people's effort. Hence, if 3.x support isn't affected by the possible changes in the current master - anything radical yet to be seen, honestly - then 3.x effort should proceed on a branch.

          Show
          Konstantin Boudnik added a comment - I think the important part is to make sure that individual schedules do not block other people's effort. Hence, if 3.x support isn't affected by the possible changes in the current master - anything radical yet to be seen, honestly - then 3.x effort should proceed on a branch.
          Hide
          Konstantin Boudnik added a comment -

          I mean, if we still try to improve current puppet like bugfixing, supporting journal based ha, etc, this might cause duplicate effort. Such development also may not fit in a re-architected puppet recipes.

          It is a pretty valid point, but it'd pretty pitiful to keep everything on hold because journal-based HA might come in at some point.
          Besides, the whole point of branch development is to make sure that new changes can be seamlessly applied on top of the current master development.

          Show
          Konstantin Boudnik added a comment - I mean, if we still try to improve current puppet like bugfixing, supporting journal based ha, etc, this might cause duplicate effort. Such development also may not fit in a re-architected puppet recipes. It is a pretty valid point, but it'd pretty pitiful to keep everything on hold because journal-based HA might come in at some point. Besides, the whole point of branch development is to make sure that new changes can be seamlessly applied on top of the current master development.
          Hide
          Nate DAmico added a comment -

          We are in home stretch of a project at the moment, will probably start diving into 3.x post Spark Summit.

          Evans Ye if you are working away on BIGTOP-1336 and have stuff to merge in the next 2-3 wks create a new "merge to 3.x branch" task and link to this one and 1336. We will pull in as necessary.

          One item group should decide on in near future is continued support for 2.7, think the current target end-of-life is at/near puppetconf this year which is in September.

          Show
          Nate DAmico added a comment - We are in home stretch of a project at the moment, will probably start diving into 3.x post Spark Summit. Evans Ye if you are working away on BIGTOP-1336 and have stuff to merge in the next 2-3 wks create a new "merge to 3.x branch" task and link to this one and 1336. We will pull in as necessary. One item group should decide on in near future is continued support for 2.7, think the current target end-of-life is at/near puppetconf this year which is in September.
          Hide
          Konstantin Boudnik added a comment -

          Guys, I don't think any specific tasks need to be created - you be merging with the master anyway Won't you?

          Show
          Konstantin Boudnik added a comment - Guys, I don't think any specific tasks need to be created - you be merging with the master anyway Won't you?
          Hide
          Nate DAmico added a comment -

          Only would be needed if 2.7 additions could be borrowed, we would merge into 3.x branch as dev continues, then when done decide if maintaining multiple implementations (2.7, 3.x)

          Show
          Nate DAmico added a comment - Only would be needed if 2.7 additions could be borrowed, we would merge into 3.x branch as dev continues, then when done decide if maintaining multiple implementations (2.7, 3.x)
          Hide
          Konstantin Boudnik added a comment -

          Right, and during the rebase you'll be able to pick-up what you want and keep out all unwanted stuff. No?

          Show
          Konstantin Boudnik added a comment - Right, and during the rebase you'll be able to pick-up what you want and keep out all unwanted stuff. No?
          Hide
          Nate DAmico added a comment -

          correct, then group would decide after everything is upgraded/rebased the repo looks like:

          bigtop/bigtop-deploy/puppet (only maintaining single puppet implementation)

          or

          bigtop/bigtop-deploy/puppet-3
          bigtop/bigtop-deploy/puppet-2.7

          Show
          Nate DAmico added a comment - correct, then group would decide after everything is upgraded/rebased the repo looks like: bigtop/bigtop-deploy/puppet (only maintaining single puppet implementation) or bigtop/bigtop-deploy/puppet-3 bigtop/bigtop-deploy/puppet-2.7
          Hide
          Konstantin Boudnik added a comment -

          Right. I have a feeling though that we might want to go to puppet3 all the way, as we aren't "supporting" any customers but ourselves on that 2.7 stuff anyways.

          Show
          Konstantin Boudnik added a comment - Right. I have a feeling though that we might want to go to puppet3 all the way, as we aren't "supporting" any customers but ourselves on that 2.7 stuff anyways.
          Hide
          Roman Shaposhnik added a comment -

          +1 to wha Konstantin Boudnik has said. Once we have puppet-3 code in a branch I think we will just cut over.

          Show
          Roman Shaposhnik added a comment - +1 to wha Konstantin Boudnik has said. Once we have puppet-3 code in a branch I think we will just cut over.
          Hide
          Richard Pelavin added a comment - - edited

          As a first step in the task of refactoring the puppet modules to handle puppet >=3.x, wanted to start with a design approach for handling attribute settings. Key goal is using defaults, but to make it easy to override any attribute that can appear in any hadoop config file. The current puppet implementation had a nice way to deal with this using the Puppet extlookup function. Changes are needed to handle Puppet >=3.x which no longer supports dynamic scoping and to take advantage of Puppet functionality, such as Hiera that provides more convenient capabilities. Will shortly propose an approach that takes the Hiera design pattern and will support Hiera, but also will support other representations that encode the same information.

          The approach I am proposing to embark on is to separate this into a number of sub-task

          • Propose a global namespace for describing hadoop config attributes that will be applicable to a wide range of hadoop topologies with various mixtures of services; I will look through all the related work in Bigtop and leverage anything done here already.
          • Put in a simple mechanism that better integrates
            • the way Linux packages include reference config files, and
            • templates (e.g Erubis, Jinja2) used by config management systems
          • that tend to override and ignore what is packaged in the Linux package. This seems like a unique opportunity for a project that handles both packaging and configuration to better align the two. I will elaborate shortly, but this would entail putting inside the packages themselves either templates or a simple meta description of what parameters can be possibly set and their defaults. An important consideration is making this so it can be incrementally and gracefully folded in; so a key design goal would be to handle for example just certain config fragments that are typically parametrized and falling back on existing approach of having a sample config files with defaults that you copy over. As a reference point will look at Augeas, which has both some successes and some challenges
          • Implement using this approach first HDFS and refine approach before moving on to the other services. Yarn we be treated second.
          Show
          Richard Pelavin added a comment - - edited As a first step in the task of refactoring the puppet modules to handle puppet >=3.x, wanted to start with a design approach for handling attribute settings. Key goal is using defaults, but to make it easy to override any attribute that can appear in any hadoop config file. The current puppet implementation had a nice way to deal with this using the Puppet extlookup function. Changes are needed to handle Puppet >=3.x which no longer supports dynamic scoping and to take advantage of Puppet functionality, such as Hiera that provides more convenient capabilities. Will shortly propose an approach that takes the Hiera design pattern and will support Hiera, but also will support other representations that encode the same information. The approach I am proposing to embark on is to separate this into a number of sub-task Propose a global namespace for describing hadoop config attributes that will be applicable to a wide range of hadoop topologies with various mixtures of services; I will look through all the related work in Bigtop and leverage anything done here already. Put in a simple mechanism that better integrates the way Linux packages include reference config files, and templates (e.g Erubis, Jinja2) used by config management systems that tend to override and ignore what is packaged in the Linux package. This seems like a unique opportunity for a project that handles both packaging and configuration to better align the two. I will elaborate shortly, but this would entail putting inside the packages themselves either templates or a simple meta description of what parameters can be possibly set and their defaults. An important consideration is making this so it can be incrementally and gracefully folded in; so a key design goal would be to handle for example just certain config fragments that are typically parametrized and falling back on existing approach of having a sample config files with defaults that you copy over. As a reference point will look at Augeas, which has both some successes and some challenges Implement using this approach first HDFS and refine approach before moving on to the other services. Yarn we be treated second.
          Hide
          Roman Shaposhnik added a comment -

          Richard Pelavin very much in favor of the approach and looking for a patch! Especially like the following bit:

          but this would entail putting inside the packages themselves either templates or a simple meta description of what parameters can be possibly set and their defaults

          Show
          Roman Shaposhnik added a comment - Richard Pelavin very much in favor of the approach and looking for a patch! Especially like the following bit: but this would entail putting inside the packages themselves either templates or a simple meta description of what parameters can be possibly set and their defaults
          Konstantin Boudnik made changes -
          Fix Version/s 0.9.0 [ 12326836 ]
          Hide
          Konstantin Boudnik added a comment -

          Guys, can we speed up this somewhat? I have tried to do the deployment on Ubuntu 14 today and miserably failed with de-facto Puppet 3 and inability to install Puppet 2.7, because it won't work with Ruby 1.9.3

          Looks like we really need 3.x support. I don't think we should wait until the perfect solution will come around. Can folks review the original patch to this ticket? Will it help us to migrate to the later Puppet?

          Show
          Konstantin Boudnik added a comment - Guys, can we speed up this somewhat? I have tried to do the deployment on Ubuntu 14 today and miserably failed with de-facto Puppet 3 and inability to install Puppet 2.7, because it won't work with Ruby 1.9.3 Looks like we really need 3.x support. I don't think we should wait until the perfect solution will come around. Can folks review the original patch to this ticket? Will it help us to migrate to the later Puppet?
          Hide
          Evans Ye added a comment - - edited

          I've reviewed the original patch several month ago, the patch is good except it require some minor fix on dynamic lookup variables which was introduced by code change since the patch was made.
          If no one has time slot to do it, I can try to provide a simple patch to make this work.
          Just let me recall my previous work first

          Show
          Evans Ye added a comment - - edited I've reviewed the original patch several month ago, the patch is good except it require some minor fix on dynamic lookup variables which was introduced by code change since the patch was made. If no one has time slot to do it, I can try to provide a simple patch to make this work. Just let me recall my previous work first
          Hide
          Konstantin Boudnik added a comment -

          that'd be great! And sorry for not committing this earlier - I am not sure what had happened there.

          Show
          Konstantin Boudnik added a comment - that'd be great! And sorry for not committing this earlier - I am not sure what had happened there.
          Evans Ye made changes -
          Attachment BIGTOP-1047.3.patch [ 12678232 ]
          Hide
          Evans Ye added a comment -

          Hi Konstantin Boudnik,
          Never mind, I didn't upload any patch at that time though.
          Now, just uploaded patch version 3, which makes our puppet recipes work with puppet 3.
          I've tested it by docker-vagrant and vagrant-puppet , so it sure just work for you, too

          Show
          Evans Ye added a comment - Hi Konstantin Boudnik , Never mind, I didn't upload any patch at that time though. Now, just uploaded patch version 3, which makes our puppet recipes work with puppet 3. I've tested it by docker-vagrant and vagrant-puppet , so it sure just work for you, too
          Evans Ye made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Konstantin Boudnik added a comment - - edited

          Evans Ye, how are you getting around a need for FQDN in the docker? Are you using a later than docker v1.2 or something? Seems like I can not using --net=host --hostname="FQDN" in the default Ubuntu14.04 (v1.0.1)

          But at least I don't see any Puppet errors so far

          Show
          Konstantin Boudnik added a comment - - edited Evans Ye , how are you getting around a need for FQDN in the docker? Are you using a later than docker v1.2 or something? Seems like I can not using --net=host --hostname="FQDN" in the default Ubuntu14.04 (v1.0.1) But at least I don't see any Puppet errors so far
          Hide
          Konstantin Boudnik added a comment -

          Ok, with Docker 1.3+ I can now have an FQDN in the container! Let's document that Docker before 1.2 won't work for the puppet deployment.

          Now, I see the following warnings with the patch applied:

          Warning: The use of 'import' is deprecated at /ws/bigtop-deploy/puppet/manifests/site.pp:46. See http://links.puppetlabs.com/puppet-import-deprecation
             (at grammar.ra:610:in `_reduce_190')
          ....
          Warning: Variable access via 'port' is deprecated. Use '@port' instead. template[/ws/bigtop-deploy/puppet/modules/solr/templates/solr]:16
             (at /ws/bigtop-deploy/puppet/modules/solr/templates/solr:16:in `result')
          Warning: Variable access via 'port_admin' is deprecated. Use '@port_admin' instead. template[/ws/bigtop-deploy/puppet/modules/solr/templates/solr]:17
             (at /ws/bigtop-deploy/puppet/modules/solr/templates/solr:17:in `result')
          Warning: Variable access via 'zk' is deprecated. Use '@zk' instead. template[/ws/bigtop-deploy/puppet/modules/solr/templates/solr]:18
             (at /ws/bigtop-deploy/puppet/modules/solr/templates/solr:18:in `result')
          Warning: Variable access via 'root_url' is deprecated. Use '@root_url' instead. template[/ws/bigtop-deploy/puppet/modules/solr/templates/solr]:19
             (at /ws/bigtop-deploy/puppet/modules/solr/templates/solr:19:in `result')
          ....
          Warning: The package type's allow_virtual parameter will be changing its default value from false to true in a future release. If you do not want to allow virtual packages, please explicitly set allow_virtual to false.
             (at /usr/lib/ruby/site_ruby/1.8/puppet/type.rb:816:in `set_default')
          
          Show
          Konstantin Boudnik added a comment - Ok, with Docker 1.3+ I can now have an FQDN in the container! Let's document that Docker before 1.2 won't work for the puppet deployment. Now, I see the following warnings with the patch applied: Warning: The use of 'import' is deprecated at /ws/bigtop-deploy/puppet/manifests/site.pp:46. See http://links.puppetlabs.com/puppet-import-deprecation (at grammar.ra:610:in `_reduce_190') .... Warning: Variable access via 'port' is deprecated. Use '@port' instead. template[/ws/bigtop-deploy/puppet/modules/solr/templates/solr]:16 (at /ws/bigtop-deploy/puppet/modules/solr/templates/solr:16:in `result') Warning: Variable access via 'port_admin' is deprecated. Use '@port_admin' instead. template[/ws/bigtop-deploy/puppet/modules/solr/templates/solr]:17 (at /ws/bigtop-deploy/puppet/modules/solr/templates/solr:17:in `result') Warning: Variable access via 'zk' is deprecated. Use '@zk' instead. template[/ws/bigtop-deploy/puppet/modules/solr/templates/solr]:18 (at /ws/bigtop-deploy/puppet/modules/solr/templates/solr:18:in `result') Warning: Variable access via 'root_url' is deprecated. Use '@root_url' instead. template[/ws/bigtop-deploy/puppet/modules/solr/templates/solr]:19 (at /ws/bigtop-deploy/puppet/modules/solr/templates/solr:19:in `result') .... Warning: The package type's allow_virtual parameter will be changing its default value from false to true in a future release. If you do not want to allow virtual packages, please explicitly set allow_virtual to false. (at /usr/lib/ruby/site_ruby/1.8/puppet/type.rb:816:in `set_default')
          Hide
          Konstantin Boudnik added a comment -

          Ok, I think it works now! Let's fix the warnings later.

          Here's my official +1 on the patch. I will commit it in the next a couple of hours, unless I hear objections from the community.

          Show
          Konstantin Boudnik added a comment - Ok, I think it works now! Let's fix the warnings later. Here's my official +1 on the patch. I will commit it in the next a couple of hours, unless I hear objections from the community.
          Hide
          Konstantin Boudnik added a comment -

          Fixed sign-off in the patch. Ready for the commit

          Show
          Konstantin Boudnik added a comment - Fixed sign-off in the patch. Ready for the commit
          Konstantin Boudnik made changes -
          Hide
          jay vyas added a comment - - edited

          commiting this now ...............done ! Thanks Evans Ye this is a great update that will make it easier for people to use bigtop in modern puppet scenarios . default for most linux distros is definetly puppet 3 iirc...

          Show
          jay vyas added a comment - - edited commiting this now ...............done ! Thanks Evans Ye this is a great update that will make it easier for people to use bigtop in modern puppet scenarios . default for most linux distros is definetly puppet 3 iirc...
          jay vyas made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Assignee Evans Ye [ evans_ye ]
          Resolution Fixed [ 1 ]
          jay vyas made changes -
          Link This issue blocks BIGTOP-1508 [ BIGTOP-1508 ]
          Konstantin Boudnik made changes -
          Link This issue incorporates BIGTOP-1509 [ BIGTOP-1509 ]
          Hide
          Konstantin Boudnik added a comment -

          Also, thanks Andrey Klochkov for the initial version of the patch! It is a BIG deal guys!

          Show
          Konstantin Boudnik added a comment - Also, thanks Andrey Klochkov for the initial version of the patch! It is a BIG deal guys!
          Hide
          jay vyas added a comment -

          ah thanks for updating the readme i guess this reviewless README policy is paying off !

          Show
          jay vyas added a comment - ah thanks for updating the readme i guess this reviewless README policy is paying off !
          Hide
          vishnu gajendran added a comment -

          I was checking the patch. I believe there are still places where variables are used assuming dynamic scoping. For example, in https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp file, I see $auth, $ha variables used in common, common-hdfs, common-yarn classes assuming that these variables are available from outer scopes. Please correct me if I am missing something here.

          Show
          vishnu gajendran added a comment - I was checking the patch. I believe there are still places where variables are used assuming dynamic scoping. For example, in https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp file, I see $auth, $ha variables used in common, common-hdfs, common-yarn classes assuming that these variables are available from outer scopes. Please correct me if I am missing something here.
          Hide
          vishnu gajendran added a comment -

          I also believe that variables like $ha, $auth should be top level scope variables (assuming that these will not change once initialized). Then, we can refer to these variables from anywhere. We should not be passing these variables as parameters to every module/class definitions.

          Show
          vishnu gajendran added a comment - I also believe that variables like $ha, $auth should be top level scope variables (assuming that these will not change once initialized). Then, we can refer to these variables from anywhere. We should not be passing these variables as parameters to every module/class definitions.
          Hide
          Konstantin Boudnik added a comment -

          I think there's a JIRA that aims at it. This one is closed as it has delivered what was expected: namely we can use Puppet 3.x now. The improvements are done elsewhere. I am pretty sure you should be able to find those tickets easily.

          Show
          Konstantin Boudnik added a comment - I think there's a JIRA that aims at it. This one is closed as it has delivered what was expected: namely we can use Puppet 3.x now. The improvements are done elsewhere. I am pretty sure you should be able to find those tickets easily.
          Konstantin Boudnik made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          vishnu gajendran added a comment -

          I did a quick search on open issues. I was not able to find one which is related to the issue I mentioned. can you please point me to the jira if you know it. Else, I can open a new jira and work on that.

          Show
          vishnu gajendran added a comment - I did a quick search on open issues. I was not able to find one which is related to the issue I mentioned. can you please point me to the jira if you know it. Else, I can open a new jira and work on that.
          Hide
          Konstantin Boudnik added a comment -

          I had BIGTOP-1508 in mind, but that seemed to got fixed already. Please open a new ticket for the fix you have in mind. Thank yoU!

          Show
          Konstantin Boudnik added a comment - I had BIGTOP-1508 in mind, but that seemed to got fixed already. Please open a new ticket for the fix you have in mind. Thank yoU!
          Konstantin Boudnik made changes -
          Fix Version/s 1.0.0 [ 12326837 ]
          Fix Version/s 0.9.0 [ 12326836 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          304d 23h 27m 1 Konstantin Boudnik 10/Jun/14 23:42
          Open Open Patch Available Patch Available
          141d 18h 20m 2 Evans Ye 30/Oct/14 16:53
          Patch Available Patch Available Resolved Resolved
          6h 58m 1 jay vyas 30/Oct/14 23:52
          Resolved Resolved Closed Closed
          97d 23m 1 Konstantin Boudnik 05/Feb/15 00:16

            People

            • Assignee:
              Evans Ye
              Reporter:
              Andrey Klochkov
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development