Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1578

Refactor vagrant-puppet provisioner to have its configurations all come from yaml

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 1.0.0
    • Component/s: deployment
    • Labels:
      None

      Description

      This is the first step for simplification and consolidation of current available two Bigtop provisioner.
      With the yaml file introduced to the provisioner, we can now have a tidy structure by removing provision.sh and do all the configuration in vagrantconfig.yaml.
      Meanwhile, the code snippet used in both vagrant-puppet and docker-puppet is now shared in bigtop-deploy/vm/utils/setup-env.sh, so that we won't be confusing where to find or add a function.

        Activity

        Hide
        evans_ye Evans Ye added a comment -

        The patch attached. Changes listed below:

        • move provision.sh code snippets into Vagrantfile
        • whether to run smoke test is now configurable in vagrantconfig.yaml
        Show
        evans_ye Evans Ye added a comment - The patch attached. Changes listed below: move provision.sh code snippets into Vagrantfile whether to run smoke test is now configurable in vagrantconfig.yaml
        Hide
        jayunit100 jay vyas added a comment -

        Evans Ye testing now.....

        Show
        jayunit100 jay vyas added a comment - Evans Ye testing now.....
        Hide
        jayunit100 jay vyas added a comment - - edited

        Evans Ye

        • i like the vagrantfile cleanup to read yaml params .... makes them directly read into the shell script
        • but, however I'm getting an error...
          - ==> bigtop1: Info: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[hadoop-yarn-proxyserver]: Unscheduling refresh on Service[hadoop-yarn-proxyserver]
          ==> bigtop1: Debug: Hadoop::Proxyserver[proxyserver]: The container Class[Hadoop_head_node] will propagate my refresh event
          ==> bigtop1: Debug: Class[Hadoop::Common-yarn]: The container Stage[main] will propagate my refresh event
          ==> bigtop1: Debug: Executing '/sbin/service hadoop-hdfs-namenode status'
          ==> bigtop1: Debug: Executing '/sbin/service hadoop-hdfs-namenode start'
          ==> bigtop1: Debug: Executing '/sbin/service hadoop-hdfs-namenode status'
          ==> bigtop1: Error: Could not start Service[hadoop-hdfs-namenode]: Execution of '/sbin/service hadoop-hdfs-namenode start' returned 1: starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-bigtop1.vagrant.out
          ==> bigtop1: Failed to start Hadoop namenode. Return value: 1[FAILED]
          ==> bigtop1: Wrapped exception:
          ==> bigtop1: Execution of '/sbin/service hadoop-hdfs-namenode start' returned 1: starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-bigtop1.vagrant.out
          ==> bigtop1: Failed to start Hadoop namenode. Return value: 1[FAILED]
          ==> bigtop1: Error: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-hdfs-namenode]/ensure: change from stopped to running failed: Could not start Service[hadoop-hdfs-namenode]: Execution of '/sbin/service hadoop-hdfs-namenode start' returned 1: starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-bigtop1.vagrant.out
          ==> bigtop1: Failed to start Hadoop namenode. Return value: 1[FAILED]
          
        Show
        jayunit100 jay vyas added a comment - - edited Evans Ye i like the vagrantfile cleanup to read yaml params .... makes them directly read into the shell script but, however I'm getting an error... - ==> bigtop1: Info: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[hadoop-yarn-proxyserver]: Unscheduling refresh on Service[hadoop-yarn-proxyserver] ==> bigtop1: Debug: Hadoop::Proxyserver[proxyserver]: The container Class[Hadoop_head_node] will propagate my refresh event ==> bigtop1: Debug: Class[Hadoop::Common-yarn]: The container Stage[main] will propagate my refresh event ==> bigtop1: Debug: Executing '/sbin/service hadoop-hdfs-namenode status' ==> bigtop1: Debug: Executing '/sbin/service hadoop-hdfs-namenode start' ==> bigtop1: Debug: Executing '/sbin/service hadoop-hdfs-namenode status' ==> bigtop1: Error: Could not start Service[hadoop-hdfs-namenode]: Execution of '/sbin/service hadoop-hdfs-namenode start' returned 1: starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-bigtop1.vagrant.out ==> bigtop1: Failed to start Hadoop namenode. Return value: 1[FAILED] ==> bigtop1: Wrapped exception: ==> bigtop1: Execution of '/sbin/service hadoop-hdfs-namenode start' returned 1: starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-bigtop1.vagrant.out ==> bigtop1: Failed to start Hadoop namenode. Return value: 1[FAILED] ==> bigtop1: Error: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-hdfs-namenode]/ensure: change from stopped to running failed: Could not start Service[hadoop-hdfs-namenode]: Execution of '/sbin/service hadoop-hdfs-namenode start' returned 1: starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-bigtop1.vagrant.out ==> bigtop1: Failed to start Hadoop namenode. Return value: 1[FAILED]
        Hide
        jayunit100 jay vyas added a comment -

        Evans Ye works ! I tested on a fresh machine. it must just be the memory is low on the other box.... sorry for the red herring.

        Show
        jayunit100 jay vyas added a comment - Evans Ye works ! I tested on a fresh machine. it must just be the memory is low on the other box.... sorry for the red herring.
        Hide
        jayunit100 jay vyas added a comment -

        +1 and commited. thanks , looking forward to cleanuip of the docker-puppet ...in that process, Evans Ye, maybe we shoudl rename docker-puppet to vagrant-puppet-docker and rename vagrant-puppet to vagrant-puppet-vm , to make it clearer.... ?

        Show
        jayunit100 jay vyas added a comment - +1 and commited. thanks , looking forward to cleanuip of the docker-puppet ...in that process, Evans Ye , maybe we shoudl rename docker-puppet to vagrant-puppet-docker and rename vagrant-puppet to vagrant-puppet-vm , to make it clearer.... ?
        Hide
        evans_ye Evans Ye added a comment -

        Sure, that sounds good.
        This will clarify the ambiguous of the naming. I'll do that on the way I cleanup the docker stuff in BIGTOP-1564.

        Show
        evans_ye Evans Ye added a comment - Sure, that sounds good. This will clarify the ambiguous of the naming. I'll do that on the way I cleanup the docker stuff in BIGTOP-1564 .
        Hide
        evans_ye Evans Ye added a comment -

        Tag to resolved since this jira already committed.

        Show
        evans_ye Evans Ye added a comment - Tag to resolved since this jira already committed.

          People

          • Assignee:
            Unassigned
            Reporter:
            evans_ye Evans Ye
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development