Bigtop
  1. Bigtop
  2. BIGTOP-313

Move puppet configuration out of manifest and into extlookup() CSV

    Details

      Description

      Previously, bigtop-deploy puppet could only be configured by editing puppet manifests, or through a small number of environment variables that set facter facts. I've rewritten cluster.pp to use extlookup to read most configuration options from a site-specific CSV file with sane defaults for most options (only hadoop_head_node and bigtop_yumrepo_uri are required to be set). These changes should make it easier to automate deploying multiple clusters with different configurations from the same manifests.

      As written, the CSV files must live in $confdir/config/, where $confdir is the puppet config directory. This means that if you are manually calling 'puppet agent' on these manifests (rather than serving them from a puppet master), you are now required to specify --confdir in addition to --modulepath.

      At some point in the future, may be worth moving to hiera, as yaml is a lot prettier and more flexible than CSV. I did this using extlookup so as not to introduce external dependencies (since extlookup is now in puppet core).

      1. patch.txt
        17 kB
        Patrick Taylor Ramsey

        Activity

        Patrick Taylor Ramsey created issue -
        Patrick Taylor Ramsey made changes -
        Field Original Value New Value
        Description Previously, bigtop-deploy puppet could only be configured by editing puppet manifests, or through a small number of environment variables that set facter facts. I've rewritten cluster.pp to use extlookup to read configuration from a site-specific CSV file with sane defaults for most options (only hadoop_head_node and bigtop_yumrepo_uri are required to be set). These changes should make it easier to automate deploying multiple clusters with different configurations from the same manifests.

        As written, the CSV files must live in $confdir/config/, where $confdir is the puppet config directory. This means that if you are manually calling 'puppet agent' on these manifests (rather than serving them from a puppet master), you are now required to specify --confdir in addition to --modulepath.

        At some point in the future, may be worth moving to hiera, as yaml is a lot prettier and more flexible than CSV. I did this using extlookup so as not to introduce external dependencies (since extlookup is now in puppet core).
        Previously, bigtop-deploy puppet could only be configured by editing puppet manifests, or through a small number of environment variables that set facter facts. I've rewritten cluster.pp to use extlookup to read most configuration options from a site-specific CSV file with sane defaults for most options (only hadoop_head_node and bigtop_yumrepo_uri are required to be set). These changes should make it easier to automate deploying multiple clusters with different configurations from the same manifests.

        As written, the CSV files must live in $confdir/config/, where $confdir is the puppet config directory. This means that if you are manually calling 'puppet agent' on these manifests (rather than serving them from a puppet master), you are now required to specify --confdir in addition to --modulepath.

        At some point in the future, may be worth moving to hiera, as yaml is a lot prettier and more flexible than CSV. I did this using extlookup so as not to introduce external dependencies (since extlookup is now in puppet core).
        Patrick Taylor Ramsey made changes -
        Attachment patch.txt [ 12508012 ]
        Patrick Taylor Ramsey made changes -
        Attachment patch.txt [ 12508012 ]
        Patrick Taylor Ramsey made changes -
        Attachment 0001-BIGTOP-313-Move-puppet-configuration-into-CSV.patch [ 12509755 ]
        Patrick Taylor Ramsey made changes -
        Attachment patch.txt [ 12509760 ]
        Patrick Taylor Ramsey made changes -
        Attachment 0001-BIGTOP-313-Move-puppet-configuration-into-CSV.patch [ 12509755 ]
        Roman Shaposhnik made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Roman Shaposhnik made changes -
        Assignee Roman Shaposhnik [ rvs ]
        Roman Shaposhnik made changes -
        Fix Version/s 0.3.0 [ 12317841 ]
        Fix Version/s 0.4.0 [ 12318889 ]
        Gavin made changes -
        Workflow no-reopen-closed, patch-avail [ 12646486 ] patch-available, re-open possible [ 12665718 ]
        Roman Shaposhnik made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Roman Shaposhnik
            Reporter:
            Patrick Taylor Ramsey
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development