Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2014

[VM provisioner] Missing FQDN on Ubuntu causes puppet deployment malfunction

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0
    • Component/s: vm
    • Labels:
      None

      Description

      We implemented and tested Bigtop VM provisioner mainly on CentOS 6 and Debian. However, noticed by Tom Zeng's user list question, I found that the puppet deployment on Ubuntu fail to identify head_node due to the missing FQDN setting. This is because different Linux distribution has different mechanism for FQDN setting.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user evans-ye opened a pull request:

          https://github.com/apache/bigtop/pull/39

          BIGTOP-2014. [VM provisioner] Missing FQDN on Ubuntu causes puppet de…

          …ployment malfunction

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/evans-ye/bigtop BIGTOP-2014

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/bigtop/pull/39.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #39


          commit 1a68239034fc49ccf1d30f44929509d97cefb1c6
          Author: Evans Ye <evansye@apache.org>
          Date: 2015-09-05T14:40:12Z

          BIGTOP-2014. [VM provisioner] Missing FQDN on Ubuntu causes puppet deployment malfunction


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user evans-ye opened a pull request: https://github.com/apache/bigtop/pull/39 BIGTOP-2014 . [VM provisioner] Missing FQDN on Ubuntu causes puppet de… …ployment malfunction You can merge this pull request into a Git repository by running: $ git pull https://github.com/evans-ye/bigtop BIGTOP-2014 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/bigtop/pull/39.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #39 commit 1a68239034fc49ccf1d30f44929509d97cefb1c6 Author: Evans Ye <evansye@apache.org> Date: 2015-09-05T14:40:12Z BIGTOP-2014 . [VM provisioner] Missing FQDN on Ubuntu causes puppet deployment malfunction
          Hide
          evans_ye Evans Ye added a comment - - edited

          In previous implementation, I empty the /etc/hosts file first and then explicitly runs bigtop.vm.provision :hostmanager to add IP to FQDN mapping back. However, on Ubuntu it only gets hostname(bigtop1) when running hostname -f instead of FQDN(bigtop1.vagrant). This is because there's no entry in /etc/hosts that maps to FQDN and hostname.
          In this patch, 1) instead of empty the whole file, just remove 127.0.0.1 to FQDN mapping entry is enough. 2) Adding IP to FQDN and hostname mapping allows system like Ubuntu to get FQDN correctly.

          Show
          evans_ye Evans Ye added a comment - - edited In previous implementation, I empty the /etc/hosts file first and then explicitly runs bigtop.vm.provision :hostmanager to add IP to FQDN mapping back. However, on Ubuntu it only gets hostname(bigtop1) when running hostname -f instead of FQDN(bigtop1.vagrant). This is because there's no entry in /etc/hosts that maps to FQDN and hostname. In this patch, 1) instead of empty the whole file, just remove 127.0.0.1 to FQDN mapping entry is enough. 2) Adding IP to FQDN and hostname mapping allows system like Ubuntu to get FQDN correctly.
          Hide
          evans_ye Evans Ye added a comment -

          The following configuration works well to deploy an Ubuntu VM:

          memory_size: 4096
          number_cpus: 1
          box: "ubuntu/trusty64"
          repo: "http://bigtop.s3.amazonaws.com/releases/1.0.0/ubuntu/trusty/x86_64"
          num_instances: 1
          distro: debian
          components: [hadoop, yarn]
          enable_local_repo: false
          run_smoke_tests: false
          smoke_test_components: [mapreduce, pig]
          jdk: "openjdk-7-jdk"
          

          I saw BIGTOP-2015 so let's discuss there.

          Show
          evans_ye Evans Ye added a comment - The following configuration works well to deploy an Ubuntu VM: memory_size: 4096 number_cpus: 1 box: "ubuntu/trusty64" repo: "http: //bigtop.s3.amazonaws.com/releases/1.0.0/ubuntu/trusty/x86_64" num_instances: 1 distro: debian components: [hadoop, yarn] enable_local_repo: false run_smoke_tests: false smoke_test_components: [mapreduce, pig] jdk: "openjdk-7-jdk" I saw BIGTOP-2015 so let's discuss there.
          Hide
          evans_ye Evans Ye added a comment -

          The patch is still valid. jay vyas would you mind to give it a review?
          The setting I posted above can give you a Hadoop environment on ubuntu.
          Thanks!

          Show
          evans_ye Evans Ye added a comment - The patch is still valid. jay vyas would you mind to give it a review? The setting I posted above can give you a Hadoop environment on ubuntu. Thanks!
          Hide
          jayunit100 jay vyas added a comment -

          +1 based on the intent of the patch.
          Please confirm that it is still working properly still for centos before committing

          Show
          jayunit100 jay vyas added a comment - +1 based on the intent of the patch. Please confirm that it is still working properly still for centos before committing
          Hide
          evans_ye Evans Ye added a comment -

          Sure. I just tested on CentOS and it works. Thanks for your review jay vyas!
          Committed and pushed to master.

          Show
          evans_ye Evans Ye added a comment - Sure. I just tested on CentOS and it works. Thanks for your review jay vyas ! Committed and pushed to master.

            People

            • Assignee:
              evans_ye Evans Ye
              Reporter:
              evans_ye Evans Ye
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development