Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1555

remove mapred-app from docs or code snippet (post-BIGTOP-1522), cleanup vagrant-puppet to core components.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 1.0.0
    • Component/s: deployment
    • Labels:
      None

      Description

      In BIGTOP-1522, yarn absorb mapred-app since yarn and mapreduce are Indivisible. As a result some code snippet and readme in vagrant or docker provisioner might need to be update in order to reflect this change.

      1. BIGTOP-1555.1.patch
        4 kB
        Evans Ye
      2. BIGTOP-1555.2.patch
        5 kB
        Evans Ye

        Issue Links

          Activity

          Hide
          evans_ye Evans Ye added a comment -

          A simple patch attached.
          I've ran through the test on both docker and vagrant provisioner.
          It should be no problem

          Show
          evans_ye Evans Ye added a comment - A simple patch attached. I've ran through the test on both docker and vagrant provisioner. It should be no problem
          Hide
          jayunit100 jay vyas added a comment -

          +1 evans, but , can we hold off for a min to commit,
          BIGTOP-1562, BIGTOP-1553 are both also puppet patches, im thinking some rebase will be required.
          we'll circle back around to this, say monday night?

          Show
          jayunit100 jay vyas added a comment - +1 evans, but , can we hold off for a min to commit, BIGTOP-1562 , BIGTOP-1553 are both also puppet patches, im thinking some rebase will be required. we'll circle back around to this, say monday night?
          Hide
          evans_ye Evans Ye added a comment -

          Sure, this is a minor one but just to keep the code looks more clarity.

          Show
          evans_ye Evans Ye added a comment - Sure, this is a minor one but just to keep the code looks more clarity.
          Hide
          jayunit100 jay vyas added a comment -

          Evans Ye Can you rebase this patch, now that the puppet upgrade is complete, and give it a whirl ? thanks!

          Show
          jayunit100 jay vyas added a comment - Evans Ye Can you rebase this patch, now that the puppet upgrade is complete, and give it a whirl ? thanks!
          Hide
          jayunit100 jay vyas added a comment - - edited

          Hi evans.

          As part of this jira, since we are fixing the vagrantconfig components list, and README docs.

          • update README to describe that we now support installing components from bigtop-home/output
          • update vagrantconfig.yaml to remove tachyon, because we don't need it. I think for the default vagrantconfig.yaml, we should stabilize on a minimal set of components to test that are in the current bigtop release version.
          • make sure vagrantconfig.yaml components are configuraable (yarn, pig, ... ) from commandline, and default them to just yarn . i.e. if its easy, lets add an argument vagrant up --components=tachyon,pig,yarn will override the components line, or maybe just a snippet in the readme about how to do it export COMPONENTS=yarn,pig && vagrant up.
          Show
          jayunit100 jay vyas added a comment - - edited Hi evans. As part of this jira, since we are fixing the vagrantconfig components list, and README docs. update README to describe that we now support installing components from bigtop-home/output update vagrantconfig.yaml to remove tachyon, because we don't need it. I think for the default vagrantconfig.yaml, we should stabilize on a minimal set of components to test that are in the current bigtop release version. make sure vagrantconfig.yaml components are configuraable (yarn, pig, ... ) from commandline, and default them to just yarn . i.e. if its easy, lets add an argument vagrant up --components=tachyon,pig,yarn will override the components line, or maybe just a snippet in the readme about how to do it export COMPONENTS=yarn,pig && vagrant up .
          Hide
          evans_ye Evans Ye added a comment -

          jay vyas as my point of view, I think it would be better to keep all the configurations in vagrantconfig.yaml.
          However, things like export COMPONENTS=yarn,pig && vagrant up can be doable.
          Is there a reason why you'd like to use it in this way?

          Show
          evans_ye Evans Ye added a comment - jay vyas as my point of view, I think it would be better to keep all the configurations in vagrantconfig.yaml . However, things like export COMPONENTS=yarn,pig && vagrant up can be doable. Is there a reason why you'd like to use it in this way?
          Hide
          jayunit100 jay vyas added a comment -

          Agree We can ignore that part for this jira.

          Show
          jayunit100 jay vyas added a comment - Agree We can ignore that part for this jira.
          Hide
          evans_ye Evans Ye added a comment -

          Attached the patch including the following:

          • remove mapred-app in all the README and default components for both provisioner
          • update README to describe how to test local built packages via vagrant-puppet provisioner
          • make local yum repo configurable and default to false
          Show
          evans_ye Evans Ye added a comment - Attached the patch including the following: remove mapred-app in all the README and default components for both provisioner update README to describe how to test local built packages via vagrant-puppet provisioner make local yum repo configurable and default to false
          Hide
          jayunit100 jay vyas added a comment -

          hiya evans. reviewing this now

          Show
          jayunit100 jay vyas added a comment - hiya evans. reviewing this now
          Hide
          jayunit100 jay vyas added a comment -

          io think this looks good. It appears you also removed tachyon and made it so that the
          vagrant-puppet stuff acts the normal way which it should.

          Show
          jayunit100 jay vyas added a comment - io think this looks good. It appears you also removed tachyon and made it so that the vagrant-puppet stuff acts the normal way which it should.
          Hide
          jayunit100 jay vyas added a comment -

          +1 and i amended the commit message before commit to match the renamed JIRA,
          otherwise it won't be clear to others exactly what the patch is doing

          Thanks evans ! commiting shortly

          Show
          jayunit100 jay vyas added a comment - +1 and i amended the commit message before commit to match the renamed JIRA, otherwise it won't be clear to others exactly what the patch is doing Thanks evans ! commiting shortly
          Hide
          jayunit100 jay vyas added a comment -

          and commited . back to clean and clear vagrant setups. we can add tachyon back in once 0.9 release comes out and i complete BIGTOP-1572.

          Show
          jayunit100 jay vyas added a comment - and commited . back to clean and clear vagrant setups. we can add tachyon back in once 0.9 release comes out and i complete BIGTOP-1572 .

            People

            • Assignee:
              evans_ye Evans Ye
              Reporter:
              evans_ye Evans Ye
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development