Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-996

Document steps needed to run BT tests on a fresh host

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.5.0
    • Fix Version/s: 0.8.0
    • Component/s: documentation
    • Labels:
      None

      Description

      I argue that a major part of becoming a contributor for Bigtop is a learning curve in how to use all the mechanics of the trade. I see this times and again.

      We need to have a HOWTO.txt document that would have a trivial description of the needed steps. Something like:

      sudo jenkins
      clone Bigtop repo
      mvn install top pom.xml
      mvn install itest
      set HADOOP_CONF_DIR, HADOOP_HOME, HADOOP_MAPRED_HOME
      mvn install test-artifacts top pom.xml
      mvn install test-execution top pom.xml
      mvn -Dorg.apache.bigtop.itest.log4j.level=TRACE -f bigtop-tests/test-execution/smokes/hadoop/pom.xml verify

      That might be sufficient.

        Activity

        Hide
        jayunit100 jay vyas added a comment -

        Good point, sorry i was broadening the JIRA unnecessarily. Resolved...

        If we just want to run tests, I think the README.md file is sufficient:

        https://github.com/apache/bigtop/blob/master/README.md

        Currently, our readme has a good explanation of (1) how to not only run tests, but (2) also how to edit them manually, and (3) how to configure and run them.

        And once BIGTOP-1222 comes around, customizing and modifying the tests will simply be done by following the comments in the build.gradle file

        Show
        jayunit100 jay vyas added a comment - Good point, sorry i was broadening the JIRA unnecessarily. Resolved... If we just want to run tests, I think the README.md file is sufficient: https://github.com/apache/bigtop/blob/master/README.md Currently, our readme has a good explanation of (1) how to not only run tests, but (2) also how to edit them manually, and (3) how to configure and run them. And once BIGTOP-1222 comes around, customizing and modifying the tests will simply be done by following the comments in the build.gradle file
        Hide
        cos Konstantin Boudnik added a comment -

        jay vyas, thanks for picking it up! perhaps we shouldn't block the effort in the jira by adding the package, because this one specifically addresses running the tests. What do you think?

        I believe most of the steps are already automated by the {[gradle installAllLocalArtifacts}} target.

        Show
        cos Konstantin Boudnik added a comment - jay vyas , thanks for picking it up! perhaps we shouldn't block the effort in the jira by adding the package, because this one specifically addresses running the tests. What do you think? I believe most of the steps are already automated by the {[gradle installAllLocalArtifacts}} target.
        Hide
        jayunit100 jay vyas added a comment -

        Okay, I've created this page:

        https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=34018936

        Which i'll flesh out as soon as possible . Sean Mackrory maybe can help on the "how to add a package tobigtop".

        The page can be a "link farm" , i guess, which just points to pertinent README files or JIRAs or whatever, that way its easy to keep up to date over time.

        Hope others will help maintain it as well.

        Once i get the first pass done, I'll close this JIRA accordingly !

        Show
        jayunit100 jay vyas added a comment - Okay, I've created this page: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=34018936 Which i'll flesh out as soon as possible . Sean Mackrory maybe can help on the "how to add a package tobigtop". The page can be a "link farm" , i guess, which just points to pertinent README files or JIRAs or whatever, that way its easy to keep up to date over time. Hope others will help maintain it as well. Once i get the first pass done, I'll close this JIRA accordingly !
        Hide
        cos Konstantin Boudnik added a comment -

        jay vyas - I was scanning through the open tickets and came across this one. Haven't it been done yet? I though you have some materials published on wiki, no? If so - please reassign the ticket to yourself and close it as you see fit.

        Show
        cos Konstantin Boudnik added a comment - jay vyas - I was scanning through the open tickets and came across this one. Haven't it been done yet? I though you have some materials published on wiki, no? If so - please reassign the ticket to yourself and close it as you see fit.
        Hide
        cos Konstantin Boudnik added a comment -

        Jay, that'd be great! I think the purpose of the JIRA was to document what environment needs to be set, etc. That has been addressed now pretty much completely by bigtop_toolchain/ module (thanks Ian Mordey.

        I'd say let's go with your proposal and build gradually from it - it's on wiki anyway.

        Show
        cos Konstantin Boudnik added a comment - Jay, that'd be great! I think the purpose of the JIRA was to document what environment needs to be set, etc. That has been addressed now pretty much completely by bigtop_toolchain/ module (thanks Ian Mordey . I'd say let's go with your proposal and build gradually from it - it's on wiki anyway.
        Hide
        jayunit100 jay vyas added a comment -

        +1 I agree, but I think running all smokes requires customization, anybody (well, most people at least..) will not run them just as they are from head since everyone's cluster will have slightly different testing needs .

        I have Something more detailed I've written up how I do it here
        http://jayunit100.blogspot.com/2013/07/customizing-apache-big-top-smoke-tests.html?m=1

        Do you think this should be ported into a wiki, and if so, maybe I can just change my blog to link to the wiki page?

        If so that would I think satisfy this jira? Then I could link my blog to it so that I don't have to maintain instructions if they change

        Show
        jayunit100 jay vyas added a comment - +1 I agree, but I think running all smokes requires customization, anybody (well, most people at least..) will not run them just as they are from head since everyone's cluster will have slightly different testing needs . I have Something more detailed I've written up how I do it here http://jayunit100.blogspot.com/2013/07/customizing-apache-big-top-smoke-tests.html?m=1 Do you think this should be ported into a wiki, and if so, maybe I can just change my blog to link to the wiki page? If so that would I think satisfy this jira? Then I could link my blog to it so that I don't have to maintain instructions if they change
        Hide
        rvs Roman Shaposhnik added a comment -

        Any help in that respect would be very much appreciated. On that note – we're trying to restart our Bigtop meetups/hackathons this summer (http://www.meetup.com/HandsOnProgrammingEvents/events/122050632/) perhaps we can find folks interested in helping us with our documentation, etc.

        Show
        rvs Roman Shaposhnik added a comment - Any help in that respect would be very much appreciated. On that note – we're trying to restart our Bigtop meetups/hackathons this summer ( http://www.meetup.com/HandsOnProgrammingEvents/events/122050632/ ) perhaps we can find folks interested in helping us with our documentation, etc.

          People

          • Assignee:
            jayunit100 jay vyas
            Reporter:
            cos Konstantin Boudnik
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development