Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Duplicate
    • Affects Version/s: 0.7.0
    • Fix Version/s: 0.8.0
    • Component/s: Tests
    • Labels:
      None

      Description

      Running all the smokes with mvn verify is painful:

      1) Diving into the XML files when stuff breaks

      2) Remembering to use TRACE level logging

      3) Leaving out export PIG_HOME , etc... leads to smokes failing late in the test run.

      4) Hopefully we can specify more env variables to control size of smokes in the future. That will make a script for running the whole thing even more useful.

      5) when hacking the groovy scripts:: To Compile or not to compile? Always forget to specify.

      A simple bash script could serve as a better tutorial/solution/template for users that want to run smokes. I'm attaching a very raw patch that we can rewrite in python and flesh out some more to do better error reporting, more verification, etc.

      1. BIGTOP-1195.1.very-raw.patch
        4 kB
        jay vyas
      2. BIGTOP-1195.patch
        4 kB
        jay vyas

        Issue Links

          Activity

          jay vyas created issue -
          Hide
          jay vyas added a comment -

          A first pass, just to illustrate the general idea of what we should have ... im open to other ideas (python, another groovy/maven layer, or just cleaner bash).

          Show
          jay vyas added a comment - A first pass, just to illustrate the general idea of what we should have ... im open to other ideas (python, another groovy/maven layer, or just cleaner bash).
          jay vyas made changes -
          Field Original Value New Value
          Attachment BIGTOP-1195.1.very-raw.patch [ 12625477 ]
          Hide
          Mikhail Antonov added a comment -

          A question/thought. What about separation between

          • smoke tests as "validating the binaries built from the source code downloaded from apache.org", e.g. that the code is not broken
            and
          • smoke tests to validate that a particular deployment is of packages, deployed thru Puppet, is not broken?

          and also separation between binaries built as jar files vs. rpms?

          For latter case we obviously need to have access to some provisioning API, to run more or less complex tests, for example for HA, for former we need probably to simplify test setup as much as possible, including testing all what's possible on local machine,

          Show
          Mikhail Antonov added a comment - A question/thought. What about separation between smoke tests as "validating the binaries built from the source code downloaded from apache.org", e.g. that the code is not broken and smoke tests to validate that a particular deployment is of packages, deployed thru Puppet, is not broken? and also separation between binaries built as jar files vs. rpms? For latter case we obviously need to have access to some provisioning API, to run more or less complex tests, for example for HA, for former we need probably to simplify test setup as much as possible, including testing all what's possible on local machine,
          Hide
          Konstantin Boudnik added a comment -

          I don't think we can do such separation. If you think about this - the only way to test say Hbase binaries downloaded and built from apache.org is to setup HDFS cluster, set up Hbase and ZK and the run some loads. Considering that we do not deploy anything else but linux packages, the smoking of the binaries is an integral part of smoking of the packages. I am not sure if it is feasible to test binaries without deployment. Please provide me with more details if I am missing something.

          In the interest of full disclosure though, there's a way to test packages functionality separately and that's what the package manager tests are for.

          Show
          Konstantin Boudnik added a comment - I don't think we can do such separation. If you think about this - the only way to test say Hbase binaries downloaded and built from apache.org is to setup HDFS cluster, set up Hbase and ZK and the run some loads. Considering that we do not deploy anything else but linux packages, the smoking of the binaries is an integral part of smoking of the packages. I am not sure if it is feasible to test binaries without deployment. Please provide me with more details if I am missing something. In the interest of full disclosure though, there's a way to test packages functionality separately and that's what the package manager tests are for.
          Hide
          Mikhail Antonov added a comment -

          I was pursuing test simplification as "can we run smokes without having cluster", i.e. on local pseude-distributed cluster. It seems we can, at least I'm trying to do that now on my desktop. But may be it's just not something most of the people really need...

          Show
          Mikhail Antonov added a comment - I was pursuing test simplification as "can we run smokes without having cluster", i.e. on local pseude-distributed cluster. It seems we can, at least I'm trying to do that now on my desktop. But may be it's just not something most of the people really need...
          Hide
          Konstantin Boudnik added a comment -

          So, how you can deploy a pseudo distributed cluster without building the packages and using Puppet?

          Show
          Konstantin Boudnik added a comment - So, how you can deploy a pseudo distributed cluster without building the packages and using Puppet?
          Hide
          jay vyas added a comment -

          Okay, finally got some time to cleanup my test runners into a "pure" bigtop incantation.

          This script will work to compile and run all the smoke tests on current bigtop , ive just tested it in VMs, and it should work right out of the box.

          Show
          jay vyas added a comment - Okay, finally got some time to cleanup my test runners into a "pure" bigtop incantation. This script will work to compile and run all the smoke tests on current bigtop , ive just tested it in VMs, and it should work right out of the box.
          jay vyas made changes -
          Attachment BIGTOP-1195.patch [ 12630526 ]
          Hide
          jay vyas added a comment -

          TL;DR,

          This JIRA has become quiet diverted from the original purpose in the email thread here:http://mail-archives.apache.org/mod_mbox/bigtop-user/201401.mbox/%3CCAAu13zFcQppjEoS6tAf2EHwnaE_hk6itmwaEESKpM997SdMTdQ@mail.gmail.com%3E

          I'd like to get this patch in as a script we all maintain for making running bigtop smokes easy. It does the following:

          • Declares all the enviornment varaibles, including maven/java ones
          • compiles the tests
          • runs the tests
          • reports an error message
          • prints Exceptions out directly, if any occur.

          I've just tested it on a pseudo distributed cluster created via vagrant-puppet modules and it works right out of the box in the current bigtop deployment

          [root@vagrant bigtop]# ./runtests.sh 
          running tests as user : root
          note you can use  to compile (if first run, or hacking). 
          Setting hadoop_home, but shouldnt need it for hadoop 2x...
          rmr: DEPRECATED: Please use 'rm -r' instead.
          rmr: `/examples': No such file or directory
          .........................
          Now checking env vars....
          .........................
          rmr: DEPRECATED: Please use 'rm -r' instead.
          rmr: `/-p*': No such file or directory
          found PIG_HOME
          found MAHOUT_HOME
          found JAVA_HOME
          found HADOOP_MAPRED_HOME
          found HADOOP_CONF_DIR
          found HIVE_HOME
          .........
          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          
          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Running org.apache.bigtop.itest.hadoop.hdfs.TestDistCpIntra
          14/02/23 02:58:51 TRACE shell.Shell: /bin/bash -s << __EOT__
          date
          __EOT__
          ....
          

          And so on , so you get the idea. Easy to run smokes.

          Once this goes we can iterate on this driver it with more logic like :

          • options to compile the smokes or not.
          • support for Cos' new gradle based perf tests if they are ready to run.
          Show
          jay vyas added a comment - TL;DR, This JIRA has become quiet diverted from the original purpose in the email thread here: http://mail-archives.apache.org/mod_mbox/bigtop-user/201401.mbox/%3CCAAu13zFcQppjEoS6tAf2EHwnaE_hk6itmwaEESKpM997SdMTdQ@mail.gmail.com%3E I'd like to get this patch in as a script we all maintain for making running bigtop smokes easy. It does the following: Declares all the enviornment varaibles, including maven/java ones compiles the tests runs the tests reports an error message prints Exceptions out directly, if any occur. I've just tested it on a pseudo distributed cluster created via vagrant-puppet modules and it works right out of the box in the current bigtop deployment [root@vagrant bigtop]# ./runtests.sh running tests as user : root note you can use to compile (if first run, or hacking). Setting hadoop_home, but shouldnt need it for hadoop 2x... rmr: DEPRECATED: Please use 'rm -r' instead. rmr: `/examples': No such file or directory ......................... Now checking env vars.... ......................... rmr: DEPRECATED: Please use 'rm -r' instead. rmr: `/-p*': No such file or directory found PIG_HOME found MAHOUT_HOME found JAVA_HOME found HADOOP_MAPRED_HOME found HADOOP_CONF_DIR found HIVE_HOME ......... ------------------------------------------------------- T E S T S ------------------------------------------------------- ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.bigtop.itest.hadoop.hdfs.TestDistCpIntra 14/02/23 02:58:51 TRACE shell.Shell: /bin/bash -s << __EOT__ date __EOT__ .... And so on , so you get the idea. Easy to run smokes. Once this goes we can iterate on this driver it with more logic like : options to compile the smokes or not. support for Cos' new gradle based perf tests if they are ready to run.
          Hide
          Konstantin Boudnik added a comment -

          I am not particularly against shell scripting but there's a Gradle infra, that is being introduced as a focal point for build/test control, e.g. build packages, install artifacts, run them, etc. Gradle has a great integration into Maven ecosystem and the rest of the JVM world. Do you think it would be better to move the script's functionality to a Gradle task?

          Show
          Konstantin Boudnik added a comment - I am not particularly against shell scripting but there's a Gradle infra, that is being introduced as a focal point for build/test control, e.g. build packages, install artifacts, run them, etc. Gradle has a great integration into Maven ecosystem and the rest of the JVM world. Do you think it would be better to move the script's functionality to a Gradle task?
          Hide
          jay vyas added a comment -

          Ah yes gradle is the future of bigtop: but Most vendors don't come with gradle installed, this allows any hadoop vendor to run bigtop tests directly out of the box...

          Forcing gradle as a requirement to this wrapper might defeat the purpose. I would rather see wrapper stay in bash until gradle is a requirement for smokes, at which point, the code in the script could change – from mvn calls to gradle calls .

          Show
          jay vyas added a comment - Ah yes gradle is the future of bigtop: but Most vendors don't come with gradle installed, this allows any hadoop vendor to run bigtop tests directly out of the box... Forcing gradle as a requirement to this wrapper might defeat the purpose. I would rather see wrapper stay in bash until gradle is a requirement for smokes, at which point, the code in the script could change – from mvn calls to gradle calls .
          Hide
          Konstantin Boudnik added a comment -

          Right, I see what you're saying. I am sorta -0 on that. My only concern is providing an entry point that once committed - we'll have to support going forward.

          What other guys are thinking about adding a script runner?

          Show
          Konstantin Boudnik added a comment - Right, I see what you're saying. I am sorta -0 on that. My only concern is providing an entry point that once committed - we'll have to support going forward. What other guys are thinking about adding a script runner?
          Konstantin Boudnik made changes -
          Affects Version/s 0.7.0 [ 12324362 ]
          Konstantin Boudnik made changes -
          Fix Version/s 0.8.0 [ 12324841 ]
          Konstantin Boudnik made changes -
          Assignee jay vyas [ jayunit100 ]
          Konstantin Boudnik made changes -
          Component/s Tests [ 12315617 ]
          Konstantin Boudnik made changes -
          Priority Major [ 3 ] Minor [ 4 ]
          Hide
          jay vyas added a comment -

          id like to take this moment to shamelssly request the biased opinions of Roman Shaposhnik and Martin Bukatovic should weigh in , they were on the original email thread regarding this.

          Show
          jay vyas added a comment - id like to take this moment to shamelssly request the biased opinions of Roman Shaposhnik and Martin Bukatovic should weigh in , they were on the original email thread regarding this.
          Konstantin Boudnik made changes -
          Link This issue relates to BIGTOP-1222 [ BIGTOP-1222 ]
          Hide
          Martin Bukatovic added a comment -

          My feeling is that the most important thing here is to clearly document all the assumptions (env variables, user, configuration) for running the bigtop smokes.
          So far, we have this wikipage: https://cwiki.apache.org/confluence/display/BIGTOP/Running+integration+and+system+tests
          I for example miss here that you need to create new user account and setup sudoers file for bigtop tests to work (as running under root is not a good idea here).

          That said, having an runner script would be great addition to that. I see the reasons why it would be difficult to include it as a proper way to run tests, but I would be ok with declaring the script as an example how one can run the tests.

          Show
          Martin Bukatovic added a comment - My feeling is that the most important thing here is to clearly document all the assumptions (env variables, user, configuration) for running the bigtop smokes. So far, we have this wikipage: https://cwiki.apache.org/confluence/display/BIGTOP/Running+integration+and+system+tests I for example miss here that you need to create new user account and setup sudoers file for bigtop tests to work (as running under root is not a good idea here). That said, having an runner script would be great addition to that. I see the reasons why it would be difficult to include it as a proper way to run tests, but I would be ok with declaring the script as an example how one can run the tests.
          Hide
          jun aoki added a comment -

          but Most vendors don't come with gradle installed, this allows any hadoop vendor to run bigtop tests directly out of the box...

          FYI: Gradle Wrapper is a good way to start gradle build wihtout gradle.
          http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.wrapper.Wrapper.html

          Show
          jun aoki added a comment - but Most vendors don't come with gradle installed, this allows any hadoop vendor to run bigtop tests directly out of the box... FYI: Gradle Wrapper is a good way to start gradle build wihtout gradle. http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.wrapper.Wrapper.html
          Hide
          Konstantin Boudnik added a comment -

          Jun, we actually already have Gradle build in place - I admit it is very simplistic, yet functional

          Show
          Konstantin Boudnik added a comment - Jun, we actually already have Gradle build in place - I admit it is very simplistic, yet functional
          Hide
          Konstantin Boudnik added a comment - - edited

          jay vyas, I just have left a comment wrapping up our earlier call. I think what we are proposing also would address these Martin's concerns

          Show
          Konstantin Boudnik added a comment - - edited jay vyas , I just have left a comment wrapping up our earlier call. I think what we are proposing also would address these Martin's concerns
          Hide
          jay vyas added a comment -

          OKay, ill close this, and we'll carry on the work in BIGTOP-1222. Thanks for all the feedback folks.

          Show
          jay vyas added a comment - OKay, ill close this, and we'll carry on the work in BIGTOP-1222 . Thanks for all the feedback folks.
          Hide
          jay vyas added a comment -

          work will carry on in BIGTOP-1222

          Show
          jay vyas added a comment - work will carry on in BIGTOP-1222
          jay vyas made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Duplicate [ 3 ]

            People

            • Assignee:
              jay vyas
              Reporter:
              jay vyas
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development