Bigtop
  1. Bigtop
  2. BIGTOP-798

introduce a fatjar collection of all the Bigtop integration tests and all their dependencies

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.5.0
    • Fix Version/s: 0.5.0
    • Component/s: Tests
    • Labels:
      None

      Description

      This is the first baby step towards ditching our antiquated Maven-based execution framework and starting using something sane: we need a self-contained artifact that can used as simply as:

      java -cp fat.jar org.junit.runner.JUnitCore org.apache.bigtop.itest.crunch.smoke.TestCrunchSmoke
      

        Activity

        Hide
        Roman Shaposhnik added a comment - - edited

        So at this point the following cool thing is now possible:

        $ java -cp ./fat-smoke-0.5.0-SNAPSHOT.jar                             \
          org.codehaus.groovy.tools.GroovyStarter --main groovy.ui.GroovyMain \
          -e "new AntBuilder().junit { 
                test(name:'org.apache.bigtop.itest.crunch.smoke.TestCrunchSmoke')
                formatter(type:'xml') 
             }"
        
        Show
        Roman Shaposhnik added a comment - - edited So at this point the following cool thing is now possible: $ java -cp ./fat-smoke-0.5.0-SNAPSHOT.jar \ org.codehaus.groovy.tools.GroovyStarter --main groovy.ui.GroovyMain \ -e "new AntBuilder().junit { test(name:'org.apache.bigtop.itest.crunch.smoke.TestCrunchSmoke') formatter(type:'xml') }"
        Hide
        Andrew Bayer added a comment -

        +1

        Show
        Andrew Bayer added a comment - +1
        Hide
        Wing Yew Poon added a comment -

        It appears that this fat jar contains the compile-time dependencies of the test artifacts.
        I'm not sure how useful this is.
        The run-time dependencies of tests may be different from the compile-time dependencies. Also, you may need other stuff in your classpath, such as HADOOP_CONF_DIR or HBASE_CONF_DIR.
        Aside from this, there may well be a whole host of parameters you need to pass in to your test as system properties.
        The current maven way does not take care of the last, but it takes care of the first two.

        Show
        Wing Yew Poon added a comment - It appears that this fat jar contains the compile-time dependencies of the test artifacts. I'm not sure how useful this is. The run-time dependencies of tests may be different from the compile-time dependencies. Also, you may need other stuff in your classpath, such as HADOOP_CONF_DIR or HBASE_CONF_DIR. Aside from this, there may well be a whole host of parameters you need to pass in to your test as system properties. The current maven way does not take care of the last, but it takes care of the first two.
        Hide
        Roman Shaposhnik added a comment -

        It appears that this fat jar contains the compile-time dependencies of the test artifacts.

        It actually contains all the dependencies. In Maven parlance compile gets included in the runtime as well.

        The run-time dependencies of tests may be different from the compile-time dependencies.

        Are you talking about dependencies that are somehow not expressed in the tests-artifact's pom files? I believe it would be extremely useful if we can also manage them via poms. Can you please give me a few examples?

        Aside from this, there may well be a whole host of parameters you need to pass in to your test as system properties.

        Yes. And that will be addressed in future JIRAs.

        Now, of course if the current set of test-execution pom files is still useful to some in the Bigtop community there's no need to remove that code. My intent here is to be able to have a single artifact with a really simple CLI interface that I can use to run tests. My ideal state is something that would let me do this:

        $ java -jar fat.jar 
        Usage: ....
        $ java -jar fat.jar -list
        // lists all the tests and suites
        $ java -jar fat.jar -info Test
        // prints required fixtures
        $ java -jar fat.jar -run Test
        // runs the tests
        
        Show
        Roman Shaposhnik added a comment - It appears that this fat jar contains the compile-time dependencies of the test artifacts. It actually contains all the dependencies. In Maven parlance compile gets included in the runtime as well. The run-time dependencies of tests may be different from the compile-time dependencies. Are you talking about dependencies that are somehow not expressed in the tests-artifact's pom files? I believe it would be extremely useful if we can also manage them via poms. Can you please give me a few examples? Aside from this, there may well be a whole host of parameters you need to pass in to your test as system properties. Yes. And that will be addressed in future JIRAs. Now, of course if the current set of test-execution pom files is still useful to some in the Bigtop community there's no need to remove that code. My intent here is to be able to have a single artifact with a really simple CLI interface that I can use to run tests. My ideal state is something that would let me do this: $ java -jar fat.jar Usage: .... $ java -jar fat.jar -list // lists all the tests and suites $ java -jar fat.jar -info Test // prints required fixtures $ java -jar fat.jar -run Test // runs the tests
        Hide
        Wing Yew Poon added a comment -

        I understand that the fatjar pulls in dependencies transitively, but that is still build-time dependencies.
        And yes, I do know of at least one example where the run-time dependencies are different from the build-time dependencies. TestJdbcDriver in hive has hardly any build-time dependencies; it only needs the java.sql.* classes to build. However, in order to run the test, you need a whole bunch of jars from hive/lib/. They are expressed in the execution pom.xml.

        Show
        Wing Yew Poon added a comment - I understand that the fatjar pulls in dependencies transitively, but that is still build-time dependencies. And yes, I do know of at least one example where the run-time dependencies are different from the build-time dependencies. TestJdbcDriver in hive has hardly any build-time dependencies; it only needs the java.sql.* classes to build. However, in order to run the test, you need a whole bunch of jars from hive/lib/. They are expressed in the execution pom.xml.

          People

          • Assignee:
            Roman Shaposhnik
            Reporter:
            Roman Shaposhnik
          • Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development