Bigtop
  1. Bigtop
  2. BIGTOP-685

provide a way to specify the parameters expected by a test

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.4.0
    • Fix Version/s: 0.5.0
    • Component/s: Tests
    • Labels:
      None

      Description

      Many tests take parameters. Parameters may be passed into a test via system properties or environment variables. These parameters can be regarded as part of the test's contract. Over time, with increasing number of tests, it becomes hard to keep track of the set of parameters that may be used by the tests.
      I propose to address this issue at two levels:
      1. Introduce an annotation type for annotating a test class with the list of system properties and environment variables used by it. This allows for the contract to be exposed in the javadoc, as well as to annotation processing tools.
      2. Provide utility functions for automatically obtaining the parameters and their values from the above annotation.

      Not addressed in this proposal: Injecting 2. into the test runner itself (currently we are running tests through JUnit).

      1. TestDummy2.groovy
        2 kB
        Wing Yew Poon
      2. TestDummy1b.java
        2 kB
        Wing Yew Poon
      3. TestDummy1a.java
        2 kB
        Wing Yew Poon
      4. BIGTOP-685-2.txt
        13 kB
        Wing Yew Poon
      5. contract.txt
        10 kB
        Wing Yew Poon

        Issue Links

          Activity

          Hide
          Wing Yew Poon added a comment -

          I have attached a patch with a concrete prototype. It includes a demonstration of how the annotation is used in a test. I have tested it by running this test. This addresses Bruno's concerns in BIGTOP-678.
          I have also tested using environment variables (not demonstrated) in a dummy test.
          I am open to calling Parameters Contract and to calling ParameterSetter something else.
          There is a performance cost to using introspection or reflection, of course. However, the utility functions are called once per class, and there can be many test methods in one class making use of the same set of parameters.

          Show
          Wing Yew Poon added a comment - I have attached a patch with a concrete prototype. It includes a demonstration of how the annotation is used in a test. I have tested it by running this test. This addresses Bruno's concerns in BIGTOP-678 . I have also tested using environment variables (not demonstrated) in a dummy test. I am open to calling Parameters Contract and to calling ParameterSetter something else. There is a performance cost to using introspection or reflection, of course. However, the utility functions are called once per class, and there can be many test methods in one class making use of the same set of parameters.
          Hide
          Bruno Mahé added a comment -

          +1
          Thanks!

          Show
          Bruno Mahé added a comment - +1 Thanks!
          Hide
          Wing Yew Poon added a comment -

          Reopening this to submit improvements on the original prototype.
          I uploaded a new patch.
          The changes are as follows:

          • Renamed Parameters annotation to Contract, to avoid confusion with the org.junit.runners.Parameterized.Parameters annotation.
          • Added convenience utility functions for setting parameters, using naming conventions for the static fields, so user does not have to specify them.
          • Added detailed javadocs.
          Show
          Wing Yew Poon added a comment - Reopening this to submit improvements on the original prototype. I uploaded a new patch. The changes are as follows: Renamed Parameters annotation to Contract, to avoid confusion with the org.junit.runners.Parameterized.Parameters annotation. Added convenience utility functions for setting parameters, using naming conventions for the static fields, so user does not have to specify them. Added detailed javadocs.
          Hide
          Wing Yew Poon added a comment -

          I also attached some files (that can be dropped into bigtop-tests/test-artifacts/hive/src/main/groovy/org/apache/bigtop/itest/hivesmoke/) that I used for manually testing this framework feature.

          Show
          Wing Yew Poon added a comment - I also attached some files (that can be dropped into bigtop-tests/test-artifacts/hive/src/main/groovy/org/apache/bigtop/itest/hivesmoke/) that I used for manually testing this framework feature.
          Hide
          Wing Yew Poon added a comment -

          If this is accepted, I'd be happy to open another JIRA to work on porting existing tests that use environment variables and system properties to use this contract feature.

          Show
          Wing Yew Poon added a comment - If this is accepted, I'd be happy to open another JIRA to work on porting existing tests that use environment variables and system properties to use this contract feature.
          Hide
          Roman Shaposhnik added a comment -

          This has now been committed and will be part of the Bigtop 0.5.0 release.

          Wing Yew, could you please open individual JIRAs to start converting tests for the Bigtop 0.6.0 release?

          Show
          Roman Shaposhnik added a comment - This has now been committed and will be part of the Bigtop 0.5.0 release. Wing Yew, could you please open individual JIRAs to start converting tests for the Bigtop 0.6.0 release?
          Hide
          jay vyas added a comment -

          Metadata for tests is now en vogue again . We are working bigtop https://issues.apache.org/jira/browse/BIGTOP-1222, gradlelizing the smokes to make them easier to control, and will be pulling in this meta data in some kind of user friendly way.

          Show
          jay vyas added a comment - Metadata for tests is now en vogue again . We are working bigtop https://issues.apache.org/jira/browse/BIGTOP-1222 , gradlelizing the smokes to make them easier to control, and will be pulling in this meta data in some kind of user friendly way.

            People

            • Assignee:
              Wing Yew Poon
              Reporter:
              Wing Yew Poon
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development