Tuscany
  1. Tuscany
  2. TUSCANY-1817

Improve SDO test infrastructure to re-use/re-execute most dynamic tests as static tests

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: Java-SDO-Next
    • Fix Version/s: Java-SDO-Next
    • Component/s: Java SDO Tools
    • Labels:
      None

      Description

      Currently, static Tuscany SDO is being inadequately tested. Based on TUSCANY-1812, I suspect there are currently numerous dynamic test cases that would fail if they were executed as static test cases. I suggest the testing infrastructure be enhanced to allow tests to be re-used as both dynamic and static tests with minimal effort. The build should automatically execute these "shared" tests in both dynamic and static contexts.

        Activity

        Ron Gavlin created issue -
        Hide
        Kelvin Goodson added a comment -

        Ron, this sounds like a good idea. Do you have specific design thoughts or plans for a contribution for this?

        Show
        Kelvin Goodson added a comment - Ron, this sounds like a good idea. Do you have specific design thoughts or plans for a contribution for this?
        Hide
        Ron Gavlin added a comment -

        Hi Kelvin,

        I opened this issue to raise awareness of the problem. I have some initial design ideas but nothing real concrete at this point. A few approaches I have thought about are:

        1. Brute force approach - have existing impl tests extract their HelperContext initialization into a template method. Then have corresponding tools-test classes extend the impl tests and override the template method with static-factory-based initialization.

        2. More Elegant approach - Record the schemas used by each impl test somewhere. Have a common base class be responsible for HelperContext initialization. Then have a tools-test driver class responsible for a. iterating over impl tests, b. performing code-generation & clean-up based on its recorded schemas, c. code-generate a subclass of the impl test which knows how to perform static HelperContext initialization, d. compile and invoke the code-generated test.

        Any thoughts?

        • Ron
        Show
        Ron Gavlin added a comment - Hi Kelvin, I opened this issue to raise awareness of the problem. I have some initial design ideas but nothing real concrete at this point. A few approaches I have thought about are: 1. Brute force approach - have existing impl tests extract their HelperContext initialization into a template method. Then have corresponding tools-test classes extend the impl tests and override the template method with static-factory-based initialization. 2. More Elegant approach - Record the schemas used by each impl test somewhere. Have a common base class be responsible for HelperContext initialization. Then have a tools-test driver class responsible for a. iterating over impl tests, b. performing code-generation & clean-up based on its recorded schemas, c. code-generate a subclass of the impl test which knows how to perform static HelperContext initialization, d. compile and invoke the code-generated test. Any thoughts? Ron
        ant elder made changes -
        Field Original Value New Value
        Fix Version/s Java-SDO-Next [ 12312262 ]
        Hide
        Kelvin Goodson added a comment -

        Thanks for your suggestions. One small problem we would have to overcome is that maven does not make test code available from one project to another project that expresses a dependency on it. Moving some part of the test code into the implementation wouldn't be nice from the point of view the run-time artifacts that are distributed. Separating testing out into higher level projects means we wouldn't get the automatic benefits of an immediate notification of the introduction of an issue when building with maven.

        We could insert an intermediate "test-core" project, that is not part of the SDO distribution, either between the API project and the lib projects (in dependency terms) or between the lib and the impl projects. The bodies of the tests could then be implemented in the main source folder hierarchy of that project (as opposed to the test code hierarchy). The test bodies would be in abstract classes with template methods as you suggest. The lib, impl and tools-tests projects can then declare a test scope dependency on that new project, and the test programs be implemented in the test code hierarchy of the implementation projects, extending the behaviour of the test programs in the new project.

        Similarly the tools-test project could declare a dependency on the new project. There would be a deficiency here that I haven't got my head round yet. If we want a single location for the schemas, then they would have to be in the new project. This means I think that we wouldn't be able to rely on maven's "generate" phase to handle the generation for us in the tools-test project. One possibility would be to use svn's "externals" property to make the same schema files available to the test resources of both projects, but that may be a trip hazard.

        Show
        Kelvin Goodson added a comment - Thanks for your suggestions. One small problem we would have to overcome is that maven does not make test code available from one project to another project that expresses a dependency on it. Moving some part of the test code into the implementation wouldn't be nice from the point of view the run-time artifacts that are distributed. Separating testing out into higher level projects means we wouldn't get the automatic benefits of an immediate notification of the introduction of an issue when building with maven. We could insert an intermediate "test-core" project, that is not part of the SDO distribution, either between the API project and the lib projects (in dependency terms) or between the lib and the impl projects. The bodies of the tests could then be implemented in the main source folder hierarchy of that project (as opposed to the test code hierarchy). The test bodies would be in abstract classes with template methods as you suggest. The lib, impl and tools-tests projects can then declare a test scope dependency on that new project, and the test programs be implemented in the test code hierarchy of the implementation projects, extending the behaviour of the test programs in the new project. Similarly the tools-test project could declare a dependency on the new project. There would be a deficiency here that I haven't got my head round yet. If we want a single location for the schemas, then they would have to be in the new project. This means I think that we wouldn't be able to rely on maven's "generate" phase to handle the generation for us in the tools-test project. One possibility would be to use svn's "externals" property to make the same schema files available to the test resources of both projects, but that may be a trip hazard.

          People

          • Assignee:
            Unassigned
            Reporter:
            Ron Gavlin
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development