Muse
  1. Muse
  2. MUSE-244 Test/compliance framework for Muse
  3. MUSE-259

Flexible test infrastructure and initial set of extensible functional tests

    Details

    • Type: Sub-task Sub-task
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 2.2.1
    • Component/s: Other
    • Labels:
      None

      Description

      Feature 244 encompases/requests multiple functionality. This sub-task is created to facilitate tracking at a more individual, granular level. This sub-task is to track the delivery of a flexible infrastructure and initial, extensible suite of functional tests

      1. Muse-Tools_TestCases_v1.0.pdf
        172 kB
        bhanu p velampati
      2. tests.zip
        1.81 MB
        Kam K. Yee
      3. WS-BaseNotification_1.3_TestCases_v1.0.pdf
        214 kB
        Nidhi Singh
      4. WS-DistributedManagement-MUWS1_1.1_TestCases_v1.0.pdf
        118 kB
        Nidhi Singh
      5. WS-DistributedManagement-MUWS2_1.1_TestCases_v1.0.pdf
        233 kB
        Nidhi Singh
      6. WS-MetadataExchange_TestCases_v1.0.pdf
        107 kB
        Nidhi Singh
      7. WS-ResourceFramework_1.2_Testcases_v1.0.pdf
        155 kB
        Nidhi Singh
      8. WS-Topics_1.3_TestCases_v1.0.pdf
        127 kB
        Nidhi Singh

        Activity

        Hide
        Kam K. Yee added a comment -

        ======================================
        PROPOSED IMPLEMENTATION - HIGH LEVEL
        ======================================
        Following are some initial thoughts/proposal for the implementation of the various tests and framework. (This proposal incorporates and adds to the input from Andrew Eberbach's previous postings and email exchanges (Thanks! Andrew).

        • The unit, functional, and stress tests will be JUnit based.
        • ANT scripts will be created to setup and invoke these JUnit test suites.

        FUNCTIONAL TESTING:

        • This would loosely follow Dan's current approach of top-level directories that are related to specific capabilties (wsrf, wsn, etc). The structure here isn't as easy to generalize, but some directory/hierarchical organization that seems sane would suffice. Again, the goal here is self-containment as to avoid an overly complex test environment.
        • A new target will be introduced to the build.xml to build a new archive (muse-test-fvt-bin) containing only the functional test code and needed artifacts (e.g., WSDLs & XSDs).
        • The identified steps for functional testing are as follows:
          1. Download or locally build muse-bin runtime archive
          2. Download or locally build muse-test-fvt-bin archive
          3. Unzip muse-bin
          4. Unzip muse-test-fvt-bin
          5. Run muse-bin code generation tool to generate endpoint(s) to be deployed to target container
          6. Deploy generated endpoint(s) to the target container
          7. Startup target container with generated endpoint(s)
          8. Invoke execution of JUnit functional test suites
        • A new functional test ANT script will be created to invoke and execute the different JUnit test suites.
          Later on when full automation is introduced, the automation framework will be invoking these ANT script target to automatically execute the test suites.
        • This design also allows for developers to do a sanity check by running these tests locally on their machines prior to checking code into the repository.

        Finally, since modifying something simple is usually easier than generating something simple from scratch, we will create two "read only" top-level directories which would lay out a reasonable skeleton for users contributing their own junit and functional tests for top-level directories.

        Show
        Kam K. Yee added a comment - ====================================== PROPOSED IMPLEMENTATION - HIGH LEVEL ====================================== Following are some initial thoughts/proposal for the implementation of the various tests and framework. (This proposal incorporates and adds to the input from Andrew Eberbach's previous postings and email exchanges (Thanks! Andrew). The unit, functional, and stress tests will be JUnit based. ANT scripts will be created to setup and invoke these JUnit test suites. FUNCTIONAL TESTING: This would loosely follow Dan's current approach of top-level directories that are related to specific capabilties (wsrf, wsn, etc). The structure here isn't as easy to generalize, but some directory/hierarchical organization that seems sane would suffice. Again, the goal here is self-containment as to avoid an overly complex test environment. A new target will be introduced to the build.xml to build a new archive (muse-test-fvt-bin) containing only the functional test code and needed artifacts (e.g., WSDLs & XSDs). The identified steps for functional testing are as follows: 1. Download or locally build muse-bin runtime archive 2. Download or locally build muse-test-fvt-bin archive 3. Unzip muse-bin 4. Unzip muse-test-fvt-bin 5. Run muse-bin code generation tool to generate endpoint(s) to be deployed to target container 6. Deploy generated endpoint(s) to the target container 7. Startup target container with generated endpoint(s) 8. Invoke execution of JUnit functional test suites A new functional test ANT script will be created to invoke and execute the different JUnit test suites. Later on when full automation is introduced, the automation framework will be invoking these ANT script target to automatically execute the test suites. This design also allows for developers to do a sanity check by running these tests locally on their machines prior to checking code into the repository. Finally, since modifying something simple is usually easier than generating something simple from scratch, we will create two "read only" top-level directories which would lay out a reasonable skeleton for users contributing their own junit and functional tests for top-level directories.
        Hide
        bhanu p velampati added a comment -

        Attached WS_Specs_Testcases.zip file contains PDF files. Each PDF file has set of Test cases (for specs MetadataExchange, WSRF, WS-Topics, Base Notification, MUWS1.1 part1, MUWS1.1 part2).

        Note: These are the initial set of Test cases

        Show
        bhanu p velampati added a comment - Attached WS_Specs_Testcases.zip file contains PDF files. Each PDF file has set of Test cases (for specs MetadataExchange, WSRF, WS-Topics, Base Notification, MUWS1.1 part1, MUWS1.1 part2). Note: These are the initial set of Test cases
        Hide
        Nidhi Singh added a comment - - edited

        Attached documents contain initial set of testcases for different components of Apache Muse v2.2.0. Brief summary of the contents of each of the documents is as follows:

        1. WS-BaseNotification_1.3_TestCases_v1.0.pdf- This document contains testcases for WS-Notification component of Apache Muse 2.2.0. This component/module is based on WS-BaseNotification v1.3 OASIS web services standard.

        2. WS-DistributedManagement-MUWS1_1.1_TestCases_v1.0.pdf & WS-DistributedManagement-MUWS2_1.1_TestCases_v1.0.pdf: These files contain testcases for WSDM MUWS component of Apache Muse 2.2.0. These components are based on WS-DistributedManagement MUWS(1.1) Part 1 and Part 2 OASIS web services standards.

        3. WS-MetadataExchange_TestCases_v1.0.pdf: This document contains testcases for WS-MetadataExchange component of Apache Muse 2.2.0. This component is based on WS-MetaDataExchange 09/04 OASIS web services standard.

        4. WS-ResourceFramework_1.2_Testcases_v1.0.pdf: This document contains a set of testcases for WS-ResourceFramework module of Apache Muse 2.2.0. This module is based on WS-ResourceFramework v1.2 OASIS web services standard.

        5. WS-Topics_1.3_TestCases_v1.0.pdf: This document contains testcases for WS-Topics component of Apache Muse 2.2.0. This component provides an implementation of WS-Topics v1.3 OASIS web services standard.

        Show
        Nidhi Singh added a comment - - edited Attached documents contain initial set of testcases for different components of Apache Muse v2.2.0. Brief summary of the contents of each of the documents is as follows: 1. WS-BaseNotification_1.3_TestCases_v1.0.pdf- This document contains testcases for WS-Notification component of Apache Muse 2.2.0. This component/module is based on WS-BaseNotification v1.3 OASIS web services standard. 2. WS-DistributedManagement-MUWS1_1.1_TestCases_v1.0.pdf & WS-DistributedManagement-MUWS2_1.1_TestCases_v1.0.pdf: These files contain testcases for WSDM MUWS component of Apache Muse 2.2.0. These components are based on WS-DistributedManagement MUWS(1.1) Part 1 and Part 2 OASIS web services standards. 3. WS-MetadataExchange_TestCases_v1.0.pdf: This document contains testcases for WS-MetadataExchange component of Apache Muse 2.2.0. This component is based on WS-MetaDataExchange 09/04 OASIS web services standard. 4. WS-ResourceFramework_1.2_Testcases_v1.0.pdf: This document contains a set of testcases for WS-ResourceFramework module of Apache Muse 2.2.0. This module is based on WS-ResourceFramework v1.2 OASIS web services standard. 5. WS-Topics_1.3_TestCases_v1.0.pdf: This document contains testcases for WS-Topics component of Apache Muse 2.2.0. This component provides an implementation of WS-Topics v1.3 OASIS web services standard.
        Hide
        Dan Jemiolo added a comment -

        Very impressive set of test cases so far. One area that you haven't covered is muse-tools - making sure that there is no unexpected changes in the generated artifacts and command output is just as important as the standards compliance, because it can have just as big an impact on deployers. Please make sure there is sufficient coverage of all of the tooling scenarios (described in the WSDL2Java sections of the reference manual).

        The test process and directory setup sounds good - my only suggestion is to separate all the test projects into /trunk/tests (sibling of /trunk/modules) so that people can still checkout all of the muse projects from /modules without all of the test infrastructure (could be very heavy when you consider the sample data). Also, in MUSE-244 you mention creating an easy-to-use unit test skeleton project, but it's not mentioned in this issue - I just want to make sure this is included because Andrew was definitely right about making it easier for people to contribute tests.

        Show
        Dan Jemiolo added a comment - Very impressive set of test cases so far. One area that you haven't covered is muse-tools - making sure that there is no unexpected changes in the generated artifacts and command output is just as important as the standards compliance, because it can have just as big an impact on deployers. Please make sure there is sufficient coverage of all of the tooling scenarios (described in the WSDL2Java sections of the reference manual). The test process and directory setup sounds good - my only suggestion is to separate all the test projects into /trunk/tests (sibling of /trunk/modules) so that people can still checkout all of the muse projects from /modules without all of the test infrastructure (could be very heavy when you consider the sample data). Also, in MUSE-244 you mention creating an easy-to-use unit test skeleton project, but it's not mentioned in this issue - I just want to make sure this is included because Andrew was definitely right about making it easier for people to contribute tests.
        Hide
        bhanu p velampati added a comment -

        Attached document contains Test cases for wsdl2java and wsdlmerge tools

        Show
        bhanu p velampati added a comment - Attached document contains Test cases for wsdl2java and wsdlmerge tools
        Hide
        Kam K. Yee added a comment -

        Attached is the implementation of this new feature contributed by the IBM Team

        Show
        Kam K. Yee added a comment - Attached is the implementation of this new feature contributed by the IBM Team
        Hide
        Chris Twiner added a comment -

        assigning to me as I want to try to introduce the tests (and the improved build script) as quickly as possible before I add other fixes in

        Show
        Chris Twiner added a comment - assigning to me as I want to try to introduce the tests (and the improved build script) as quickly as possible before I add other fixes in

          People

          • Assignee:
            Chris Twiner
            Reporter:
            Kam K. Yee
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development