Harmony
  1. Harmony
  2. HARMONY-3206

[drlvm][testing] DRLVM Validation Test Suite

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: build - test - ci
    • Labels:
      None

      Description

      Test suite consist of about 4400 functional tests covering
      various parts of DRLVM including byte code execution, verifier,
      JNI and JVMTI implementation.

      1. drlvm-vts-bundle.tar.gz
        1.81 MB
        Alexander Kleymenov
      2. fixed-hidden.zip
        2 kB
        Alexander Kleymenov
      3. th_update.zip
        10 kB
        Vladimir Ivanov
      4. th_update.zip
        10 kB
        Vladimir Ivanov
      5. vtsvm2bt-updated.zip
        4 kB
        Alexander Kleymenov

        Issue Links

          Activity

          Hide
          Alexander Kleymenov added a comment -

          Attached drlvm-vts-bundle.tar.gz archive contains the suite.

          Show
          Alexander Kleymenov added a comment - Attached drlvm-vts-bundle.tar.gz archive contains the suite.
          Hide
          Pavel Pervov added a comment -

          I have the problem with this archive. Two files in vts/vm/src/test/vm/jni/share/ - ConstructedClass.hidden and LoadedClass.hidden - are packed as text files (although they are Java class files) and end-of-line symbols are converted to Windows format (0xD 0xA) from original byte (0xA) in them.
          Numerous VTS tests fail because of this.

          Could you please check the archive and reattach corrected version?

          Show
          Pavel Pervov added a comment - I have the problem with this archive. Two files in vts/vm/src/test/vm/jni/share/ - ConstructedClass.hidden and LoadedClass.hidden - are packed as text files (although they are Java class files) and end-of-line symbols are converted to Windows format (0xD 0xA) from original byte (0xA) in them. Numerous VTS tests fail because of this. Could you please check the archive and reattach corrected version?
          Hide
          Alexander Kleymenov added a comment -

          Here you are! Thanks for your work with this!

          Show
          Alexander Kleymenov added a comment - Here you are! Thanks for your work with this!
          Hide
          Vladimir Ivanov added a comment -

          Minor update to harness.
          These classes are useful to store results in "CC log" supported format.
          Now this suite can be easily added under CC/CI.

          Show
          Vladimir Ivanov added a comment - Minor update to harness. These classes are useful to store results in "CC log" supported format. Now this suite can be easily added under CC/CI.
          Hide
          Vladimir Ivanov added a comment -

          copyright message for new file was updated from "Copyright 2005-2006" to "Copyright 2007"

          Show
          Vladimir Ivanov added a comment - copyright message for new file was updated from "Copyright 2005-2006" to "Copyright 2007"
          Hide
          Alexander Kleymenov added a comment -

          Vladimir,

          I've looked on your improvement and found it quite useful.
          As I see your enhancement implements JUnit XML formatter for the Test Harness Tool used for VTSVM tests execution and placed under VTSVM contribution archive. So now it's possible to have VTSVM test results in commonly used XML format! It's definitely big quality improvement of the test suite.
          Now I'm working on integration of VTSVM into new BT Infrastructure proposed with HARMONY-3501 JIRA issue and it will use your formatter for VTSVM Run Status notifications.

          Thanks you!
          Alexander

          Show
          Alexander Kleymenov added a comment - Vladimir, I've looked on your improvement and found it quite useful. As I see your enhancement implements JUnit XML formatter for the Test Harness Tool used for VTSVM tests execution and placed under VTSVM contribution archive. So now it's possible to have VTSVM test results in commonly used XML format! It's definitely big quality improvement of the test suite. Now I'm working on integration of VTSVM into new BT Infrastructure proposed with HARMONY-3501 JIRA issue and it will use your formatter for VTSVM Run Status notifications. Thanks you! Alexander
          Hide
          Alexander Kleymenov added a comment -

          Hello,

          Attached archive contains the implementation of the adaptor interface necessary for VTSVM to BTI integration (Build and Testing Infrastructure is supplied with HARMONY-3501).
          To integrate the suite (with all of the current updates) to the BTI, you need to perform the following steps:
          1. cd $BTI_ROOT_DIR
          2. unzip vtsvm2bt.zip
          3. mkdir $BTI_ROOT_DIR/tests/vtsvm
          4. cd $BTI_ROOT_DIR/tests/vtsvm
          5. untargzip drlvm-vts-bundle.tar.gz
          6. unzip fixed-hidden.zip (do overwrite old files)
          7. cd $BTI_ROOT_DIR/tests/vtsvm/tools/harness
          8. unzip th_update.zip (do overwrite old files)

          Now all of the files are on their places. To execute VTSVM under BTI:
          1. tune required environment variables in buildtest shell script.
          2. type
          > buildtest install
          to install the BTI
          3. type
          > buildtest -Dtest.suites="classlib,drlvm,vtsvm" run
          to launch single VTSVM run.
          It will checkout and build the latest versions of Harmony DRLVM and CLASSLIB and execute VTSVM over them.
          It's possible to not to select classlib and drlvm test suites for run. In such a case you will be demanded by framework to setup the path to tested runtime.
          4. type
          > buildtest -Dtest.suites="classlib,drlvm,vtsvm" run-cc
          to launch continuous run under CruiseControl system.
          It will checkout and build the latest versions of Harmony DRLVM and CLASSLIB and execute VTSVM over them after every update made into SVN repository. To receive VTSVM status notifications you need to uncomment framework.parameters.usemail.* properties in framework.local.properties file and provide correct values for them.

          Please, try this integration scripts. Any feedback is welcome!

          Thanks,
          Alexander

          Show
          Alexander Kleymenov added a comment - Hello, Attached archive contains the implementation of the adaptor interface necessary for VTSVM to BTI integration (Build and Testing Infrastructure is supplied with HARMONY-3501 ). To integrate the suite (with all of the current updates) to the BTI, you need to perform the following steps: 1. cd $BTI_ROOT_DIR 2. unzip vtsvm2bt.zip 3. mkdir $BTI_ROOT_DIR/tests/vtsvm 4. cd $BTI_ROOT_DIR/tests/vtsvm 5. untargzip drlvm-vts-bundle.tar.gz 6. unzip fixed-hidden.zip (do overwrite old files) 7. cd $BTI_ROOT_DIR/tests/vtsvm/tools/harness 8. unzip th_update.zip (do overwrite old files) Now all of the files are on their places. To execute VTSVM under BTI: 1. tune required environment variables in buildtest shell script. 2. type > buildtest install to install the BTI 3. type > buildtest -Dtest.suites="classlib,drlvm,vtsvm" run to launch single VTSVM run. It will checkout and build the latest versions of Harmony DRLVM and CLASSLIB and execute VTSVM over them. It's possible to not to select classlib and drlvm test suites for run. In such a case you will be demanded by framework to setup the path to tested runtime. 4. type > buildtest -Dtest.suites="classlib,drlvm,vtsvm" run-cc to launch continuous run under CruiseControl system. It will checkout and build the latest versions of Harmony DRLVM and CLASSLIB and execute VTSVM over them after every update made into SVN repository. To receive VTSVM status notifications you need to uncomment framework.parameters.usemail.* properties in framework.local.properties file and provide correct values for them. Please, try this integration scripts. Any feedback is welcome! Thanks, Alexander
          Hide
          Mikhail Loenko added a comment -

          vote passed

          Show
          Mikhail Loenko added a comment - vote passed
          Hide
          Alexander Kleymenov added a comment -

          Attached archive contains updated BTI integration scripts. Old one is obsolete.
          These scripts were tested on windows and linux.

          Show
          Alexander Kleymenov added a comment - Attached archive contains updated BTI integration scripts. Old one is obsolete. These scripts were tested on windows and linux.
          Hide
          Stepan Mishura added a comment -

          The contribution was committed to BTI branch 2.0 at r545757.
          Please check that it looks as you expected.

          Show
          Stepan Mishura added a comment - The contribution was committed to BTI branch 2.0 at r545757. Please check that it looks as you expected.
          Hide
          Stepan Mishura added a comment -

          Alexander, could you update adaptor - it doesn't correspond to the current repository state?

          Show
          Stepan Mishura added a comment - Alexander, could you update adaptor - it doesn't correspond to the current repository state?
          Hide
          Alexander Kleymenov added a comment -

          The contribution was not applied exactly as expected - the directory structure has been changed.
          I've fixed the BTI integration scripts to comply with this new structure.
          Please, find them in reattached archive.

          Show
          Alexander Kleymenov added a comment - The contribution was not applied exactly as expected - the directory structure has been changed. I've fixed the BTI integration scripts to comply with this new structure. Please, find them in reattached archive.
          Hide
          Stepan Mishura added a comment -

          Thanks Alexander - the patch with adaptor for the suite was applied to BTI branch 2.0 at r550398.
          Please check that the patch was applied as you expected and close the issue if everything is OK for you.

          Show
          Stepan Mishura added a comment - Thanks Alexander - the patch with adaptor for the suite was applied to BTI branch 2.0 at r550398. Please check that the patch was applied as you expected and close the issue if everything is OK for you.
          Hide
          Alexander Kleymenov added a comment -

          Looks as expected. I'm closing it.

          Show
          Alexander Kleymenov added a comment - Looks as expected. I'm closing it.

            People

            • Assignee:
              Stepan Mishura
              Reporter:
              Alexander Kleymenov
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development