Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 10.4.1.3, 10.5.1.1
    • Component/s: JMX, Test
    • Labels:
      None

      Description

      DERBY-1387 implements a new feature set in Derby: Java Management eXtentions (JMX), i.e. standards-based management and monitoring capabilities.

      As these are completely new features providing additional interfaces to a running Derby system, a set of functional regression tests should be added. The new tests should verify that Derby's public JMX API works as intended, and should be able to detect regressions in this area in the future. There is a functional specification (jmxFuncspec.html) attached to DERBY-1387 which may be used as reference.

      1. TestJMXp9.java
        5 kB
        John H. Embretsen
      2. d3385_versionMBeanTest_01.diff
        25 kB
        John H. Embretsen
      3. d3385_ant-junit-all.diff
        2 kB
        John H. Embretsen
      4. d3385_versionMBeanTest_02.diff
        8 kB
        John H. Embretsen
      5. d3385_jdbcMBeanTest_01.diff
        14 kB
        John H. Embretsen
      6. d3385_jdbcMBeanTest_02.diff
        22 kB
        John H. Embretsen
      7. d3385_jdbcMBeanTest_03.diff
        12 kB
        John H. Embretsen

        Activity

        Hide
        John H. Embretsen added a comment -

        Attaching a simple sample program, TestJMXp9.java, demonstrating how to access Derby's JMX features programmatically, from a separate JVM on a local or remote machine. Will probably only work with patch 9 of DERBY-1387.

        Show
        John H. Embretsen added a comment - Attaching a simple sample program, TestJMXp9.java, demonstrating how to access Derby's JMX features programmatically, from a separate JVM on a local or remote machine. Will probably only work with patch 9 of DERBY-1387 .
        Hide
        John H. Embretsen added a comment -

        Attaching patch d3385_versionMBeanTest_01.diff in case anyone wants to review it while I'm on vacation (1 week).

        The patch only includes tests for the Version MBean. Currently only attributes (names and types) for the VersionMBean instance for derby.jar are tested (I expect to add more later).

        A java/testing/org/apache/derbyTesting/functionTests/tests/management

        New package for JMX management tests. Built with JDK1.5.

        A java/testing/org/apache/derbyTesting/functionTests/tests/management/_Suite.java

        Test suite including all relevant management tests.

        A java/testing/org/apache/derbyTesting/functionTests/tests/management/VersionMBeanTest.java

        Test fixtures/logic for the VersionMBean.

        A java/testing/org/apache/derbyTesting/functionTests/tests/management/build.xml

        Build script for management tests.

        M java/testing/org/apache/derbyTesting/junit/TestConfiguration.java

        Modifications to add jmxPort as a field in TestConfiguration. Default value is 9999 (arbitrarily chosen) for now. Let me know if there is a better way to do this.

        M java/testing/build.xml

        Builds management tests as part of the overall test build.

        This patch does not add the new tests to a larger suite such as suites.All or junit-all.

        1 of the 10 tests will fail until the VersionMBean is updated (change attribute name "MaintVersion" to MaintenanceVersion"). Running the tests requires JDK1.5 or better.

        Show
        John H. Embretsen added a comment - Attaching patch d3385_versionMBeanTest_01.diff in case anyone wants to review it while I'm on vacation (1 week). The patch only includes tests for the Version MBean. Currently only attributes (names and types) for the VersionMBean instance for derby.jar are tested (I expect to add more later). A java/testing/org/apache/derbyTesting/functionTests/tests/management New package for JMX management tests. Built with JDK1.5. A java/testing/org/apache/derbyTesting/functionTests/tests/management/_Suite.java Test suite including all relevant management tests. A java/testing/org/apache/derbyTesting/functionTests/tests/management/VersionMBeanTest.java Test fixtures/logic for the VersionMBean. A java/testing/org/apache/derbyTesting/functionTests/tests/management/build.xml Build script for management tests. M java/testing/org/apache/derbyTesting/junit/TestConfiguration.java Modifications to add jmxPort as a field in TestConfiguration. Default value is 9999 (arbitrarily chosen) for now. Let me know if there is a better way to do this. M java/testing/build.xml Builds management tests as part of the overall test build. This patch does not add the new tests to a larger suite such as suites.All or junit-all. 1 of the 10 tests will fail until the VersionMBean is updated (change attribute name "MaintVersion" to MaintenanceVersion"). Running the tests requires JDK1.5 or better.
        Hide
        Daniel John Debrunner added a comment -

        I'll work on committing this. I think an improvement would be to separate out the generic jmx code from VersionMbeanTest into an abstract JMXTest class that VersionMbean would extend, then allowing other tests for other MBeans to be written easily.

        One thought is to ensure that the fixtures are flexible enough to run with different mechanisms to get the MBean, e.g. local jmx, remote jmx, accessing the platform MBean server from the same vm etc. I think what you have provided will allow this.

        Thanks for providing the initial setup, makes writing tests for others easier.

        Show
        Daniel John Debrunner added a comment - I'll work on committing this. I think an improvement would be to separate out the generic jmx code from VersionMbeanTest into an abstract JMXTest class that VersionMbean would extend, then allowing other tests for other MBeans to be written easily. One thought is to ensure that the fixtures are flexible enough to run with different mechanisms to get the MBean, e.g. local jmx, remote jmx, accessing the platform MBean server from the same vm etc. I think what you have provided will allow this. Thanks for providing the initial setup, makes writing tests for others easier.
        Hide
        Daniel John Debrunner added a comment -

        Patch committed as-is - revision 631355.

        Improved testing could include checking the values returned by getting the VersionMBean's attributues.

        Show
        Daniel John Debrunner added a comment - Patch committed as-is - revision 631355. Improved testing could include checking the values returned by getting the VersionMBean's attributues.
        Hide
        Daniel John Debrunner added a comment -

        Moved the utility code out of VersionMBean class into an MBeanTest class. Left the code that sets up the network server etc. in VersionMBean, not sure of its correct location. The functionality seems more suited to being decorators, e.g. jmx decorators that setup jmx and provide configfuration over how the JMX connection is made. Thus allowing the same fixtures to be used in multiple environments.

        Show
        Daniel John Debrunner added a comment - Moved the utility code out of VersionMBean class into an MBeanTest class. Left the code that sets up the network server etc. in VersionMBean, not sure of its correct location. The functionality seems more suited to being decorators, e.g. jmx decorators that setup jmx and provide configfuration over how the JMX connection is made. Thus allowing the same fixtures to be used in multiple environments.
        Hide
        John H. Embretsen added a comment -

        Attached a patch for review, d3385_ant-junit-all.diff, which adds the management test suite as a new test run target ( junit-jmx ) in the top-level build.xml file. The tests are run (and automatically included in junit-all) only if the VM running Ant supports J2SE 5.0 or higher.

        Show
        John H. Embretsen added a comment - Attached a patch for review, d3385_ant-junit-all.diff, which adds the management test suite as a new test run target ( junit-jmx ) in the top-level build.xml file. The tests are run (and automatically included in junit-all) only if the VM running Ant supports J2SE 5.0 or higher.
        Hide
        Daniel John Debrunner added a comment -

        Junit patch committed Revision: 634284 - Thanks John.

        Show
        Daniel John Debrunner added a comment - Junit patch committed Revision: 634284 - Thanks John.
        Hide
        John H. Embretsen added a comment -

        Thanks for committing both patches, Dan. Also thank you very much for your contributions to JMX test configuration and setup the past week, and for testing the ManagementMBean!

        I intend to upload a patch soon adding fixtures for testing the VersionMBean for derbynet.jar (probably of limited value, but it doesn't hurt), and some updated comments base on the latest committed changes.

        I don't plan on adding checks for the actual values of the attributes yet, as I'm not sure how to determine a lot of the expected values (version numbers etc.) in a way that won't require a lot of maintenance.

        Show
        John H. Embretsen added a comment - Thanks for committing both patches, Dan. Also thank you very much for your contributions to JMX test configuration and setup the past week, and for testing the ManagementMBean! I intend to upload a patch soon adding fixtures for testing the VersionMBean for derbynet.jar (probably of limited value, but it doesn't hurt), and some updated comments base on the latest committed changes. I don't plan on adding checks for the actual values of the attributes yet, as I'm not sure how to determine a lot of the expected values (version numbers etc.) in a way that won't require a lot of maintenance.
        Hide
        Daniel John Debrunner added a comment -

        On the version numbers, one approach is to cross-check with other mechanisms to return the same information, such as sysinfo, or DatabaseMetaData, JDBC Driver etc. Then at least one knows they are consistent.

        Show
        Daniel John Debrunner added a comment - On the version numbers, one approach is to cross-check with other mechanisms to return the same information, such as sysinfo, or DatabaseMetaData, JDBC Driver etc. Then at least one knows they are consistent.
        Hide
        John H. Embretsen added a comment -

        Attached a patch improving the VersionMBeanTest by adding fixtures for the derbynet.jar MBean, and brushing up some comments.

        M java/testing/org/apache/derbyTesting/functionTests/tests/management/MBeanTest.java
        M java/testing/org/apache/derbyTesting/functionTests/tests/management/VersionMBeanTest.java

        I will consider adding attribute value value checking later...

        Show
        John H. Embretsen added a comment - Attached a patch improving the VersionMBeanTest by adding fixtures for the derbynet.jar MBean, and brushing up some comments. M java/testing/org/apache/derbyTesting/functionTests/tests/management/MBeanTest.java M java/testing/org/apache/derbyTesting/functionTests/tests/management/VersionMBeanTest.java I will consider adding attribute value value checking later...
        Hide
        John H. Embretsen added a comment -

        Thanks for committing patch 02 for the VersionMBeanTest with revision 634433

        Now attaching a new patch, d3385_jdbcMBeanTest_01.diff, which implements a junit test for the JDBCMBean. With this patch, the Emma tool reports 100% test coverage of org.apache.derby.jdbc.JDBC.java.

        M java/testing/org/apache/derbyTesting/functionTests/tests/management/MBeanTest.java

        • Extending BaseJDBCTestCase instead of BaseTestCase, since the new test subclass requires a JDBC connection in order to verify certain attribute values.
        • Added a return type (Object) to the invokeOperation method, and added a similar method for invoking an operation with arguments.
        • Added methods for retrieving sysinfo output from either the network server or the local VM (the latter is not used currently). This could potentially be useful for other tests as well, but I wasn't sure where to put it, so it lives in MBeanTest for now.
        • Added a couple of new assert methods.
        • Fixed some comments.

        A java/testing/org/apache/derbyTesting/functionTests/tests/management/JDBCMBeanTest.java

        • The new test class testing the presence, name, return type and return value of all expected attributes of the JDBCMBean, as well as the result of invoking the (only) operation.

        M java/testing/org/apache/derbyTesting/functionTests/tests/management/_Suite.java

        • Adding JDBCMBeanTest to the management test suite.

        Please review...

        Show
        John H. Embretsen added a comment - Thanks for committing patch 02 for the VersionMBeanTest with revision 634433 Now attaching a new patch, d3385_jdbcMBeanTest_01.diff, which implements a junit test for the JDBCMBean. With this patch, the Emma tool reports 100% test coverage of org.apache.derby.jdbc.JDBC.java. M java/testing/org/apache/derbyTesting/functionTests/tests/management/MBeanTest.java Extending BaseJDBCTestCase instead of BaseTestCase, since the new test subclass requires a JDBC connection in order to verify certain attribute values. Added a return type (Object) to the invokeOperation method, and added a similar method for invoking an operation with arguments. Added methods for retrieving sysinfo output from either the network server or the local VM (the latter is not used currently). This could potentially be useful for other tests as well, but I wasn't sure where to put it, so it lives in MBeanTest for now. Added a couple of new assert methods. Fixed some comments. A java/testing/org/apache/derbyTesting/functionTests/tests/management/JDBCMBeanTest.java The new test class testing the presence, name, return type and return value of all expected attributes of the JDBCMBean, as well as the result of invoking the (only) operation. M java/testing/org/apache/derbyTesting/functionTests/tests/management/_Suite.java Adding JDBCMBeanTest to the management test suite. Please review...
        Hide
        Daniel John Debrunner added a comment -

        I'll commit the jdbc patch but it needs a follow-on change. This code is not testing what it is meant to be testing:

        DatabaseMetaData dbmd = getConnection().getMetaData();
        int expected = dbmd.getDriverMinorVersion();
        assertIntAttribute(expected, getJdbcMBeanObjectName(), "MinorVersion");

        Since the suite wrapped the tests in a network server decorator, the connection, DatabaseMetaData and hence the driver version are all coming
        from the client's driver, whereas the JDBCMBean is wrapping the embedded driver. While the versions are typically the same there is no requirement,
        especially if folks try out combination testing.

        Show
        Daniel John Debrunner added a comment - I'll commit the jdbc patch but it needs a follow-on change. This code is not testing what it is meant to be testing: DatabaseMetaData dbmd = getConnection().getMetaData(); int expected = dbmd.getDriverMinorVersion(); assertIntAttribute(expected, getJdbcMBeanObjectName(), "MinorVersion"); Since the suite wrapped the tests in a network server decorator, the connection, DatabaseMetaData and hence the driver version are all coming from the client's driver, whereas the JDBCMBean is wrapping the embedded driver. While the versions are typically the same there is no requirement, especially if folks try out combination testing.
        Hide
        John H. Embretsen added a comment -

        OK, thanks! I kind of knew that, but it must have slipped my mind when writing the code. And those tests passed by accident, so I didn't notice it...

        Show
        John H. Embretsen added a comment - OK, thanks! I kind of knew that, but it must have slipped my mind when writing the code. And those tests passed by accident, so I didn't notice it...
        Hide
        Daniel John Debrunner added a comment -

        As a style issue I think the sysinfo code would be better as a test generic utility in o.a.dt.junit.something since it is independent of MBeans or JMX.

        As a community we do seem to have a habit of adding generic utilities into specific tests which leads to the same utility in multiple places (but sometimes implemented differently Something the old test harness was very prone to, it would be good if we can avoid this with Junit tests.

        Show
        Daniel John Debrunner added a comment - As a style issue I think the sysinfo code would be better as a test generic utility in o.a.dt.junit.something since it is independent of MBeans or JMX. As a community we do seem to have a habit of adding generic utilities into specific tests which leads to the same utility in multiple places (but sometimes implemented differently Something the old test harness was very prone to, it would be good if we can avoid this with Junit tests.
        Hide
        John H. Embretsen added a comment -

        OK, I will move the sysinfo code to somewhere in o.a.dt.junit - that was my original plan anyways. I will also look into fixing the metadata/driver issue, but I most likely won't be able to do any of this before feature freeze and the cutting of the new branch.

        Show
        John H. Embretsen added a comment - OK, I will move the sysinfo code to somewhere in o.a.dt.junit - that was my original plan anyways. I will also look into fixing the metadata/driver issue, but I most likely won't be able to do any of this before feature freeze and the cutting of the new branch.
        Hide
        John H. Embretsen added a comment - - edited

        Attached patch file d3385_jdbcMBeanTest_02.diff:

        M java/testing/org/apache/derbyTesting/functionTests/tests/management/MBeanTest.java
        M java/testing/org/apache/derbyTesting/functionTests/tests/management/JDBCMBeanTest.java
        M java/testing/org/apache/derbyTesting/junit/TestConfiguration.java
        M java/testing/org/apache/derbyTesting/junit/NetworkServerTestSetup.java
        M java/testing/org/apache/derbyTesting/junit/Utilities.java

        This patch modifies the JDBCMBeanTest so that the expected values for the MajorVersion and MinorVersion attributes are obtained from the embedded driver instead of the client driver.

        I wasn't sure how to do this, but I ended up decorating these two test fixtures a bit differently than the rest of the fixtures, by using a ServerSetup which JDBCClient is set to embedded. The patch adds some logic in TestConfiguration.java and MBeanTest.java for handling this.

        The patch also moves the two generic utility methods for fetching sysinfo output from MBeanTest.java to org.apache.derbyTesting.junit.Utilities.java. In addition, the patch revises some Javadoc comments in NetworkServerTestSetup.java.

        When testing this patch I once ran into timeout issues similar to those reported in DERBY-3515 when running the management Suite on WinXP (single-CPU) - but I have not been able to reproduce this yet, and it was likely not related to this specific patch - so I'm asking for review/commit at this point.

        Show
        John H. Embretsen added a comment - - edited Attached patch file d3385_jdbcMBeanTest_02.diff: M java/testing/org/apache/derbyTesting/functionTests/tests/management/MBeanTest.java M java/testing/org/apache/derbyTesting/functionTests/tests/management/JDBCMBeanTest.java M java/testing/org/apache/derbyTesting/junit/TestConfiguration.java M java/testing/org/apache/derbyTesting/junit/NetworkServerTestSetup.java M java/testing/org/apache/derbyTesting/junit/Utilities.java This patch modifies the JDBCMBeanTest so that the expected values for the MajorVersion and MinorVersion attributes are obtained from the embedded driver instead of the client driver. I wasn't sure how to do this, but I ended up decorating these two test fixtures a bit differently than the rest of the fixtures, by using a ServerSetup which JDBCClient is set to embedded. The patch adds some logic in TestConfiguration.java and MBeanTest.java for handling this. The patch also moves the two generic utility methods for fetching sysinfo output from MBeanTest.java to org.apache.derbyTesting.junit.Utilities.java. In addition, the patch revises some Javadoc comments in NetworkServerTestSetup.java. When testing this patch I once ran into timeout issues similar to those reported in DERBY-3515 when running the management Suite on WinXP (single-CPU) - but I have not been able to reproduce this yet, and it was likely not related to this specific patch - so I'm asking for review/commit at this point.
        Hide
        Daniel John Debrunner added a comment -

        Does this method make sense?

        MBeanTest.decorateWithRemoteJmxAndEmbeddedJDBC

        In this case Derby's MBean will be running against the embedded jdbc driver on the remote machine,
        but the junit test will be running against the embedded driver in the test vm.

        I guess they are the same but that is an assumption, it could change in the future, maybe any improvements could be addressed then.

        Show
        Daniel John Debrunner added a comment - Does this method make sense? MBeanTest.decorateWithRemoteJmxAndEmbeddedJDBC In this case Derby's MBean will be running against the embedded jdbc driver on the remote machine, but the junit test will be running against the embedded driver in the test vm. I guess they are the same but that is an assumption, it could change in the future, maybe any improvements could be addressed then.
        Hide
        Daniel John Debrunner added a comment -

        Another thought is just use a really simple approach instead of hiding all of this through decorators:

        Driver d = new org.apache.derby.jdbc.EmbeddedDriver();

        d.getMinorVersion();

        d.getMajorVersion();

        Sometimes simplicity & directness is better.

        Show
        Daniel John Debrunner added a comment - Another thought is just use a really simple approach instead of hiding all of this through decorators: Driver d = new org.apache.derby.jdbc.EmbeddedDriver(); d.getMinorVersion(); d.getMajorVersion(); Sometimes simplicity & directness is better.
        Hide
        John H. Embretsen added a comment -

        Ah, so I don't really need to use DatabaseMetaData.... Much better, I will create a new patch with this change (using the Driver interface directly).
        It still does not necessarily access the exact same embedded driver as the MBean implementation, but I have no idea how to access the embedded driver in a remote VM. It is better than using the client driver though, I guess.

        Show
        John H. Embretsen added a comment - Ah, so I don't really need to use DatabaseMetaData.... Much better, I will create a new patch with this change (using the Driver interface directly). It still does not necessarily access the exact same embedded driver as the MBean implementation, but I have no idea how to access the embedded driver in a remote VM. It is better than using the client driver though, I guess.
        Hide
        John H. Embretsen added a comment -

        Attached patch file d3385_jdbcMBeanTest_03.diff, to replace d3385_jdbcMBeanTest_02.diff:

        M java/testing/org/apache/derbyTesting/functionTests/tests/management/MBeanTest.java
        M java/testing/org/apache/derbyTesting/functionTests/tests/management/JDBCMBeanTest.java
        M java/testing/org/apache/derbyTesting/junit/NetworkServerTestSetup.java
        M java/testing/org/apache/derbyTesting/junit/Utilities.java

        This patch modifies the JDBCMBeanTest so that the expected values for the MajorVersion and MinorVersion attributes are obtained from the embedded driver instead of the client driver.

        The difference from patch 02 is that the expected values are retrieved directly from the embedded driver (assuming this is available in the classpath) instead of going through DatabaseMetaData and for that reason jumping through various hoops in order to get a JDBC connection using the embedded driver.

        As with patch 02, this patch also revises some javadoc comments in NetworkServerTestSetup, and moves the generic utility methods for getting sysinfo output, from MBeanTest to the Utilities class in org.apache.derbyTesting.junit.

        Show
        John H. Embretsen added a comment - Attached patch file d3385_jdbcMBeanTest_03.diff, to replace d3385_jdbcMBeanTest_02.diff: M java/testing/org/apache/derbyTesting/functionTests/tests/management/MBeanTest.java M java/testing/org/apache/derbyTesting/functionTests/tests/management/JDBCMBeanTest.java M java/testing/org/apache/derbyTesting/junit/NetworkServerTestSetup.java M java/testing/org/apache/derbyTesting/junit/Utilities.java This patch modifies the JDBCMBeanTest so that the expected values for the MajorVersion and MinorVersion attributes are obtained from the embedded driver instead of the client driver. The difference from patch 02 is that the expected values are retrieved directly from the embedded driver (assuming this is available in the classpath) instead of going through DatabaseMetaData and for that reason jumping through various hoops in order to get a JDBC connection using the embedded driver. As with patch 02, this patch also revises some javadoc comments in NetworkServerTestSetup, and moves the generic utility methods for getting sysinfo output, from MBeanTest to the Utilities class in org.apache.derbyTesting.junit.
        Hide
        Daniel John Debrunner added a comment -

        patch d3385_jdbcMBeanTest_03.diff applied revision 636417 - Thanks John.

        Show
        Daniel John Debrunner added a comment - patch d3385_jdbcMBeanTest_03.diff applied revision 636417 - Thanks John.
        Hide
        John H. Embretsen added a comment -

        Currently planned tests have been committed to trunk and 10.4 branch, so I am resolving this issue. Additional improvements and additions can be tracked by new Jira issues.

        Show
        John H. Embretsen added a comment - Currently planned tests have been committed to trunk and 10.4 branch, so I am resolving this issue. Additional improvements and additions can be tracked by new Jira issues.

          People

          • Assignee:
            John H. Embretsen
            Reporter:
            John H. Embretsen
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development