JDO
  1. JDO
  2. JDO-299

Signature test needed checking compliance of implementations with JDO2.0 API

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: JDO 2 final (2.0)
    • Component/s: tck
    • Labels:
      None
    1. api20_defs.txt
      41 kB
      Martin Zaun
    2. JUnitSignatureTest.java
      2 kB
      Craig L Russell
    3. signature_test_final_review.zip
      20 kB
      Martin Zaun
    4. SignatureTest.java
      56 kB
      Martin Zaun
    5. sigtest.patch
      1 kB
      Craig L Russell

      Activity

      Hide
      Martin Zaun added a comment -

      This test uses reflection to verify that a set of JDO2.0 API classes conforms with the specification.

      Since the number of features to be tested is quite large (~690 interfaces, classes, methods, or constants),
      the feature definitions are not hard-coded into the test but parsed from a text file – see attached file.

      The format of the feature definition file is simple to parse, read, and maintain: basically, it's a stripped
      version of the JDO2.0 Java interface and class sources.

      Show
      Martin Zaun added a comment - This test uses reflection to verify that a set of JDO2.0 API classes conforms with the specification. Since the number of features to be tested is quite large (~690 interfaces, classes, methods, or constants), the feature definitions are not hard-coded into the test but parsed from a text file – see attached file. The format of the feature definition file is simple to parse, read, and maintain: basically, it's a stripped version of the JDO2.0 Java interface and class sources.
      Hide
      Craig L Russell added a comment -

      DataStoreCache ok.
      JDOConnection ok.
      Extent ok.
      FetchPlan should verify the values of the static fields.
      Do you intend to test the "implements" clause of the interfaces?
      In general, I think we should not verify protected or private methods.
      XXXIdentity ok
      Reviewed up to but not including InstanceCallbacks.

      Show
      Craig L Russell added a comment - DataStoreCache ok. JDOConnection ok. Extent ok. FetchPlan should verify the values of the static fields. Do you intend to test the "implements" clause of the interfaces? In general, I think we should not verify protected or private methods. XXXIdentity ok Reviewed up to but not including InstanceCallbacks.
      Hide
      Martin Zaun added a comment -

      attached is an updated version of the API signature descriptor file.

      > FetchPlan should verify the values of the static fields.

      I updated the descriptor file for these constant field values:

      public interface javax.jdo.FetchPlan:
      public static final String DEFAULT = "default";
      public static final String ALL = "all";
      public static final int DETACH_UNLOAD_FIELDS = 2;
      public static final int DETACH_LOAD_FIELDS = 1;
      public static final int FETCH_SIZE_GREEDY = -1;
      public static final int FETCH_SIZE_OPTIMAL = 0;

      public class javax.jdo.listener.InstanceLifecycleEvent:
      public static final int CREATE = 0;
      public static final int LOAD = 1;
      public static final int STORE = 2;
      public static final int CLEAR = 3;
      public static final int DELETE = 4;
      public static final int DIRTY = 5;
      public static final int DETACH = 6;
      public static final int ATTACH = 7;

      public interface javax.jdo.Query:
      public static final String JDOQL = "javax.jdo.query.JDOQL";
      public static final String SQL = "javax.jdo.query.SQL";

      public interface javax.jdo.spi.PersistenceCapable:
      public static final byte READ_WRITE_OK = 0;
      public static final byte LOAD_REQUIRED = 1;
      public static final byte READ_OK = -1;
      public static final byte CHECK_READ = 1;
      public static final byte MEDIATE_READ = 2;
      public static final byte CHECK_WRITE = 4;
      public static final byte MEDIATE_WRITE = 8;
      public static final byte SERIALIZABLE = 16;

      There are 4 public static fields in JDOPermission for which I cannot (easily)
      add constant definitions (not literals). But perhaps they don't belong into
      the descriptor file anyway (they're not mentioned in the spec):

      public final class javax.jdo.spi.JDOPermission:
      public static final javax.jdo.spi.JDOPermission GET_METADATA;
      public static final javax.jdo.spi.JDOPermission MANAGE_METADATA;
      public static final javax.jdo.spi.JDOPermission SET_STATE_MANAGER;
      public static final javax.jdo.spi.JDOPermission CLOSE_PERSISTENCE_MANAGER_FACTORY;

      > Do you intend to test the "implements" clause of the interfaces?

      I'm testing the 'extends' clause for interfaces (they cannot have 'implements',
      which is also tested).

      > In general, I think we should not verify protected or private methods.

      I've removed all protected declarations from the descriptor file.

      > Reviewed up to but not including InstanceCallbacks.

      Thx, so far.

      Show
      Martin Zaun added a comment - attached is an updated version of the API signature descriptor file. > FetchPlan should verify the values of the static fields. I updated the descriptor file for these constant field values: public interface javax.jdo.FetchPlan: public static final String DEFAULT = "default"; public static final String ALL = "all"; public static final int DETACH_UNLOAD_FIELDS = 2; public static final int DETACH_LOAD_FIELDS = 1; public static final int FETCH_SIZE_GREEDY = -1; public static final int FETCH_SIZE_OPTIMAL = 0; public class javax.jdo.listener.InstanceLifecycleEvent: public static final int CREATE = 0; public static final int LOAD = 1; public static final int STORE = 2; public static final int CLEAR = 3; public static final int DELETE = 4; public static final int DIRTY = 5; public static final int DETACH = 6; public static final int ATTACH = 7; public interface javax.jdo.Query: public static final String JDOQL = "javax.jdo.query.JDOQL"; public static final String SQL = "javax.jdo.query.SQL"; public interface javax.jdo.spi.PersistenceCapable: public static final byte READ_WRITE_OK = 0; public static final byte LOAD_REQUIRED = 1; public static final byte READ_OK = -1; public static final byte CHECK_READ = 1; public static final byte MEDIATE_READ = 2; public static final byte CHECK_WRITE = 4; public static final byte MEDIATE_WRITE = 8; public static final byte SERIALIZABLE = 16; There are 4 public static fields in JDOPermission for which I cannot (easily) add constant definitions (not literals). But perhaps they don't belong into the descriptor file anyway (they're not mentioned in the spec): public final class javax.jdo.spi.JDOPermission: public static final javax.jdo.spi.JDOPermission GET_METADATA; public static final javax.jdo.spi.JDOPermission MANAGE_METADATA; public static final javax.jdo.spi.JDOPermission SET_STATE_MANAGER; public static final javax.jdo.spi.JDOPermission CLOSE_PERSISTENCE_MANAGER_FACTORY; > Do you intend to test the "implements" clause of the interfaces? I'm testing the 'extends' clause for interfaces (they cannot have 'implements', which is also tested). > In general, I think we should not verify protected or private methods. I've removed all protected declarations from the descriptor file. > Reviewed up to but not including InstanceCallbacks. Thx, so far.
      Hide
      Martin Zaun added a comment -

      Attached is a fully testable version of the Signature Test for review/comments.

      Except for the final integration with the TCK, I consider the version feature-complete:

      • testing of superclasses, interfaces, parameters, exceptions, modifers
      • testing public non-standard features found in the classes but not in the descriptor
      • testing constant field values (static public final fields only, as in interfaces)
      • support for (Java) user types name (opposed to reflection names) in the descriptor
      • reports as many errors as possible (except for parse errors in the descriptor file)
      • detailed reporting of missings, mismatches, loading problems
      • reports summary statistics, current results:

      Signature Test Results
      ======================

      tested features: 671

      Successes:
      matching features: 671
      loadable classes: 91

      Failures:
      missing features: 0
      non-matching features: 0
      non-standard features: 3
      unloadable classes: 0
      other problems: 0

      Here's how to test. Save SignatureTest.java into
      tck20/src/java/org/apache/jdo/tck/api
      (or let me know of a better place). Should build with the rest.

      Invoke the test with 'java' by it's (nested) CLI class:
      org.apache.jdo.tck.api.SignatureTest\$CLI

      The classpath needs to include
      tck20/target/classes
      api20/target/jdo2-api-SNAPSHOT.jar
      j2ee.jar

      Without the j2ee.jar, there are failures reported on javax.jdo.Transaction,
      which uses javax.transaction.Synchronization.

      Provide as command-line argument a path to the downloaded signature descriptor
      file:
      api20_defs.txt

      Any progress/verbose output is send to stdout while any failures/problems are being
      sent to stderr. So, you can save/look just the failure reports to a file: java ... 2> err.txt

      Try changing the descriptor and check the result statistics and failure logs.

      On the implementation, the code is organized into 5 classes, all in SignatureTest.java.
      Not much redundancy, still ~1500 lines of code. if you have an idea on how to simplify,
      let me know

      Martin

      Show
      Martin Zaun added a comment - Attached is a fully testable version of the Signature Test for review/comments. Except for the final integration with the TCK, I consider the version feature-complete: testing of superclasses, interfaces, parameters, exceptions, modifers testing public non-standard features found in the classes but not in the descriptor testing constant field values (static public final fields only, as in interfaces) support for (Java) user types name (opposed to reflection names) in the descriptor reports as many errors as possible (except for parse errors in the descriptor file) detailed reporting of missings, mismatches, loading problems reports summary statistics, current results: Signature Test Results ====================== tested features: 671 Successes: matching features: 671 loadable classes: 91 Failures: missing features: 0 non-matching features: 0 non-standard features: 3 unloadable classes: 0 other problems: 0 Here's how to test. Save SignatureTest.java into tck20/src/java/org/apache/jdo/tck/api (or let me know of a better place). Should build with the rest. Invoke the test with 'java' by it's (nested) CLI class: org.apache.jdo.tck.api.SignatureTest\$CLI The classpath needs to include tck20/target/classes api20/target/jdo2-api-SNAPSHOT.jar j2ee.jar Without the j2ee.jar, there are failures reported on javax.jdo.Transaction, which uses javax.transaction.Synchronization. Provide as command-line argument a path to the downloaded signature descriptor file: api20_defs.txt Any progress/verbose output is send to stdout while any failures/problems are being sent to stderr. So, you can save/look just the failure reports to a file: java ... 2> err.txt Try changing the descriptor and check the result statistics and failure logs. On the implementation, the code is organized into 5 classes, all in SignatureTest.java. Not much redundancy, still ~1500 lines of code. if you have an idea on how to simplify, let me know Martin
      Hide
      Craig L Russell added a comment -

      Looks good. A few comments.

      1. The structure of the test classes is nonstandard. The classes in the file should either become members of the SignatureTest class or put into their own .java files.

      2. The package structure of the classes needs to be improved. Should these classes all be put into org.apache.jdo.tck.util perhaps?

      3. There should be a class that extends JDO_Test that internally calls the main method of the signature test. This could be a simple test class that adheres to the pattern for the tck tests. I've attached a sample test that you can use as a template.

      4. The test needs to be integrated into the maven tck test framework. You will need to update the maven.xml and project.properties to implement a maven system property like "jdo.tck.signature.testdata" that is defined to be $

      {basedir}

      /src/conf/api20-signature.txt. This can then be retrieved by the JUnit test program.

      5. The output should be split so only errors are shown. Maybe a -q quiet option and no output at all unless there is an error. Or a -v verbose option that prints everything. Need to have a way for the output to be returned to the calling program so it can be printed out as the test case exception message. This would be just the -q output.

      6. When combining the verbose output and the error output, there is a threading problem. Maybe instead of using the system output and system error, you can use parameter arguments with file names that direct output to the logs directory.

      Show
      Craig L Russell added a comment - Looks good. A few comments. 1. The structure of the test classes is nonstandard. The classes in the file should either become members of the SignatureTest class or put into their own .java files. 2. The package structure of the classes needs to be improved. Should these classes all be put into org.apache.jdo.tck.util perhaps? 3. There should be a class that extends JDO_Test that internally calls the main method of the signature test. This could be a simple test class that adheres to the pattern for the tck tests. I've attached a sample test that you can use as a template. 4. The test needs to be integrated into the maven tck test framework. You will need to update the maven.xml and project.properties to implement a maven system property like "jdo.tck.signature.testdata" that is defined to be $ {basedir} /src/conf/api20-signature.txt. This can then be retrieved by the JUnit test program. 5. The output should be split so only errors are shown. Maybe a -q quiet option and no output at all unless there is an error. Or a -v verbose option that prints everything. Need to have a way for the output to be returned to the calling program so it can be printed out as the test case exception message. This would be just the -q output. 6. When combining the verbose output and the error output, there is a threading problem. Maybe instead of using the system output and system error, you can use parameter arguments with file names that direct output to the logs directory.
      Hide
      Craig L Russell added a comment -

      This is a simple class that fits into the jdo tck test framework and simply calls the CLI SignatureTest interface.

      Also see the patch for some non-optimal but sample changes to JDO_Test, maven.xml.

      Show
      Craig L Russell added a comment - This is a simple class that fits into the jdo tck test framework and simply calls the CLI SignatureTest interface. Also see the patch for some non-optimal but sample changes to JDO_Test, maven.xml.
      Hide
      Martin Zaun added a comment -

      Attached is a bundle signature_test_final_review.zip for review before checkin.
      It contains the files:
      maven.xml.patch
      project.properties.patch
      src/conf/signatures.conf
      src/conf/jdo-2_0-signatures.txt
      src/java/org/apache/jdo/tck/api/SignatureTest.java
      src/java/org/apache/jdo/tck/util/signature/Formatter.java
      src/java/org/apache/jdo/tck/util/signature/SignatureVerifier.java
      src/java/org/apache/jdo/tck/util/signature/TypeHelper.java

      Addressed are comments 1-6 from Craig:

      • added integration with TCK
      • refactored code into directories: ...tck/api and .../tck/utils/signature
      • removed separate stderr stream but introduced options: -q and -v

      A minor nuisance is that the test is run twice (datastore+application identity).

      Currently the test failes with 3 public non-standard features reported.

      Show
      Martin Zaun added a comment - Attached is a bundle signature_test_final_review.zip for review before checkin. It contains the files: maven.xml.patch project.properties.patch src/conf/signatures.conf src/conf/jdo-2_0-signatures.txt src/java/org/apache/jdo/tck/api/SignatureTest.java src/java/org/apache/jdo/tck/util/signature/Formatter.java src/java/org/apache/jdo/tck/util/signature/SignatureVerifier.java src/java/org/apache/jdo/tck/util/signature/TypeHelper.java Addressed are comments 1-6 from Craig: added integration with TCK refactored code into directories: ...tck/api and .../tck/utils/signature removed separate stderr stream but introduced options: -q and -v A minor nuisance is that the test is run twice (datastore+application identity). Currently the test failes with 3 public non-standard features reported.
      Hide
      Craig L Russell added a comment -

      The patch contains the maven.xml patch twice. I assume that the project.properties actually sets the value of the signaturetest file?

      The signature.conf has the wrong description.

      Is there a way to only run the signature test once, since it's no value to run with both app identity and ds identitiy.

      SignatureTest comment need not refer to JDO 2.0, as it's generic. When we update the spec to e.g. 2.0.1 or 2.1, we won't need to change this class, but just update the reference signature file.

      You might define an additional SignatureTest constructor that has an explicit classloader argument. If not specified, the default is the classloader that you define here: classLoader = this.getClass().getClassLoader(). Then the comment "Tests the signature of classes (in the classpath) " would change to "Tests the signature of classes (in the specified classloader) ".

      After you check this in, please file a JIRA on the three exceptions.

      Show
      Craig L Russell added a comment - The patch contains the maven.xml patch twice. I assume that the project.properties actually sets the value of the signaturetest file? The signature.conf has the wrong description. Is there a way to only run the signature test once, since it's no value to run with both app identity and ds identitiy. SignatureTest comment need not refer to JDO 2.0, as it's generic. When we update the spec to e.g. 2.0.1 or 2.1, we won't need to change this class, but just update the reference signature file. You might define an additional SignatureTest constructor that has an explicit classloader argument. If not specified, the default is the classloader that you define here: classLoader = this.getClass().getClassLoader(). Then the comment "Tests the signature of classes (in the classpath) " would change to "Tests the signature of classes (in the specified classloader) ". After you check this in, please file a JIRA on the three exceptions.
      Hide
      Martin Zaun added a comment -

      Updated the bundle for Craig's comments.

      Show
      Martin Zaun added a comment - Updated the bundle for Craig's comments.
      Hide
      Craig L Russell added a comment -

      Looks good. Go ahead and check it in.

      Show
      Craig L Russell added a comment - Looks good. Go ahead and check it in.
      Hide
      Martin Zaun added a comment -

      Created issue JDO-341 for fixing the found discrepancies between: JDO2.0 spec v. API20 v. SignatureTest's descriptor file.

      Show
      Martin Zaun added a comment - Created issue JDO-341 for fixing the found discrepancies between: JDO2.0 spec v. API20 v. SignatureTest's descriptor file.
      Hide
      Martin Zaun added a comment -

      Committed the signature test with revision 386448.

      Show
      Martin Zaun added a comment - Committed the signature test with revision 386448.
      Hide
      Martin Zaun added a comment -

      I noticed that the SignatureTest does not get run as part of the maven runtck.jdori goal.

      So, I've added "signatures.conf" to the 'jdo.tck.cfglist' property in src/conf/configurations.list.

      An unresolved, remaining nuisance is that the test get's run twice (app/datastore identity), but
      we don't know of any solution to this.

      Unless there are any objections, I'm committing the following patch tomorrow.
      Martin

      $ svn diff src/conf/configurations.list
      Index: src/conf/configurations.list
      ===================================================================
      — src/conf/configurations.list (revision 386483)
      +++ src/conf/configurations.list (working copy)
      @@ -26,4 +26,5 @@
      inheritance4.conf \
      schemaAttributeClass.conf \
      schemaAttributeOrm.conf \

      • schemaAttributePackage.conf
        + schemaAttributePackage.conf \
        + signatures.conf
      Show
      Martin Zaun added a comment - I noticed that the SignatureTest does not get run as part of the maven runtck.jdori goal. So, I've added "signatures.conf" to the 'jdo.tck.cfglist' property in src/conf/configurations.list. An unresolved, remaining nuisance is that the test get's run twice (app/datastore identity), but we don't know of any solution to this. Unless there are any objections, I'm committing the following patch tomorrow. Martin $ svn diff src/conf/configurations.list Index: src/conf/configurations.list =================================================================== — src/conf/configurations.list (revision 386483) +++ src/conf/configurations.list (working copy) @@ -26,4 +26,5 @@ inheritance4.conf \ schemaAttributeClass.conf \ schemaAttributeOrm.conf \ schemaAttributePackage.conf + schemaAttributePackage.conf \ + signatures.conf
      Hide
      Craig L Russell added a comment -

      The maven.xml was updated so that tests in the runonce.conf are only run once per maven run, not once per identitytype and database. The configurations.list file was updated to include runonce.conf.

      svn commit -m "JDO-299 Changed maven.xml to run signature test only once" maven.xml src/conf/runonce.conf src/conf/configurations.list src/conf/signatures.conf
      Sending maven.xml
      Sending src/conf/configurations.list
      Adding src/conf/runonce.conf
      Deleting src/conf/signatures.conf
      Transmitting file data ...
      Committed revision 386743.

      Show
      Craig L Russell added a comment - The maven.xml was updated so that tests in the runonce.conf are only run once per maven run, not once per identitytype and database. The configurations.list file was updated to include runonce.conf. svn commit -m " JDO-299 Changed maven.xml to run signature test only once" maven.xml src/conf/runonce.conf src/conf/configurations.list src/conf/signatures.conf Sending maven.xml Sending src/conf/configurations.list Adding src/conf/runonce.conf Deleting src/conf/signatures.conf Transmitting file data ... Committed revision 386743.
      Hide
      Craig L Russell added a comment -

      The maven.xml was updated so that tests in the runonce.conf are only run once per maven run, not once per identitytype and database. The configurations.list file was updated to include runonce.conf.

      svn commit -m "JDO-299 Changed maven.xml to run signature test only once" maven.xml src/conf/runonce.conf src/conf/configurations.list src/conf/signatures.conf
      Sending maven.xml
      Sending src/conf/configurations.list
      Adding src/conf/runonce.conf
      Deleting src/conf/signatures.conf
      Transmitting file data ...
      Committed revision 386743.

      Show
      Craig L Russell added a comment - The maven.xml was updated so that tests in the runonce.conf are only run once per maven run, not once per identitytype and database. The configurations.list file was updated to include runonce.conf. svn commit -m " JDO-299 Changed maven.xml to run signature test only once" maven.xml src/conf/runonce.conf src/conf/configurations.list src/conf/signatures.conf Sending maven.xml Sending src/conf/configurations.list Adding src/conf/runonce.conf Deleting src/conf/signatures.conf Transmitting file data ... Committed revision 386743.

        People

        • Assignee:
          Martin Zaun
          Reporter:
          Martin Zaun
        • Votes:
          0 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development