Jackrabbit Oak
  1. Jackrabbit Oak
  2. OAK-6

Setup integration tests and TCK tests

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.1
    • Component/s: it
    • Labels:

      Issue Links

        Activity

        Hide
        Michael Dürig added a comment -

        There are also a lot more JCR tests in jackrabbit-jcr2spi [1] which we should leverage. At the very least they will give us a clearer picture on some of the differences wrt. to jr2.

        [1] http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit-jcr2spi/src/test/java/org/apache/jackrabbit/jcr2spi/

        Show
        Michael Dürig added a comment - There are also a lot more JCR tests in jackrabbit-jcr2spi [1] which we should leverage. At the very least they will give us a clearer picture on some of the differences wrt. to jr2. [1] http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit-jcr2spi/src/test/java/org/apache/jackrabbit/jcr2spi/
        Hide
        Jukka Zitting added a comment -

        I added a basic JCR TCK setup to oak-it/jcr in revision 1306945.

        Show
        Jukka Zitting added a comment - I added a basic JCR TCK setup to oak-it/jcr in revision 1306945.
        Hide
        Jukka Zitting added a comment -

        Resolving as fixed for 0.1 with the oak-it-jcr component being in place. It still has all tests failing and marked as known issues, but we can deal with that in followup issues starting with 0.2.

        Show
        Jukka Zitting added a comment - Resolving as fixed for 0.1 with the oak-it-jcr component being in place. It still has all tests failing and marked as known issues, but we can deal with that in followup issues starting with 0.2.
        Hide
        Julian Reschke added a comment -

        TCK status 2012-04-25: Tests: 1905, Errors: 1471, Failures: 47

        Probably the main blockers: path/namespace handling, missing primary node type, missing jcr:primaryType property

        Show
        Julian Reschke added a comment - TCK status 2012-04-25: Tests: 1905, Errors: 1471, Failures: 47 Probably the main blockers: path/namespace handling, missing primary node type, missing jcr:primaryType property
        Hide
        Julian Reschke added a comment -

        TCK status 2012-04-30: Tests: 1905, Errors: 223, Failures: 541

        Show
        Julian Reschke added a comment - TCK status 2012-04-30: Tests: 1905, Errors: 223, Failures: 541
        Hide
        Julian Reschke added a comment -

        TCK status 2012-05-01: Tests: 1905, Errors: 220, Failures: 541

        Show
        Julian Reschke added a comment - TCK status 2012-05-01: Tests: 1905, Errors: 220, Failures: 541
        Hide
        Julian Reschke added a comment -

        TCK status 2012-05-02: Tests: 1905, Errors: 228, Failures: 525

        Show
        Julian Reschke added a comment - TCK status 2012-05-02: Tests: 1905, Errors: 228, Failures: 525
        Hide
        Julian Reschke added a comment -

        TCK status 2012-05-03: Tests: 1905, Errors: 227, Failures: 515

        Show
        Julian Reschke added a comment - TCK status 2012-05-03: Tests: 1905, Errors: 227, Failures: 515
        Hide
        Julian Reschke added a comment -

        TCK status 2012-05-04: Tests: 1905, Errors: 223, Failures: 501

        Show
        Julian Reschke added a comment - TCK status 2012-05-04: Tests: 1905, Errors: 223, Failures: 501
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 222, Failures: 499

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 222, Failures: 499
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 186, Failures: 156

        (after making sure that versioning and access control tests check the repository descriptors consistently)

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 186, Failures: 156 (after making sure that versioning and access control tests check the repository descriptors consistently)
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 172, Failures: 155

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 172, Failures: 155
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 171, Failures: 154

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 171, Failures: 154
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 45, Failures: 237

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 45, Failures: 237
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 65, Failures: 178

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 65, Failures: 178
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 57, Failures: 181

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 57, Failures: 181
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 52, Failures: 179

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 52, Failures: 179
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 41, Failures: 146

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 41, Failures: 146
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 36, Failures: 115

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 36, Failures: 115
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 32, Failures: 115

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 32, Failures: 115
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 30, Failures: 114

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 30, Failures: 114
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 28, Failures: 115

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 28, Failures: 115
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 27, Failures: 114

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 27, Failures: 114
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 27, Failures: 111

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 27, Failures: 111
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 25, Failures: 111

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 25, Failures: 111
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 24, Failures: 110

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 24, Failures: 110
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 24, Failures: 107

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 24, Failures: 107
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 25, Failures: 108 (with Observation temporarily disabled)

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 25, Failures: 108 (with Observation temporarily disabled)
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 26, Failures: 119

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 26, Failures: 119
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 26, Failures: 118

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 26, Failures: 118
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 26, Failures: 121

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 26, Failures: 121
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 27, Failures: 118

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 27, Failures: 118
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 27, Failures: 121

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 27, Failures: 121
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 26, Failures: 114

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 26, Failures: 114
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 19, Failures: 115

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 19, Failures: 115
        Show
        Michael Dürig added a comment - Here's a graph on our progress so far: https://docs.google.com/spreadsheet/pub?key=0AqWOLyLNdfKedFl2eXdEYmFLRnhNQWx5WUNtcU5hZXc&output=html
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 18, Failures: 99

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 18, Failures: 99
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 18, Failures: 105

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 18, Failures: 105
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 18, Failures: 103

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 18, Failures: 103
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 18, Failures: 97

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 18, Failures: 97
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 18, Failures: 96

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 18, Failures: 96
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 18, Failures: 90

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 18, Failures: 90
        Hide
        Michael Dürig added a comment -

        We need more test content since many tests seem to pass vacuously. That is, they throw an NotExecutableException due to missing content which counts them as 'passed'.

        Show
        Michael Dürig added a comment - We need more test content since many tests seem to pass vacuously. That is, they throw an NotExecutableException due to missing content which counts them as 'passed'.
        Hide
        Michael Dürig added a comment -

        In revision 1384354 I committed loading the test node types from a CND file. This results in a few additional test failures which is expected. I updated the test exclusions accordingly at revision 1384355.

        Show
        Michael Dürig added a comment - In revision 1384354 I committed loading the test node types from a CND file. This results in a few additional test failures which is expected. I updated the test exclusions accordingly at revision 1384355.
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 15, Failures: 95

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 15, Failures: 95
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 21, Failures: 101

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 21, Failures: 101
        Hide
        Michael Dürig added a comment -

        At revision 1384406 I committed a tweak which results in a more honest reporting of test failures by reducing the number of tests which pass silently by throwing a NotExecutableException.

        The remaining tests which throw a NotExecutableException for reasons others than querying a repository descriptor for a not supported feature are:

        testVersionException(org.apache.jackrabbit.test.api.query.SaveTest) not executable: null
        testDocOrderIndexedNotation(org.apache.jackrabbit.test.api.query.XPathPosIndexTest) not executable: Repository does not support document order on result set.
        testDocOrderPositionFunction(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set.
        testDocOrderPositionIndex(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set.
        testDocOrderLastFunction(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set.
        testDocOrderFirstFunction(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set.
        testDerefSinglePropWithNodeTest(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query:
        testDerefSinglePropWithNodeStar(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query:
        testDerefMultiPropWithNodeTest(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query:
        testDerefMultiPropWithNodeStar(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query:
        testTextNodeTestWithPosition(org.apache.jackrabbit.test.api.query.TextNodeTest) not executable: Repository does not support position index
        testGetAttribute(org.apache.jackrabbit.test.api.SessionReadMethodsTest) not executable: No attributes set in this session.
        testReferenceableRootNode(org.apache.jackrabbit.test.api.ReferenceableRootNodesTest) not executable: Cannot compare uuid behaviour of different workspaces. Only a single workspace configured.
        testCheckedIn(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Versioning is not supported.
        testCheckedInUnlock(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Repository does not support versioning.
        testReorder(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Node type nt:unstructured does not support same-name-siblings
        testReorder2(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Node type nt:unstructured does not support same-name-siblings
        testUnlockSameNameSibling(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Node does not seem to allow same name siblings
        testReferenceProperty(org.apache.jackrabbit.test.api.SetPropertyConstraintViolationExceptionTest) not executable: Not able to create required test items.
        testValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items.
        testString(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items.
        testValues(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items.
        testValueAssumeTypeOfValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items.
        testNodeAssumeTypeOfValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items.
        testValuesAssumeTypeOfValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items.
        testRemoveSuccessfully(org.apache.jackrabbit.test.api.NodeRemoveMixinTest) not executable: Repository does not support remove of mixin.
        
        Show
        Michael Dürig added a comment - At revision 1384406 I committed a tweak which results in a more honest reporting of test failures by reducing the number of tests which pass silently by throwing a NotExecutableException . The remaining tests which throw a NotExecutableException for reasons others than querying a repository descriptor for a not supported feature are: testVersionException(org.apache.jackrabbit.test.api.query.SaveTest) not executable: null testDocOrderIndexedNotation(org.apache.jackrabbit.test.api.query.XPathPosIndexTest) not executable: Repository does not support document order on result set. testDocOrderPositionFunction(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set. testDocOrderPositionIndex(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set. testDocOrderLastFunction(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set. testDocOrderFirstFunction(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set. testDerefSinglePropWithNodeTest(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query: testDerefSinglePropWithNodeStar(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query: testDerefMultiPropWithNodeTest(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query: testDerefMultiPropWithNodeStar(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query: testTextNodeTestWithPosition(org.apache.jackrabbit.test.api.query.TextNodeTest) not executable: Repository does not support position index testGetAttribute(org.apache.jackrabbit.test.api.SessionReadMethodsTest) not executable: No attributes set in this session. testReferenceableRootNode(org.apache.jackrabbit.test.api.ReferenceableRootNodesTest) not executable: Cannot compare uuid behaviour of different workspaces. Only a single workspace configured. testCheckedIn(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Versioning is not supported. testCheckedInUnlock(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Repository does not support versioning. testReorder(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Node type nt:unstructured does not support same-name-siblings testReorder2(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Node type nt:unstructured does not support same-name-siblings testUnlockSameNameSibling(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Node does not seem to allow same name siblings testReferenceProperty(org.apache.jackrabbit.test.api.SetPropertyConstraintViolationExceptionTest) not executable: Not able to create required test items. testValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items. testString(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items. testValues(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items. testValueAssumeTypeOfValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items. testNodeAssumeTypeOfValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items. testValuesAssumeTypeOfValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items. testRemoveSuccessfully(org.apache.jackrabbit.test.api.NodeRemoveMixinTest) not executable: Repository does not support remove of mixin.
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 200, Failures: 65

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 200, Failures: 65
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 196, Failures: 55

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 196, Failures: 55

          People

          • Assignee:
            Unassigned
            Reporter:
            Michael Dürig
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development