Jackrabbit Oak
  1. Jackrabbit Oak
  2. OAK-6

Setup integration tests and TCK tests

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.1
    • Component/s: it
    • Labels:

      Issue Links

        Activity

        Michael Dürig created issue -
        Michael Dürig made changes -
        Field Original Value New Value
        Link This issue depends on OAK-5 [ OAK-5 ]
        Hide
        Michael Dürig added a comment -

        There are also a lot more JCR tests in jackrabbit-jcr2spi [1] which we should leverage. At the very least they will give us a clearer picture on some of the differences wrt. to jr2.

        [1] http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit-jcr2spi/src/test/java/org/apache/jackrabbit/jcr2spi/

        Show
        Michael Dürig added a comment - There are also a lot more JCR tests in jackrabbit-jcr2spi [1] which we should leverage. At the very least they will give us a clearer picture on some of the differences wrt. to jr2. [1] http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit-jcr2spi/src/test/java/org/apache/jackrabbit/jcr2spi/
        Michael Dürig made changes -
        Labels test
        angela made changes -
        Component/s it [ 12317805 ]
        Hide
        Jukka Zitting added a comment -

        I added a basic JCR TCK setup to oak-it/jcr in revision 1306945.

        Show
        Jukka Zitting added a comment - I added a basic JCR TCK setup to oak-it/jcr in revision 1306945.
        Hide
        Jukka Zitting added a comment -

        Resolving as fixed for 0.1 with the oak-it-jcr component being in place. It still has all tests failing and marked as known issues, but we can deal with that in followup issues starting with 0.2.

        Show
        Jukka Zitting added a comment - Resolving as fixed for 0.1 with the oak-it-jcr component being in place. It still has all tests failing and marked as known issues, but we can deal with that in followup issues starting with 0.2.
        Jukka Zitting made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Alex Parvulescu made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Julian Reschke added a comment -

        TCK status 2012-04-25: Tests: 1905, Errors: 1471, Failures: 47

        Probably the main blockers: path/namespace handling, missing primary node type, missing jcr:primaryType property

        Show
        Julian Reschke added a comment - TCK status 2012-04-25: Tests: 1905, Errors: 1471, Failures: 47 Probably the main blockers: path/namespace handling, missing primary node type, missing jcr:primaryType property
        Hide
        Julian Reschke added a comment -

        TCK status 2012-04-30: Tests: 1905, Errors: 223, Failures: 541

        Show
        Julian Reschke added a comment - TCK status 2012-04-30: Tests: 1905, Errors: 223, Failures: 541
        Hide
        Julian Reschke added a comment -

        TCK status 2012-05-01: Tests: 1905, Errors: 220, Failures: 541

        Show
        Julian Reschke added a comment - TCK status 2012-05-01: Tests: 1905, Errors: 220, Failures: 541
        Hide
        Julian Reschke added a comment -

        TCK status 2012-05-02: Tests: 1905, Errors: 228, Failures: 525

        Show
        Julian Reschke added a comment - TCK status 2012-05-02: Tests: 1905, Errors: 228, Failures: 525
        Hide
        Julian Reschke added a comment -

        TCK status 2012-05-03: Tests: 1905, Errors: 227, Failures: 515

        Show
        Julian Reschke added a comment - TCK status 2012-05-03: Tests: 1905, Errors: 227, Failures: 515
        Hide
        Julian Reschke added a comment -

        TCK status 2012-05-04: Tests: 1905, Errors: 223, Failures: 501

        Show
        Julian Reschke added a comment - TCK status 2012-05-04: Tests: 1905, Errors: 223, Failures: 501
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 222, Failures: 499

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 222, Failures: 499
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 186, Failures: 156

        (after making sure that versioning and access control tests check the repository descriptors consistently)

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 186, Failures: 156 (after making sure that versioning and access control tests check the repository descriptors consistently)
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 172, Failures: 155

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 172, Failures: 155
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 171, Failures: 154

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 171, Failures: 154
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 45, Failures: 237

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 45, Failures: 237
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 65, Failures: 178

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 65, Failures: 178
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 57, Failures: 181

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 57, Failures: 181
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 52, Failures: 179

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 52, Failures: 179
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 41, Failures: 146

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 41, Failures: 146
        Julian Reschke committed 1341564 (1 file)
        Reviews: none

        OAK-6: update test exclusions

        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 36, Failures: 115

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 36, Failures: 115
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 32, Failures: 115

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 32, Failures: 115
        Julian Reschke committed 1342238 (1 file)
        Reviews: none

        OAK-6: update test exclusions

        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 30, Failures: 114

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 30, Failures: 114
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 28, Failures: 115

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 28, Failures: 115
        Julian Reschke committed 1344730 (1 file)
        Reviews: none

        OAK-6: update test exclusions

        Thomas Memenga committed 1344754 (1 file)
        Reviews: none

        OAK-6 update test exclusions

        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 27, Failures: 114

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 27, Failures: 114
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 27, Failures: 111

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 27, Failures: 111
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 25, Failures: 111

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 25, Failures: 111
        Julian Reschke committed 1346987 (1 file)
        Reviews: none

        OAK-6: update test exclusions

        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 24, Failures: 110

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 24, Failures: 110
        Michael Dürig committed 1350288 (1 file)
        Reviews: none

        OAK-6: update test exclusions (induced by OAK-133)

        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 24, Failures: 107

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 24, Failures: 107
        Michael Dürig committed 1354433 (1 file)
        Reviews: none

        OAK-144: Implement observation
        OAK-6: update test exclusions

        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1905, Errors: 25, Failures: 108 (with Observation temporarily disabled)

        Show
        Julian Reschke added a comment - TCK status: Tests: 1905, Errors: 25, Failures: 108 (with Observation temporarily disabled)
        Julian Reschke committed 1360208 (1 file)
        Reviews: none

        OAK-6: update exclusion list

        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 26, Failures: 119

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 26, Failures: 119
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 26, Failures: 118

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 26, Failures: 118
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 26, Failures: 121

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 26, Failures: 121
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 27, Failures: 118

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 27, Failures: 118
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 27, Failures: 121

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 27, Failures: 121
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 26, Failures: 114

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 26, Failures: 114
        Julian Reschke committed 1375984 (1 file)
        Reviews: none

        OAK-6: update test exclusions

        Julian Reschke committed 1376021 (1 file)
        Reviews: none

        OAK-6: fix typos (test case was renamed in TCK)

        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 19, Failures: 115

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 19, Failures: 115
        Julian Reschke committed 1376041 (1 file)
        Julian Reschke committed 1376044 (1 file)
        Show
        Michael Dürig added a comment - Here's a graph on our progress so far: https://docs.google.com/spreadsheet/pub?key=0AqWOLyLNdfKedFl2eXdEYmFLRnhNQWx5WUNtcU5hZXc&output=html
        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 18, Failures: 99

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 18, Failures: 99
        Julian Reschke committed 1376956 (1 file)
        Reviews: none

        OAK-6: update test exclusions

        Julian Reschke committed 1377248 (2 files)
        Reviews: none

        OAK-6: add more property definitions to test node types (work-in-progress)

        Hide
        Julian Reschke added a comment -

        TCK status: Tests: 1906, Errors: 18, Failures: 105

        Show
        Julian Reschke added a comment - TCK status: Tests: 1906, Errors: 18, Failures: 105
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 18, Failures: 103

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 18, Failures: 103
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 18, Failures: 97

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 18, Failures: 97
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 18, Failures: 96

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 18, Failures: 96
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 18, Failures: 90

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 18, Failures: 90
        Hide
        Michael Dürig added a comment -

        We need more test content since many tests seem to pass vacuously. That is, they throw an NotExecutableException due to missing content which counts them as 'passed'.

        Show
        Michael Dürig added a comment - We need more test content since many tests seem to pass vacuously. That is, they throw an NotExecutableException due to missing content which counts them as 'passed'.
        Michael Dürig committed 1384355 (1 file)
        Reviews: none

        OAK-6: Setup integration tests and TCK tests
         update test exclusions

        Hide
        Michael Dürig added a comment -

        In revision 1384354 I committed loading the test node types from a CND file. This results in a few additional test failures which is expected. I updated the test exclusions accordingly at revision 1384355.

        Show
        Michael Dürig added a comment - In revision 1384354 I committed loading the test node types from a CND file. This results in a few additional test failures which is expected. I updated the test exclusions accordingly at revision 1384355.
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 15, Failures: 95

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 15, Failures: 95
        Michael Dürig committed 1384382 (1 file)
        Reviews: none

        OAK-6: Setup integration tests and TCK tests
         set log level for TCK tests to debug in order to get "not executable" entries for test which throw a NotExecutableException

        Michael Dürig committed 1384383 (2 files)
        Reviews: none

        OAK-66: JCR Node Type Management
        update repository descriptors to reflect actual features
        OAK-6: Setup integration tests and TCK tests
        update test exclusions

        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 21, Failures: 101

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 21, Failures: 101
        Michael Dürig committed 1384406 (2 files)
        Reviews: none

        OAK-6: Setup integration tests and TCK tests
        tweak to stop tests throwing NotExecutableException due to mixin not supported on testnode/node1
        update test exclusions

        Hide
        Michael Dürig added a comment -

        At revision 1384406 I committed a tweak which results in a more honest reporting of test failures by reducing the number of tests which pass silently by throwing a NotExecutableException.

        The remaining tests which throw a NotExecutableException for reasons others than querying a repository descriptor for a not supported feature are:

        testVersionException(org.apache.jackrabbit.test.api.query.SaveTest) not executable: null
        testDocOrderIndexedNotation(org.apache.jackrabbit.test.api.query.XPathPosIndexTest) not executable: Repository does not support document order on result set.
        testDocOrderPositionFunction(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set.
        testDocOrderPositionIndex(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set.
        testDocOrderLastFunction(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set.
        testDocOrderFirstFunction(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set.
        testDerefSinglePropWithNodeTest(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query:
        testDerefSinglePropWithNodeStar(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query:
        testDerefMultiPropWithNodeTest(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query:
        testDerefMultiPropWithNodeStar(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query:
        testTextNodeTestWithPosition(org.apache.jackrabbit.test.api.query.TextNodeTest) not executable: Repository does not support position index
        testGetAttribute(org.apache.jackrabbit.test.api.SessionReadMethodsTest) not executable: No attributes set in this session.
        testReferenceableRootNode(org.apache.jackrabbit.test.api.ReferenceableRootNodesTest) not executable: Cannot compare uuid behaviour of different workspaces. Only a single workspace configured.
        testCheckedIn(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Versioning is not supported.
        testCheckedInUnlock(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Repository does not support versioning.
        testReorder(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Node type nt:unstructured does not support same-name-siblings
        testReorder2(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Node type nt:unstructured does not support same-name-siblings
        testUnlockSameNameSibling(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Node does not seem to allow same name siblings
        testReferenceProperty(org.apache.jackrabbit.test.api.SetPropertyConstraintViolationExceptionTest) not executable: Not able to create required test items.
        testValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items.
        testString(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items.
        testValues(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items.
        testValueAssumeTypeOfValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items.
        testNodeAssumeTypeOfValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items.
        testValuesAssumeTypeOfValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items.
        testRemoveSuccessfully(org.apache.jackrabbit.test.api.NodeRemoveMixinTest) not executable: Repository does not support remove of mixin.
        
        Show
        Michael Dürig added a comment - At revision 1384406 I committed a tweak which results in a more honest reporting of test failures by reducing the number of tests which pass silently by throwing a NotExecutableException . The remaining tests which throw a NotExecutableException for reasons others than querying a repository descriptor for a not supported feature are: testVersionException(org.apache.jackrabbit.test.api.query.SaveTest) not executable: null testDocOrderIndexedNotation(org.apache.jackrabbit.test.api.query.XPathPosIndexTest) not executable: Repository does not support document order on result set. testDocOrderPositionFunction(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set. testDocOrderPositionIndex(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set. testDocOrderLastFunction(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set. testDocOrderFirstFunction(org.apache.jackrabbit.test.api.query.XPathDocOrderTest) not executable: Repository does not support document order on result set. testDerefSinglePropWithNodeTest(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query: testDerefSinglePropWithNodeStar(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query: testDerefMultiPropWithNodeTest(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query: testDerefMultiPropWithNodeStar(org.apache.jackrabbit.test.api.query.DerefQueryLevel1Test) not executable: java.text.ParseException: Query: testTextNodeTestWithPosition(org.apache.jackrabbit.test.api.query.TextNodeTest) not executable: Repository does not support position index testGetAttribute(org.apache.jackrabbit.test.api.SessionReadMethodsTest) not executable: No attributes set in this session. testReferenceableRootNode(org.apache.jackrabbit.test.api.ReferenceableRootNodesTest) not executable: Cannot compare uuid behaviour of different workspaces. Only a single workspace configured. testCheckedIn(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Versioning is not supported. testCheckedInUnlock(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Repository does not support versioning. testReorder(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Node type nt:unstructured does not support same-name-siblings testReorder2(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Node type nt:unstructured does not support same-name-siblings testUnlockSameNameSibling(org.apache.jackrabbit.test.api.lock.LockTest) not executable: Node does not seem to allow same name siblings testReferenceProperty(org.apache.jackrabbit.test.api.SetPropertyConstraintViolationExceptionTest) not executable: Not able to create required test items. testValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items. testString(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items. testValues(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items. testValueAssumeTypeOfValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items. testNodeAssumeTypeOfValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items. testValuesAssumeTypeOfValue(org.apache.jackrabbit.test.api.SetPropertyAssumeTypeTest) not executable: Not able to set up test items. testRemoveSuccessfully(org.apache.jackrabbit.test.api.NodeRemoveMixinTest) not executable: Repository does not support remove of mixin.
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 200, Failures: 65

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 200, Failures: 65
        Hide
        Michael Dürig added a comment -

        TCK status: Tests: 1906, Errors: 196, Failures: 55

        Show
        Michael Dürig added a comment - TCK status: Tests: 1906, Errors: 196, Failures: 55
        Gavin made changes -
        Link This issue depends on OAK-5 [ OAK-5 ]
        Gavin made changes -
        Link This issue depends upon OAK-5 [ OAK-5 ]
        Show more

          People

          • Assignee:
            Unassigned
            Reporter:
            Michael Dürig
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development