ManifoldCF
  1. ManifoldCF
  2. CONNECTORS-236

Tests and test server needed for CMIS connector

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: ManifoldCF 0.3
    • Component/s: CMIS connector
    • Labels:
      None

      Description

      The CMIS connector needs tests, and a CMIS test server to run against.

      1. chemistry-opencmis-server-inmemory-war-0.5.0-SNAPSHOT.war
        5.56 MB
        Piergiorgio Lucidi
      2. patch.txt
        39 kB
        Piergiorgio Lucidi
      3. patch2.txt
        8 kB
        Piergiorgio Lucidi
      4. patch3.txt
        5 kB
        Piergiorgio Lucidi
      5. patch4.txt
        19 kB
        Piergiorgio Lucidi

        Activity

        Hide
        Karl Wright added a comment -

        r1154789.

        Show
        Karl Wright added a comment - r1154789.
        Hide
        Karl Wright added a comment -

        Merged the CONNECTORS-236 branch into trunk. r1154315.

        Show
        Karl Wright added a comment - Merged the CONNECTORS-236 branch into trunk. r1154315.
        Hide
        Karl Wright added a comment -

        Patch 4 committed to the branch.

        Show
        Karl Wright added a comment - Patch 4 committed to the branch.
        Hide
        Piergiorgio Lucidi added a comment -

        This patch must be applied on the branch and contains the unit tests and integration tests for the CMIS Repository Connector - version fixed

        Show
        Piergiorgio Lucidi added a comment - This patch must be applied on the branch and contains the unit tests and integration tests for the CMIS Repository Connector - version fixed
        Hide
        Piergiorgio Lucidi added a comment -

        This patch must be applied on the branch and contains the unit tests and integration tests for the CMIS Repository Connector.

        Show
        Piergiorgio Lucidi added a comment - This patch must be applied on the branch and contains the unit tests and integration tests for the CMIS Repository Connector.
        Hide
        Karl Wright added a comment -

        I checked in the patch changes.

        Show
        Karl Wright added a comment - I checked in the patch changes.
        Hide
        Piergiorgio Lucidi added a comment -

        This patch includes

        • a bugfix for a wrong setting in the Base class for the CMIS Server instance
        • refactor for the tests/cmis/pom.xml: added a new property to set the absolute path for the root lib directory

        Remind to add section here:

        http://incubator.apache.org/connectors/how-to-build-and-deploy.html#Preparation

        about the new property named opencmis.server.war.path.

        Show
        Piergiorgio Lucidi added a comment - This patch includes a bugfix for a wrong setting in the Base class for the CMIS Server instance refactor for the tests/cmis/pom.xml: added a new property to set the absolute path for the root lib directory Remind to add section here: http://incubator.apache.org/connectors/how-to-build-and-deploy.html#Preparation about the new property named opencmis.server.war.path.
        Hide
        Karl Wright added a comment -

        I updated build.xml and Base.java to use relative paths to the wars. Now I can run the test under ant, but it fails in just the manner you describe (timeout waiting for job to finish). I was able to see manifoldcf.log when I exited; it's full of these:

        Error tossed: org.apache.chemistry.opencmis.client.runtime.FolderImpl
        java.lang.ClassCastException: org.apache.chemistry.opencmis.client.runtime.Folde
        rImpl
        at org.apache.manifoldcf.crawler.connectors.cmis.CmisRepositoryConnector
        .getDocumentVersions(CmisRepositoryConnector.java:1026)
        at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.getD
        ocumentVersions(BaseRepositoryConnector.java:355)
        at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.getD
        ocumentVersions(BaseRepositoryConnector.java:336)
        at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.getD
        ocumentVersions(BaseRepositoryConnector.java:315)
        at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.getD
        ocumentVersions(BaseRepositoryConnector.java:292)
        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.ja
        va:321)

        Show
        Karl Wright added a comment - I updated build.xml and Base.java to use relative paths to the wars. Now I can run the test under ant, but it fails in just the manner you describe (timeout waiting for job to finish). I was able to see manifoldcf.log when I exited; it's full of these: Error tossed: org.apache.chemistry.opencmis.client.runtime.FolderImpl java.lang.ClassCastException: org.apache.chemistry.opencmis.client.runtime.Folde rImpl at org.apache.manifoldcf.crawler.connectors.cmis.CmisRepositoryConnector .getDocumentVersions(CmisRepositoryConnector.java:1026) at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.getD ocumentVersions(BaseRepositoryConnector.java:355) at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.getD ocumentVersions(BaseRepositoryConnector.java:336) at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.getD ocumentVersions(BaseRepositoryConnector.java:315) at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.getD ocumentVersions(BaseRepositoryConnector.java:292) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.ja va:321)
        Hide
        Karl Wright added a comment -

        I think we should do what we do for other artifacts that are not yet available: ask the user to install them into maven, by just adding to the procedure described here:

        http://incubator.apache.org/connectors/how-to-build-and-deploy.html#Preparation

        Show
        Karl Wright added a comment - I think we should do what we do for other artifacts that are not yet available: ask the user to install them into maven, by just adding to the procedure described here: http://incubator.apache.org/connectors/how-to-build-and-deploy.html#Preparation
        Hide
        Piergiorgio Lucidi added a comment -

        Thanks.

        Karl,

        the CMIS server webapp is in the lib directory, but this directory is outside of the Maven module for CMIS tests.

        This means that now we need to add a new property in the tests/cmis/pom.xml to set the absolute path of the CMIS server.

        In the future we will fix this, when the new version of the CMIS Server will be available in the Maven repo, so we will not have any problem.

        But now it could be not good to leave the war artifact in the main lib directory, because to build and run tests we need to insert the needed path as a property value.

        A possible solution could be create a lib folder inside tests/cmis and move the CMIS server webapp in this new folder. In this way we can access using Maven because we are now in the module and we can refer to the webapp using a relative path.

        With the war artifact in the following path:

        tests/cmis/lib/chemistry-opencmis-server-inmemory-war-0.5.0-SNAPSHOT.war
        

        We can refer to it using a relative path in the pom.xml:

        <openCmisServerWarPath>lib/chemistry-opencmis-server-inmemory-war-0.5.0-SNAPSHOT.war</openCmisServerWarPath>
        

        WDYT?

        Show
        Piergiorgio Lucidi added a comment - Thanks. Karl, the CMIS server webapp is in the lib directory, but this directory is outside of the Maven module for CMIS tests. This means that now we need to add a new property in the tests/cmis/pom.xml to set the absolute path of the CMIS server. In the future we will fix this, when the new version of the CMIS Server will be available in the Maven repo, so we will not have any problem. But now it could be not good to leave the war artifact in the main lib directory, because to build and run tests we need to insert the needed path as a property value. A possible solution could be create a lib folder inside tests/cmis and move the CMIS server webapp in this new folder. In this way we can access using Maven because we are now in the module and we can refer to the webapp using a relative path. With the war artifact in the following path: tests/cmis/lib/chemistry-opencmis-server-inmemory-war-0.5.0-SNAPSHOT.war We can refer to it using a relative path in the pom.xml: <openCmisServerWarPath>lib/chemistry-opencmis-server-inmemory-war-0.5.0-SNAPSHOT.war</openCmisServerWarPath> WDYT?
        Hide
        Karl Wright added a comment -

        Added.

        Show
        Karl Wright added a comment - Added.
        Hide
        Piergiorgio Lucidi added a comment -

        This patch must be applied to the new branch in order to add the test/cmis/pom.xml.

        I forgot to add it before.
        I'm sorry.

        Show
        Piergiorgio Lucidi added a comment - This patch must be applied to the new branch in order to add the test/cmis/pom.xml. I forgot to add it before. I'm sorry.
        Hide
        Karl Wright added a comment -

        I checked this in with two changes. First, the chemistry war I put into the lib directory. I presume that the artifact under maven will be deployed from there, and that the ant build will refer to the war directly. Second, I did not include the changes to jetty-runner, since they didn't make any sense to me.

        The branch these changes are in is: https://svn.apache.org/repos/asf/incubator/lcf/branches/CONNECTORS-236.

        Show
        Karl Wright added a comment - I checked this in with two changes. First, the chemistry war I put into the lib directory. I presume that the artifact under maven will be deployed from there, and that the ant build will refer to the war directly. Second, I did not include the changes to jetty-runner, since they didn't make any sense to me. The branch these changes are in is: https://svn.apache.org/repos/asf/incubator/lcf/branches/CONNECTORS-236 .
        Hide
        Karl Wright added a comment -

        Ok, never mind - the pair of jetty servers is indeed separate as you say.

        Show
        Karl Wright added a comment - Ok, never mind - the pair of jetty servers is indeed separate as you say.
        Hide
        Karl Wright added a comment -

        Another change I'd like to see is to pull the snapshot war out of the maven repository. The filesystem tests do it that way by allowing optional -D arguments, which maven provides, to point at the repository war. Copying that would also make it possible to run the same tests under ant.

        Show
        Karl Wright added a comment - Another change I'd like to see is to pull the snapshot war out of the maven repository. The filesystem tests do it that way by allowing optional -D arguments, which maven provides, to point at the repository war. Copying that would also make it possible to run the same tests under ant.
        Hide
        Piergiorgio Lucidi added a comment -

        In this patch I configured two instances of the Jetty server: an instance dedicated to ManifoldCF and a second instance dedicated for the CMIS server on port 9090.

        Show
        Piergiorgio Lucidi added a comment - In this patch I configured two instances of the Jetty server: an instance dedicated to ManifoldCF and a second instance dedicated for the CMIS server on port 9090.
        Hide
        Karl Wright added a comment -

        Hi Piergiorgio,

        I had a look at your Setup method, and what you do is create a second instance of Jetty that includes all the ManifoldCF web applications, as well as the CMIS server. This should not be needed since the MCF web applications are already deployed on the other instance of Jetty. You just need to use the right port number to talk with them, which I believe is 8346.

        I'm not sure this is causing your difficulties or not but it certainly isn't good.

        Show
        Karl Wright added a comment - Hi Piergiorgio, I had a look at your Setup method, and what you do is create a second instance of Jetty that includes all the ManifoldCF web applications, as well as the CMIS server. This should not be needed since the MCF web applications are already deployed on the other instance of Jetty. You just need to use the right port number to talk with them, which I believe is 8346. I'm not sure this is causing your difficulties or not but it certainly isn't good.
        Hide
        Piergiorgio Lucidi added a comment -

        This is the OpenCMIS InMemory Repository that is distributed as a web application.

        Now we need to include this web app in the SVN, because Chemistry guys have patched for us this component.

        When the latest version of this artifact will be released in the official Chemistry Maven repository, it will be taken from Maven and we will remove it from SVN.

        This artifact must be dropped in the following folder in order to execute tests:

        tests/cmis/war/
        
        Show
        Piergiorgio Lucidi added a comment - This is the OpenCMIS InMemory Repository that is distributed as a web application. Now we need to include this web app in the SVN, because Chemistry guys have patched for us this component. When the latest version of this artifact will be released in the official Chemistry Maven repository, it will be taken from Maven and we will remove it from SVN. This artifact must be dropped in the following folder in order to execute tests: tests/cmis/war/
        Hide
        Piergiorgio Lucidi added a comment -

        This patch includes integration test for the CMIS Repository Connector.

        Probably there is an issue about starving on Jetty.

        Must be applied together with the latest version of the OpenCMIS InMemory Repository that is one of the attachments of this patch release.

        Follow another attachment.

        Show
        Piergiorgio Lucidi added a comment - This patch includes integration test for the CMIS Repository Connector. Probably there is an issue about starving on Jetty. Must be applied together with the latest version of the OpenCMIS InMemory Repository that is one of the attachments of this patch release. Follow another attachment.
        Hide
        Piergiorgio Lucidi added a comment -

        I have just started to configure yesterday the CMIS test server that will be started before the test.

        OpenCMIS includes some framework components that helps us to integrate OpenCMIS in a totally way and includes an InMemory Repository dedicated to execute test:

        http://chemistry.apache.org/java/developing/repositories/dev-repositories-inmemory.html

        The InMemory Repository is distributed as a war Maven dependencies that I have configured in the Manifold test as a simple Maven ArtifactItem.

        So now the CMIS test server is managed by the Maven test goal.
        Now I can start the CMIS Server from Maven and during these two days I hope to finish a first version of this integration test.

        I'm following the same approach used in the end-to-end test of the filesystem connector trying to re-use the same methods with test annotations: createTestArea, removeTestArea.

        TODO:

        • implement createTestArea method
        • implement removeTestArea method
        • implement the rest of the test class with the CMIS configuration initialization
        • implement all the assert to check the test execution
        Show
        Piergiorgio Lucidi added a comment - I have just started to configure yesterday the CMIS test server that will be started before the test. OpenCMIS includes some framework components that helps us to integrate OpenCMIS in a totally way and includes an InMemory Repository dedicated to execute test: http://chemistry.apache.org/java/developing/repositories/dev-repositories-inmemory.html The InMemory Repository is distributed as a war Maven dependencies that I have configured in the Manifold test as a simple Maven ArtifactItem. So now the CMIS test server is managed by the Maven test goal. Now I can start the CMIS Server from Maven and during these two days I hope to finish a first version of this integration test. I'm following the same approach used in the end-to-end test of the filesystem connector trying to re-use the same methods with test annotations: createTestArea, removeTestArea. TODO: implement createTestArea method implement removeTestArea method implement the rest of the test class with the CMIS configuration initialization implement all the assert to check the test execution
        Hide
        Karl Wright added a comment -

        If there is any way for the tests to start their own local test server, and tear it back down again when done, that's ideal. Otherwise, we need to come up with some sort of "standard testbed" notion for each repository out there.

        Show
        Karl Wright added a comment - If there is any way for the tests to start their own local test server, and tear it back down again when done, that's ideal. Otherwise, we need to come up with some sort of "standard testbed" notion for each repository out there.

          People

          • Assignee:
            Karl Wright
            Reporter:
            Piergiorgio Lucidi
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development