Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4
    • Component/s: file manager
    • Labels:

      Description

      Move the current contents of http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examples, i.e. the default File Manager policy files 'elements.xml', 'product-types.xml' and 'product-type-element-map.xml' to a subdirectory named 'core', i.e.: http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examples/core.

      (ensure assembly.xml, unit tests, etc still work after moving!)

        Activity

        Hide
        Ross Laidlaw added a comment - - edited

        When building OODT, the following code in assembly.xml (in filemgr/src/main/assembly) copies the contents of the filemgr/src/main/resources/examples directory to the filemgr/policy directory in the build target:

        <fileSet>
          <directory>${basedir}/src/main/resources/examples</directory>
          <outputDirectory>policy</outputDirectory>
          <includes/>
        </fileSet>
        

        It also copies subdirectories and their contents too. This means that the newly created core and geo subdirectories will be copied over along with any files they contain. I've checked the documentation for the maven assembly plugin and this is by design rather than by accident:

        "When <include> subelements are present, they define a set of project coordinates to include. If none is present, then <includes> represents all valid values..."

        I've tested this by building the latest OODT trunk and confirming that the core and geo directories and contents were copied over to the target policy directory, as expected.

        This means that in the future we could add more example configurations as subdirectories and these would automatically be picked up.

        Therefore, to complete this task, a committer would need to perform svn moves for the three core files and commit the changes. As a newly appointed OODT committer I'd be happy to move and commit the files if the OODT team are happy for me to go ahead.

        Following the changes, to access the core files the filemgr.properties file can be updated reference the policy/core subdirectory rather than the policy directory, for example as follows:

        org.apache.oodt.cas.filemgr.repositorymgr.dirs=file:///usr/local/oodt/cas-filemgr/policy/core
        org.apache.oodt.cas.filemgr.validation.dirs=file:///usr/local/oodt/cas-filemgr/policy/core
        
        Show
        Ross Laidlaw added a comment - - edited When building OODT, the following code in assembly.xml (in filemgr/src/main/assembly ) copies the contents of the filemgr/src/main/resources/examples directory to the filemgr/policy directory in the build target: <fileSet> <directory>${basedir}/src/main/resources/examples</directory> <outputDirectory>policy</outputDirectory> <includes/> </fileSet> It also copies subdirectories and their contents too. This means that the newly created core and geo subdirectories will be copied over along with any files they contain. I've checked the documentation for the maven assembly plugin and this is by design rather than by accident: "When <include> subelements are present, they define a set of project coordinates to include. If none is present, then <includes> represents all valid values..." I've tested this by building the latest OODT trunk and confirming that the core and geo directories and contents were copied over to the target policy directory, as expected. This means that in the future we could add more example configurations as subdirectories and these would automatically be picked up. Therefore, to complete this task, a committer would need to perform svn moves for the three core files and commit the changes. As a newly appointed OODT committer I'd be happy to move and commit the files if the OODT team are happy for me to go ahead. Following the changes, to access the core files the filemgr.properties file can be updated reference the policy/core subdirectory rather than the policy directory, for example as follows: org.apache.oodt.cas.filemgr.repositorymgr.dirs=file: ///usr/local/oodt/cas-filemgr/policy/core org.apache.oodt.cas.filemgr.validation.dirs=file: ///usr/local/oodt/cas-filemgr/policy/core
        Hide
        Chris A. Mattmann added a comment -

        Hey Ross, thanks for investigating this. Can you confirm that the unit tests still pass in this scenario? If so, my +1 to commit it.

        Show
        Chris A. Mattmann added a comment - Hey Ross, thanks for investigating this. Can you confirm that the unit tests still pass in this scenario? If so, my +1 to commit it.
        Hide
        Ross Laidlaw added a comment - - edited

        Hi Chris!

        Apologies! After reading your comment, I realised that I had made a mistake when testing the build after moving the files. And indeed some unit tests for the filemgr component are failing with the build:

        • org.apache.oodt.cas.filemgr.catalog.TestLuceneCatalog
        • org.apache.oodt.cas.filemgr.ingest.TestLocalCache
        • org.apache.oodt.cas.filemgr.ingest.TestStdIngester
        • org.apache.oodt.cas.filemgr.ingest.TestRmiCache
        • org.apache.oodt.cas.filemgr.tools.TestMetadataBasedProductMover
        • org.apache.oodt.cas.filemgr.ingest.TestCachedIngester

        So there's a bit more work to be done other than just moving the files over. I found that each test is failing because of the following code (same in each test class):

        System.setProperty("org.apache.oodt.cas.filemgr.validation.dirs",
              "file://" + new File("./src/main/resources/examples").getAbsolutePath());
        

        And if I change the above code to the following, the tests all pass and the build succeeds:

        System.setProperty("org.apache.oodt.cas.filemgr.validation.dirs",
              "file://" + new File("./src/main/resources/examples/core").getAbsolutePath());
        

        Would you be happy for me to commit the moves and also patch the tests? I'll upload the patch soon after posting this comment.

        Show
        Ross Laidlaw added a comment - - edited Hi Chris! Apologies! After reading your comment, I realised that I had made a mistake when testing the build after moving the files. And indeed some unit tests for the filemgr component are failing with the build: org.apache.oodt.cas.filemgr.catalog.TestLuceneCatalog org.apache.oodt.cas.filemgr.ingest.TestLocalCache org.apache.oodt.cas.filemgr.ingest.TestStdIngester org.apache.oodt.cas.filemgr.ingest.TestRmiCache org.apache.oodt.cas.filemgr.tools.TestMetadataBasedProductMover org.apache.oodt.cas.filemgr.ingest.TestCachedIngester So there's a bit more work to be done other than just moving the files over. I found that each test is failing because of the following code (same in each test class): System .setProperty( "org.apache.oodt.cas.filemgr.validation.dirs" , "file: //" + new File( "./src/main/resources/examples" ).getAbsolutePath()); And if I change the above code to the following, the tests all pass and the build succeeds: System .setProperty( "org.apache.oodt.cas.filemgr.validation.dirs" , "file: //" + new File( "./src/main/resources/examples/core" ).getAbsolutePath()); Would you be happy for me to commit the moves and also patch the tests? I'll upload the patch soon after posting this comment.
        Hide
        Ross Laidlaw added a comment -

        Here's the patch to update the six filemgr tests to use the 'src/main/resources/examples/core' subdirectory.

        Show
        Ross Laidlaw added a comment - Here's the patch to update the six filemgr tests to use the 'src/main/resources/examples/core' subdirectory.
        Hide
        Chris A. Mattmann added a comment -

        +1 from me, go for it Ross!

        Show
        Chris A. Mattmann added a comment - +1 from me, go for it Ross!
        Hide
        Ross Laidlaw added a comment -

        I have committed the patch in r1342585. Next, I will update the user guides on the wiki to make sure that they refer to the filemgr/policy/core subdirectory.

        Show
        Ross Laidlaw added a comment - I have committed the patch in r1342585. Next, I will update the user guides on the wiki to make sure that they refer to the filemgr/policy/core subdirectory.
        Hide
        Ross Laidlaw added a comment -

        I have made some minor updates to the following user guides to account for the new location of the core policy files in File Manager:

        OODT Filemgr User Guide

        Everything you want to know about File Manager Policy

        OODT File Manager to SIS Connection Demo

        Together with the patch and file moves, this resolves 452.

        Show
        Ross Laidlaw added a comment - I have made some minor updates to the following user guides to account for the new location of the core policy files in File Manager: OODT Filemgr User Guide Everything you want to know about File Manager Policy OODT File Manager to SIS Connection Demo Together with the patch and file moves, this resolves 452.

          People

          • Assignee:
            Ross Laidlaw
            Reporter:
            Ross Laidlaw
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development