ManifoldCF
  1. ManifoldCF
  2. CONNECTORS-92

Move from ant to maven or other build system with decent library management

    Details

    • Type: Wish Wish
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: ManifoldCF 0.3
    • Component/s: Build
    • Labels:
      None

      Description

      I am looking at the current project structure. If we want to make another build tool available I think we need to change the directory structure. I tried to place a suggestion in an image. Can you please have a look at it. If we agree that this is a good way to go, than I will continue to work on a patch. Which might be a bit hard with all these changing directories, but I'll do my best to at least get an idea whether it would be working.

      So I have three questions:

      • Do you want to move to maven or put maven next to ant?
      • Do you prefer another build mechanism [ant with ivy, gradle, maven3]
      • Do you have an idea about the amount of scripts that need to be changed if we change the project structure

      The image of a possible project layout (that is based on the maven standards) is attached to the issue

      1. Screen shot 2010-08-23 at 16.31.07.png
        69 kB
        Jettro Coenradie
      2. patch-connectors.zip
        5 kB
        Jettro Coenradie
      3. move-to-maven-acf-framework.patch
        139 kB
        Jettro Coenradie
      4. maven-start-jar.patch
        0.4 kB
        Jettro Coenradie
      5. maven-poms-problem-starting-jetty-and-derby.patch
        23 kB
        Jettro Coenradie
      6. maven-poms-including-start-jar.patch
        25 kB
        Jettro Coenradie

        Activity

        Hide
        Karl Wright added a comment -

        We now have a maven build system, in addition to ant, that was contributed elsewhere, so I'm closing this ticket.

        Show
        Karl Wright added a comment - We now have a maven build system, in addition to ant, that was contributed elsewhere, so I'm closing this ticket.
        Hide
        Karl Wright added a comment - - edited

        This issue has stalled. The work done so far has created a reasonable directory structure with a pom.xml everywhere it needs to be. However, beyond that, nothing more has happened.

        It might be worth researching ant with ivy at this point, since that would be a natural extension of the current build system, rather than trying to go the maven route entirely.

        Show
        Karl Wright added a comment - - edited This issue has stalled. The work done so far has created a reasonable directory structure with a pom.xml everywhere it needs to be. However, beyond that, nothing more has happened. It might be worth researching ant with ivy at this point, since that would be a natural extension of the current build system, rather than trying to go the maven route entirely.
        Hide
        Jettro Coenradie added a comment -

        I think maven is not the right tool at the moment due to the libraries that are used that are not available in any repository, the way the sample is started. The dependencies that seem to be copied to each project. I am spending a lot of time on creating an assembly, but that is not really the part where maven shines.

        Show
        Jettro Coenradie added a comment - I think maven is not the right tool at the moment due to the libraries that are used that are not available in any repository, the way the sample is started. The dependencies that seem to be copied to each project. I am spending a lot of time on creating an assembly, but that is not really the part where maven shines.
        Hide
        Jettro Coenradie added a comment -

        Not a patch, but a zip containing a few poms for the connectors. The null connector and the jdbc connector have a pom

        Show
        Jettro Coenradie added a comment - Not a patch, but a zip containing a few poms for the connectors. The null connector and the jdbc connector have a pom
        Hide
        Jettro Coenradie added a comment -

        patch containing a good start to create the start.jar from the jetty-runner project

        Show
        Jettro Coenradie added a comment - patch containing a good start to create the start.jar from the jetty-runner project
        Hide
        Karl Wright added a comment -

        Jettro,

        Please go ahead and submit everything you have. I'd also like to know what you believe the stumbling blocks to be. Thanks for all your work on this so far.

        Show
        Karl Wright added a comment - Jettro, Please go ahead and submit everything you have. I'd also like to know what you believe the stumbling blocks to be. Thanks for all your work on this so far.
        Hide
        Jettro Coenradie added a comment -

        I worked on it tonight but I decided to stop. This path is not leading in a direction that I would like. To make most out of maven I would like to change more than you would be willing to right now. I cannot blame you, because you have something working right now. Maybe someone else wants to step in and finish what I have done. I can submit another patch with the stuff I have right now.

        Show
        Jettro Coenradie added a comment - I worked on it tonight but I decided to stop. This path is not leading in a direction that I would like. To make most out of maven I would like to change more than you would be willing to right now. I cannot blame you, because you have something working right now. Maybe someone else wants to step in and finish what I have done. I can submit another patch with the stuff I have right now.
        Hide
        Jettro Coenradie added a comment -

        No, time is my enemy at the moment

        Show
        Jettro Coenradie added a comment - No, time is my enemy at the moment
        Hide
        Karl Wright added a comment -

        Has there been any further progress on this issue?

        Show
        Karl Wright added a comment - Has there been any further progress on this issue?
        Hide
        Jettro Coenradie added a comment -

        As for the start.jar I do not see a problem. I think I am almost there. THe classpath already contains the lib part, so I only need to add the dependencies to the jetty runner project. As for the example code, I do not mind to keep using and to create the example. I only wanted to have maven to make it easier to setup my development environment and to do the dependency management.

        I'll try to come up with an improve pom for the start.jar, if that is not enough please let me know.

        Show
        Jettro Coenradie added a comment - As for the start.jar I do not see a problem. I think I am almost there. THe classpath already contains the lib part, so I only need to add the dependencies to the jetty runner project. As for the example code, I do not mind to keep using and to create the example. I only wanted to have maven to make it easier to setup my development environment and to do the dependency management. I'll try to come up with an improve pom for the start.jar, if that is not enough please let me know.
        Hide
        Karl Wright added a comment -

        I've thought about start.jar some more. I think it is essential that maven produce a jar that is functionally identical to the start.jar produced by the ant build, including the relative paths for all the classpath entries. Maven may be unable to directly produce the modules/dist/example output directory in the same way, but that's a different problem. I don't think there's going to be room in the world for two entirely distinct ways of running the example, and I want to avoid that - for documentation reasons, if for none other.

        The other reason for wanting the example setup to be identical is because the example organizational assumptions are partly baked into the jetty-runner code at this time. Those could in part be passed in as command-line arguments, but that would make running the example considerably harder, so I'd like to avoid that if possible.

        Show
        Karl Wright added a comment - I've thought about start.jar some more. I think it is essential that maven produce a jar that is functionally identical to the start.jar produced by the ant build, including the relative paths for all the classpath entries. Maven may be unable to directly produce the modules/dist/example output directory in the same way, but that's a different problem. I don't think there's going to be room in the world for two entirely distinct ways of running the example, and I want to avoid that - for documentation reasons, if for none other. The other reason for wanting the example setup to be identical is because the example organizational assumptions are partly baked into the jetty-runner code at this time. Those could in part be passed in as command-line arguments, but that would make running the example considerably harder, so I'd like to avoid that if possible.
        Hide
        Karl Wright added a comment -

        Another way you can determine what's supposed to be a dependency is to look at the start.jar produced by the ant build:

        <attribute name="Class-Path" value="lib/commons-codec.jar lib/commons-collections.jar lib/commons-el.jar lib/commons-fileupload.jar lib/commons-httpclient-acf.jar lib/commons-io.jar lib/commons-logging.jar lib/derbyclient.jar lib/derby.jar lib/derbyLocale_cs.jar lib/derbyLocale_de_DE.jar lib/derbyLocale_es.jar lib/derbyLocale_fr.jar lib/derbyLocale_hu.jar lib/derbyLocale_it.jar lib/derbyLocale_ja_JP.jar lib/derbyLocale_ko_KR.jar lib/derbyLocale_pl.jar lib/derbyLocale_pt_BR.jar lib/derbyLocale_ru.jar lib/derbyLocale_zh_CN.jar lib/derbyLocale_zh_TW.jar lib/derbynet.jar lib/derbyrun.jar lib/derbytools.jar lib/eclipse-ecj.jar lib/jasper-6.0.24.jar lib/jasper-el-6.0.24.jar lib/jdbcpool-0.99.jar lib/jetty-6.1.22.jar lib/jetty-util-6.1.22.jar lib/jsp-api-2.1-glassfish-9.1.1.B60.25.p2.jar lib/json.jar lib/acf-agents.jar lib/acf-core.jar lib/acf-jetty-runner.jar lib/acf-pull-agent.jar lib/acf-ui-core.jar lib/log4j-1.2.jar lib/postgresql.jar lib/serializer.jar lib/servlet-api-2.5-20081211.jar lib/tomcat-juli-6.0.24.jar lib/xalan2.jar lib/xercesImpl-lcf.jar lib/xml-apis.jar"/>

        Note that commons-httpclient-acf.jar is our own version of commons-httpclient, and must therefore NOT be an external dependency.

        Show
        Karl Wright added a comment - Another way you can determine what's supposed to be a dependency is to look at the start.jar produced by the ant build: <attribute name="Class-Path" value="lib/commons-codec.jar lib/commons-collections.jar lib/commons-el.jar lib/commons-fileupload.jar lib/commons-httpclient-acf.jar lib/commons-io.jar lib/commons-logging.jar lib/derbyclient.jar lib/derby.jar lib/derbyLocale_cs.jar lib/derbyLocale_de_DE.jar lib/derbyLocale_es.jar lib/derbyLocale_fr.jar lib/derbyLocale_hu.jar lib/derbyLocale_it.jar lib/derbyLocale_ja_JP.jar lib/derbyLocale_ko_KR.jar lib/derbyLocale_pl.jar lib/derbyLocale_pt_BR.jar lib/derbyLocale_ru.jar lib/derbyLocale_zh_CN.jar lib/derbyLocale_zh_TW.jar lib/derbynet.jar lib/derbyrun.jar lib/derbytools.jar lib/eclipse-ecj.jar lib/jasper-6.0.24.jar lib/jasper-el-6.0.24.jar lib/jdbcpool-0.99.jar lib/jetty-6.1.22.jar lib/jetty-util-6.1.22.jar lib/jsp-api-2.1-glassfish-9.1.1.B60.25.p2.jar lib/json.jar lib/acf-agents.jar lib/acf-core.jar lib/acf-jetty-runner.jar lib/acf-pull-agent.jar lib/acf-ui-core.jar lib/log4j-1.2.jar lib/postgresql.jar lib/serializer.jar lib/servlet-api-2.5-20081211.jar lib/tomcat-juli-6.0.24.jar lib/xalan2.jar lib/xercesImpl-lcf.jar lib/xml-apis.jar"/> Note that commons-httpclient-acf.jar is our own version of commons-httpclient, and must therefore NOT be an external dependency.
        Hide
        Karl Wright added a comment -

        I noticed that jetty-runner/pom.xml does not seem to have anything specifying external jars in a classpath. Therefore, I believe that the produced start.jar will not properly function. Have you tried this in practice yet? You will want to include the derby jars, and probably most of the jars specified on the maven dependency page. Be aware that that page is somewhat out of date, and (for instance) the tomcat jars are not needed unless you actually try to compile the jsp's.

        https://cwiki.apache.org/confluence/display/CONNECTORS/Maven+Dependencies

        Show
        Karl Wright added a comment - I noticed that jetty-runner/pom.xml does not seem to have anything specifying external jars in a classpath. Therefore, I believe that the produced start.jar will not properly function. Have you tried this in practice yet? You will want to include the derby jars, and probably most of the jars specified on the maven dependency page. Be aware that that page is somewhat out of date, and (for instance) the tomcat jars are not needed unless you actually try to compile the jsp's. https://cwiki.apache.org/confluence/display/CONNECTORS/Maven+Dependencies
        Hide
        Karl Wright added a comment -

        Committed those additions. r990908.

        Show
        Karl Wright added a comment - Committed those additions. r990908.
        Hide
        Jettro Coenradie added a comment -

        This patch creates all the framework jars and wars as well as a start.jar in the jetty-runner project. The jars need to be in the same location as in the ant generated jar file. The lib folder.

        Later on we can create a module that grabs all libraries and wars, puts them in the right directory structure.

        Show
        Jettro Coenradie added a comment - This patch creates all the framework jars and wars as well as a start.jar in the jetty-runner project. The jars need to be in the same location as in the ant generated jar file. The lib folder. Later on we can create a module that grabs all libraries and wars, puts them in the right directory structure.
        Hide
        Karl Wright added a comment -

        I am still thinking about why this is so hard. Would be nice to have something like a servlet or filter that initializes everything that you do in your special runner now.

        The issues have to do with these facts:

        • Embedded derby is single-process. You cannot run more than one process against a given database at a given time.
        • ACF supports both single-process and multi-process models, but IF you're going to use single-process, you need to have a main class that starts up all the threads that would otherwise be different processes. That's what jetty-runner does, in part.

        So, obviously, something like jetty-runner needs to exist if you are going to use derby. I don't think maven magic will suffice to replace the code that does that.

        Furthermore, I think trying to get maven to do this for us is overkill. I'm open to suggestions, but I still don't think you need to solve this problem in order to have ACF be built effectively by maven.

        What I think we need to build at the framework level are all the jars and wars (which it looks like you have pretty well specified), PLUS a start.jar (which I didn't see anywhere - did I miss it?). Then your example execution will not be a "jetty instance" per se, but will simply fire off the equivalent of "java -jar start.jar". I can't believe there isn't a maven plugin for that. This, of course, must happen at the modules level, because no connectors will be available at the framework level.

        Show
        Karl Wright added a comment - I am still thinking about why this is so hard. Would be nice to have something like a servlet or filter that initializes everything that you do in your special runner now. The issues have to do with these facts: Embedded derby is single-process. You cannot run more than one process against a given database at a given time. ACF supports both single-process and multi-process models, but IF you're going to use single-process, you need to have a main class that starts up all the threads that would otherwise be different processes. That's what jetty-runner does, in part. So, obviously, something like jetty-runner needs to exist if you are going to use derby. I don't think maven magic will suffice to replace the code that does that. Furthermore, I think trying to get maven to do this for us is overkill. I'm open to suggestions, but I still don't think you need to solve this problem in order to have ACF be built effectively by maven. What I think we need to build at the framework level are all the jars and wars (which it looks like you have pretty well specified), PLUS a start.jar (which I didn't see anywhere - did I miss it?). Then your example execution will not be a "jetty instance" per se, but will simply fire off the equivalent of "java -jar start.jar". I can't believe there isn't a maven plugin for that. This, of course, must happen at the modules level, because no connectors will be available at the framework level.
        Hide
        Jettro Coenradie added a comment -

        I am afraid the crawler-ui is not done yet. I think we are missing libraries. What I was trying to do is make it run and see what I was missing. But the way I understand it now, I must choose another path to make it run. This seemed the most easy thing to do. I am still thinking about why this is so hard. Would be nice to have something like a servlet or filter that initializes everything that you do in your special runner now. But if you spend a lot of time getting it right, it must not be the easiest thing to do.

        I will come with a better patch, hope to be able to build something that I can actually test using the sample.

        Show
        Jettro Coenradie added a comment - I am afraid the crawler-ui is not done yet. I think we are missing libraries. What I was trying to do is make it run and see what I was missing. But the way I understand it now, I must choose another path to make it run. This seemed the most easy thing to do. I am still thinking about why this is so hard. Would be nice to have something like a servlet or filter that initializes everything that you do in your special runner now. But if you spend a lot of time getting it right, it must not be the easiest thing to do. I will come with a better patch, hope to be able to build something that I can actually test using the sample.
        Hide
        Karl Wright added a comment -

        I take that back for crawler-ui/pom.xml. This seems to be trying to set up a running version of the framework plus a couple of connectors. I don't think that's the right place for that logic; I'd prefer it just did a straight build of the crawler-ui web application. The example runner, if we do one under maven, really belongs at the "modules" level.

        Will you be submitting a patch that pares down crawler-ui/pom.xml, or shall I just edit the current artifact accordingly?

        Show
        Karl Wright added a comment - I take that back for crawler-ui/pom.xml. This seems to be trying to set up a running version of the framework plus a couple of connectors. I don't think that's the right place for that logic; I'd prefer it just did a straight build of the crawler-ui web application. The example runner, if we do one under maven, really belongs at the "modules" level. Will you be submitting a patch that pares down crawler-ui/pom.xml, or shall I just edit the current artifact accordingly?
        Hide
        Karl Wright added a comment -

        I've had a cursory glance at the pom files and they all look reasonable. I'm going to play around with this a bit locally to see how it behaves, and then if all seems OK I am happy to commit those.

        Show
        Karl Wright added a comment - I've had a cursory glance at the pom files and they all look reasonable. I'm going to play around with this a bit locally to see how it behaves, and then if all seems OK I am happy to commit those.
        Hide
        Karl Wright added a comment -

        Jettro,

        If you are using maven to start jetty directly, it will not work. You are missing the jetty runner, which only starts jetty at the end of a number of steps, including creating the database properly and setting up the schema and registering the connectors. Then, the crawler itself is started as a separate thread.

        It took me many weeks to get everything to work properly using jetty. Changing all this stuff around does not seem either warranted or useful at this time. I strongly recommend that you concentrate on using maven to actually build the software, and not try to re-engineer the example right now.

        Show
        Karl Wright added a comment - Jettro, If you are using maven to start jetty directly, it will not work. You are missing the jetty runner, which only starts jetty at the end of a number of steps, including creating the database properly and setting up the schema and registering the connectors. Then, the crawler itself is started as a separate thread. It took me many weeks to get everything to work properly using jetty. Changing all this stuff around does not seem either warranted or useful at this time. I strongly recommend that you concentrate on using maven to actually build the software, and not try to re-engineer the example right now.
        Hide
        Jettro Coenradie added a comment -

        This is a patch that addes poms to the framework part. It runs after you have installed the one missing jar.

        In the end the mvn clean install should be successful in the root of the framework directory

        In the crawler-ui project you can do : mvn clean jetty:run-war

        This will start up the crawler-ui, only after you have copied the conenctors that you need to the src/config/local/connector-lib

        Than asking for a list of connectors results in a database error. I just cannot get the Derby instance to run. The database is created but if seems not to be running. Any help in this area would be appreciated.

        Caused by: ERROR 42Y07: Schema 'ACF' does not exist
        at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
        at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSchemaDescriptor(Unknown Source)
        at org.apache.derby.impl.sql.compile.QueryTreeNode.getSchemaDescriptor(Unknown Source)
        at org.apache.derby.impl.sql.compile.QueryTreeNode.getSchemaDescriptor(Unknown Source)
        at org.apache.derby.impl.sql.compile.FromBaseTable.bindTableDescriptor(Unknown Source)
        at org.apache.derby.impl.sql.compile.FromBaseTable.bindNonVTITables(Unknown Source)
        at org.apache.derby.impl.sql.compile.FromList.bindTables(Unknown Source)
        at org.apache.derby.impl.sql.compile.SelectNode.bindNonVTITables(Unknown Source)
        at org.apache.derby.impl.sql.compile.DMLStatementNode.bindTables(Unknown Source)
        at org.apache.derby.impl.sql.compile.DMLStatementNode.bind(Unknown Source)
        at org.apache.derby.impl.sql.compile.CursorNode.bindStatement(Unknown Source)
        at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)
        at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
        at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source)
        ... 4 more

        Show
        Jettro Coenradie added a comment - This is a patch that addes poms to the framework part. It runs after you have installed the one missing jar. In the end the mvn clean install should be successful in the root of the framework directory In the crawler-ui project you can do : mvn clean jetty:run-war This will start up the crawler-ui, only after you have copied the conenctors that you need to the src/config/local/connector-lib Than asking for a list of connectors results in a database error. I just cannot get the Derby instance to run. The database is created but if seems not to be running. Any help in this area would be appreciated. Caused by: ERROR 42Y07: Schema 'ACF' does not exist at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSchemaDescriptor(Unknown Source) at org.apache.derby.impl.sql.compile.QueryTreeNode.getSchemaDescriptor(Unknown Source) at org.apache.derby.impl.sql.compile.QueryTreeNode.getSchemaDescriptor(Unknown Source) at org.apache.derby.impl.sql.compile.FromBaseTable.bindTableDescriptor(Unknown Source) at org.apache.derby.impl.sql.compile.FromBaseTable.bindNonVTITables(Unknown Source) at org.apache.derby.impl.sql.compile.FromList.bindTables(Unknown Source) at org.apache.derby.impl.sql.compile.SelectNode.bindNonVTITables(Unknown Source) at org.apache.derby.impl.sql.compile.DMLStatementNode.bindTables(Unknown Source) at org.apache.derby.impl.sql.compile.DMLStatementNode.bind(Unknown Source) at org.apache.derby.impl.sql.compile.CursorNode.bindStatement(Unknown Source) at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source) at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source) at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source) ... 4 more
        Hide
        Karl Wright added a comment -

        No, the directories ending in -service produce wars. Those ending in -servlet produce a jar.

        Show
        Karl Wright added a comment - No, the directories ending in -service produce wars. Those ending in -servlet produce a jar.
        Hide
        Jettro Coenradie added a comment -

        Oke, thats fine. But is the projects *-servlet a war? is that the web project?

        Show
        Jettro Coenradie added a comment - Oke, thats fine. But is the projects *-servlet a war? is that the web project?
        Hide
        Karl Wright added a comment -

        I should also clarify that, to me, "servlet" is not just a single class in any case, but a body of functionality responsible for fielding web requests. So I think the "servlet" label is quite accurate. Others, of course, doubtless have different definitions.

        Show
        Karl Wright added a comment - I should also clarify that, to me, "servlet" is not just a single class in any case, but a body of functionality responsible for fielding web requests. So I think the "servlet" label is quite accurate. Others, of course, doubtless have different definitions.
        Hide
        Karl Wright added a comment -

        Wouldn't it be better to rename the *-servlet into something like war or web. There will probably be more things in there than a servlet.

        No, really, there's just the servlet.
        All that I did was break the authority service into a separate web application and jar file. Both of these were built before under the heading of "authority-service", but since we're getting rigorous, I separated out the targets. Did the same thing for the api - there's now a servlet, and a service, one yields a jar, the other a war (which includes the jar).

        Show
        Karl Wright added a comment - Wouldn't it be better to rename the *-servlet into something like war or web. There will probably be more things in there than a servlet. No, really, there's just the servlet. All that I did was break the authority service into a separate web application and jar file. Both of these were built before under the heading of "authority-service", but since we're getting rigorous, I separated out the targets. Did the same thing for the api - there's now a servlet, and a service, one yields a jar, the other a war (which includes the jar).
        Hide
        Jettro Coenradie added a comment -

        Wouldn't it be better to rename the *-servlet into something like war or web. There will probably be more things in there than a servlet.

        Show
        Jettro Coenradie added a comment - Wouldn't it be better to rename the *-servlet into something like war or web. There will probably be more things in there than a servlet.
        Hide
        Jettro Coenradie added a comment -

        I see you have created two modules for the api-service. Is there a good reason for that? The servlet is not within the web project. In my original idea I created two modules, but maybe you have a good reason to create two?

        Show
        Jettro Coenradie added a comment - I see you have created two modules for the api-service. Is there a good reason for that? The servlet is not within the web project. In my original idea I created two modules, but maybe you have a good reason to create two?
        Hide
        Karl Wright added a comment -

        Connectors reorg also committed. r990100.

        Show
        Karl Wright added a comment - Connectors reorg also committed. r990100.
        Hide
        Jettro Coenradie added a comment -

        I did a quick check and it seems to be fine. If you commit the connector changes as well I'll try to commit a patch for both tonight.

        Show
        Jettro Coenradie added a comment - I did a quick check and it seems to be fine. If you commit the connector changes as well I'll try to commit a patch for both tonight.
        Hide
        Karl Wright added a comment -

        I am now ready to commit the connectors reorganization also, once I hear back.

        Show
        Karl Wright added a comment - I am now ready to commit the connectors reorganization also, once I hear back.
        Hide
        Karl Wright added a comment -

        I rearranged the framework part of the tree to what I believe will satisfy maven. The rest of the tree I will cover in a subsequent check-in, provided I got this part right. Can you verify that the current tree is correct, and can you upload a new maven patch based on the new tree?

        Show
        Karl Wright added a comment - I rearranged the framework part of the tree to what I believe will satisfy maven. The rest of the tree I will cover in a subsequent check-in, provided I got this part right. Can you verify that the current tree is correct, and can you upload a new maven patch based on the new tree?
        Hide
        Robert Muir added a comment -

        I am the last person to comment about maven, but i noticed tika has a maven build that builds a large single jar that contains
        all classes from dependencies

        Perhaps it would be a useful example if you want to do this 'jar coalescing'

        Show
        Robert Muir added a comment - I am the last person to comment about maven, but i noticed tika has a maven build that builds a large single jar that contains all classes from dependencies Perhaps it would be a useful example if you want to do this 'jar coalescing'
        Hide
        Jettro Coenradie added a comment -

        Very cool

        Show
        Jettro Coenradie added a comment - Very cool
        Hide
        Karl Wright added a comment -

        I am ok with postponing any jar coalescing until later. All I'd hoped to do was to make fewer jars, but that's not necessary for now.

        I notice you did not supply a build.xml change with the patch. I will need to supply the necessary hierarchy reorganizations and the build.xml changes before you will be able to post anything like a complete patch. My goal is going to be to maintain the current build in a working fashion, at least until it is clear that maven can meet all the requirements that are covered by the ant build, and that is not clear yet. I will work on the reorganization tonight, and commit the changes when I believe they are correct and have been verified as working. Then, please comment as to whether you feel they are adequate for your needs.

        Show
        Karl Wright added a comment - I am ok with postponing any jar coalescing until later. All I'd hoped to do was to make fewer jars, but that's not necessary for now. I notice you did not supply a build.xml change with the patch. I will need to supply the necessary hierarchy reorganizations and the build.xml changes before you will be able to post anything like a complete patch. My goal is going to be to maintain the current build in a working fashion, at least until it is clear that maven can meet all the requirements that are covered by the ant build, and that is not clear yet. I will work on the reorganization tonight, and commit the changes when I believe they are correct and have been verified as working. Then, please comment as to whether you feel they are adequate for your needs.
        Hide
        Jettro Coenradie added a comment -

        Sorry, but why is it easier to have one src for the complete framework. To my opinion the source code becomes a lot easier to understand if you make it explicit what the different modules are. Besides that, maven just does not work that way. It is also a lot harder when depending on these parts. The way I see it, we have already three different war files. The other jars are shared between the three war files. This way it becomes clear what deliverables there are without looking at the code. Just he directory structure is enough. It is also very easy to just compile/package one component. Step into the right directory and execute mvn clean install and you are done.

        I can imagine that the scripts and ant files become a little more complicated to change, but I think it is worth the effort to get a cleaner project layout.

        I did not have a good look at the Jetty runner yet. But I see it is a class with a very large main method. With maven it is not hard to make an executable jar file. Together with the assembly plugin and an overlay of the original crawler this should not be to hard to create.

        The integration tests can be put in one project to store them all, but you can also place the integration tests into the project that it is actually testing. Maven has a lifecycle phase for integration tests. Not sure that for instance the FileSystems tests are actually doing. What to they need to run? I see they have a requirement for multiple war files. So it seems logical to me to put them in a separate module. Give them dependencies on the framework modules and run them of you are executing the integration-test phase of maven.

        The environment related projects can be executed like they are executed now using ant. We can for instance create one maven module on top of them that executes these builds together with the package phase of maven. We could also provide a profile that the build is only executed if you explicitly tell it to.

        I hope this makes sense to you. Sometime it is hard to explain for me because I am so used to using maven nowadays that I might overlook something why others think it is hard to use.

        Show
        Jettro Coenradie added a comment - Sorry, but why is it easier to have one src for the complete framework. To my opinion the source code becomes a lot easier to understand if you make it explicit what the different modules are. Besides that, maven just does not work that way. It is also a lot harder when depending on these parts. The way I see it, we have already three different war files. The other jars are shared between the three war files. This way it becomes clear what deliverables there are without looking at the code. Just he directory structure is enough. It is also very easy to just compile/package one component. Step into the right directory and execute mvn clean install and you are done. I can imagine that the scripts and ant files become a little more complicated to change, but I think it is worth the effort to get a cleaner project layout. I did not have a good look at the Jetty runner yet. But I see it is a class with a very large main method. With maven it is not hard to make an executable jar file. Together with the assembly plugin and an overlay of the original crawler this should not be to hard to create. The integration tests can be put in one project to store them all, but you can also place the integration tests into the project that it is actually testing. Maven has a lifecycle phase for integration tests. Not sure that for instance the FileSystems tests are actually doing. What to they need to run? I see they have a requirement for multiple war files. So it seems logical to me to put them in a separate module. Give them dependencies on the framework modules and run them of you are executing the integration-test phase of maven. The environment related projects can be executed like they are executed now using ant. We can for instance create one maven module on top of them that executes these builds together with the package phase of maven. We could also provide a profile that the build is only executed if you explicitly tell it to. I hope this makes sense to you. Sometime it is hard to explain for me because I am so used to using maven nowadays that I might overlook something why others think it is hard to use.
        Hide
        Karl Wright added a comment -

        It looks to me like you adopted the one-jar-per-maven-script approach, with no coalescing of jars, but instead introducing /src/main under each of the subtargets within framework. I'd really like instead to make our job easier by at least combining the framework main jars together into one target first, along the lines I described above. I'd also like to get a sense of the overall picture before proceeding, so can we discuss what individual maven targets there are that you are proposing, and what each of them is, before we undertake any changes of this kind? The individual connector ones are obvious, but I'm concerned about stuff like the integration tests and the quick-start jetty package. How do you cover those in a maven build?

        Show
        Karl Wright added a comment - It looks to me like you adopted the one-jar-per-maven-script approach, with no coalescing of jars, but instead introducing /src/main under each of the subtargets within framework. I'd really like instead to make our job easier by at least combining the framework main jars together into one target first, along the lines I described above. I'd also like to get a sense of the overall picture before proceeding, so can we discuss what individual maven targets there are that you are proposing, and what each of them is, before we undertake any changes of this kind? The individual connector ones are obvious, but I'm concerned about stuff like the integration tests and the quick-start jetty package. How do you cover those in a maven build?
        Hide
        Jettro Coenradie added a comment -

        This is a first go on moving to maven. I started refactoring the framework part. This should give a good idea of what is possible. Of course we are nog there. I did not really touch any of the scripts. There is also one dependency you have to install locally. You'll see it when you try to run maven. Advice on ho to install it locally is provided when it fails.

        I am not sure if this patch will run, it is a big one. Hope for the best

        When the patch is successfully applied, go to the modules/framework folder and try:
        mvn clean install

        then step into crawler-ui and do
        mvn clean jetty:run-war

        Use you browser to go to localhost:8080 and you can see the crawler-ui web app. If you connect to an existing database, you should be able to see the configure connections lists. Not more than that, than we need to put the connectors on the classpath.

        I'd be happy to do something similar for the connectors. But than we must be sure that this is the way to go. It takes a lot of time.

        Show
        Jettro Coenradie added a comment - This is a first go on moving to maven. I started refactoring the framework part. This should give a good idea of what is possible. Of course we are nog there. I did not really touch any of the scripts. There is also one dependency you have to install locally. You'll see it when you try to run maven. Advice on ho to install it locally is provided when it fails. I am not sure if this patch will run, it is a big one. Hope for the best When the patch is successfully applied, go to the modules/framework folder and try: mvn clean install then step into crawler-ui and do mvn clean jetty:run-war Use you browser to go to localhost:8080 and you can see the crawler-ui web app. If you connect to an existing database, you should be able to see the configure connections lists. Not more than that, than we need to put the connectors on the classpath. I'd be happy to do something similar for the connectors. But than we must be sure that this is the way to go. It takes a lot of time.
        Hide
        Karl Wright added a comment -

        Web projects are no problem at all. You can even have dependencies between webproject. Althought I would try to make dependencies on jars only.

        The question is, who would want to depend on any individual ACF war files? If there's a need, then fine, but I don't see one here. The only use case I can come up with for anybody depending on ACF is on the main framework jars, which could be consolidated into one jar quite readily. I would therefore propose breaking up modules/framework into two pieces: "modules/framework-core", and "modules/framework", or some such. "framework-core" would contain what's currently in framework/core, framework/ui-core, framework/agents, and framework/pull-agent, and would have both an ant build and a maven build that wraps it. "framework" would contain "crawler-ui", "api', "authorityservice", and the jetty stuff, and would have a straight ant build and an ant-with-ivy build wrapping that. Each connector would have an ant build and a wrapping ant-ivy build also.

        Thoughts?

        Show
        Karl Wright added a comment - Web projects are no problem at all. You can even have dependencies between webproject. Althought I would try to make dependencies on jars only. The question is, who would want to depend on any individual ACF war files? If there's a need, then fine, but I don't see one here. The only use case I can come up with for anybody depending on ACF is on the main framework jars, which could be consolidated into one jar quite readily. I would therefore propose breaking up modules/framework into two pieces: "modules/framework-core", and "modules/framework", or some such. "framework-core" would contain what's currently in framework/core, framework/ui-core, framework/agents, and framework/pull-agent, and would have both an ant build and a maven build that wraps it. "framework" would contain "crawler-ui", "api', "authorityservice", and the jetty stuff, and would have a straight ant build and an ant-with-ivy build wrapping that. Each connector would have an ant build and a wrapping ant-ivy build also. Thoughts?
        Hide
        Karl Wright added a comment -

        It further occurs to me that the only jars you will ever need to depend on in ACF are the ones that you'd build connectors against. So, now I'm getting glimmers of a possible combined maven/ivy build where we have only one maven target, which is basically the jars acf-core, acf-core-ui, acf-agents, and acf-pull-agent, and where every other module in ACF is built using ivy to pull down dependencies that are needed. Of course, this would presume you wanted to build connectors independently of building ACF itself, which I could go either way on at the moment...

        Show
        Karl Wright added a comment - It further occurs to me that the only jars you will ever need to depend on in ACF are the ones that you'd build connectors against. So, now I'm getting glimmers of a possible combined maven/ivy build where we have only one maven target, which is basically the jars acf-core, acf-core-ui, acf-agents, and acf-pull-agent, and where every other module in ACF is built using ivy to pull down dependencies that are needed. Of course, this would presume you wanted to build connectors independently of building ACF itself, which I could go either way on at the moment...
        Hide
        Jettro Coenradie added a comment -

        I'll try to come up with something tonight. Today I am at a customer so I cannot spend to much time.

        Web projects are no problem at all. You can even have dependencies between webproject. Althought I would try to make dependencies on jars only.

        I'll come up with a directory structure and the components based on the deliverables or the targets.

        Show
        Jettro Coenradie added a comment - I'll try to come up with something tonight. Today I am at a customer so I cannot spend to much time. Web projects are no problem at all. You can even have dependencies between webproject. Althought I would try to make dependencies on jars only. I'll come up with a directory structure and the components based on the deliverables or the targets.
        Hide
        Karl Wright added a comment -

        It occurs to me that maybe ACF is better suited to work with ivy, since that is specifically designed to deal with input dependencies, and does have a rigid notion of targets.

        Show
        Karl Wright added a comment - It occurs to me that maybe ACF is better suited to work with ivy, since that is specifically designed to deal with input dependencies, and does have a rigid notion of targets.
        Hide
        Karl Wright added a comment -

        Planning things out in the wiki is not a bad idea at all. This ticket is also a reasonable forum.

        I think the first step is to agree on the individual maven deliverables/targets.

        The one-jar-per-maven-package restriction is going to dictate significant changes, because the way things are jar'd right now that would require either the breakup of the "framework" module into multiple components, OR the consolidation of the entire framework into a single jar. But the fact is that framework is NOT just a bunch of jars, it's also 3 war files. Those are as much a necessary part of the framework as are the jars. Is there a maven model for war files, too? I understand maven was primarily conceived as a way of handling jar dependencies, which may make it less suitable for dealing with ACF than we'd hoped. When I originally was thinking about maven, I worried more about the input dependencies than the targets. It seems we need to understand the targets. Can you make a proposal as to what maven targets you think we should have, and what they should include?

        Show
        Karl Wright added a comment - Planning things out in the wiki is not a bad idea at all. This ticket is also a reasonable forum. I think the first step is to agree on the individual maven deliverables/targets. The one-jar-per-maven-package restriction is going to dictate significant changes, because the way things are jar'd right now that would require either the breakup of the "framework" module into multiple components, OR the consolidation of the entire framework into a single jar. But the fact is that framework is NOT just a bunch of jars, it's also 3 war files. Those are as much a necessary part of the framework as are the jars. Is there a maven model for war files, too? I understand maven was primarily conceived as a way of handling jar dependencies, which may make it less suitable for dealing with ACF than we'd hoped. When I originally was thinking about maven, I worried more about the input dependencies than the targets. It seems we need to understand the targets. Can you make a proposal as to what maven targets you think we should have, and what they should include?
        Hide
        Jettro Coenradie added a comment -
        • Compiling C with maven : I have no experience with that
        • Multiple jars: I know people have done it. I would not do it however. Create a pom and structure for each module. Group the module by using a parent pom and a directory structure.
        • I agree that each connector should be a component on its own, with its own pom.
        • The fact that jetty has a dependency on the framework is no problem at all. We can just create an internal dependency and release them together if necessary.

        What is the next best step? Come up with a directory structure that is synchronized with our ideas of the different components?

        I can change the image, but we can also do something on the wiki or on the description of this issue.

        Show
        Jettro Coenradie added a comment - Compiling C with maven : I have no experience with that Multiple jars: I know people have done it. I would not do it however. Create a pom and structure for each module. Group the module by using a parent pom and a directory structure. I agree that each connector should be a component on its own, with its own pom. The fact that jetty has a dependency on the framework is no problem at all. We can just create an internal dependency and release them together if necessary. What is the next best step? Come up with a directory structure that is synchronized with our ideas of the different components? I can change the image, but we can also do something on the wiki or on the description of this issue.
        Hide
        Karl Wright added a comment -

        As a response to the remark from Karl

        (1) Breaking up modules and putting pieces of that all over the place
        I do not think they are all over the place, maybe I am thinking wrong about the modules part, but for me modules is not really clear. At the moment we have documentation, modules and tests. I suggest a slightly more separated mode with: documentation, integration-tests, framework, connectors and environment. The only change is to move some stuff from modules into a new part environment en move the other parts of modules one level up.

        Each thing under modules is something you'd want to build separately, which is why I chose the arrangement in the first place. If I were deploying these on a debian system, each would be its own package. That is, each connector would necessarily be its own package, as would mod-authz-annotate, and java-environment. Indeed, java-environment was originally a debian package that was part of the LCF software grant and has not been modified even to build, because it in effect represented a Debian java deployment framework rather than actual code. Same thing with postgres-config, except that was for postgresql configuration under Debian. Furthermore, mod-authz-annotate is C, and probably cannot be built under maven (or do I have that wrong?)

        Therefore, for a maven build we should plan on building the following as SEPARATE maven deliverables/targets:

        • (1) Each connector
        • (2) The framework
          If there is a way to build C stuff under maven, then this too should be a maven deliverable/target:
        • (3) mod-authz-annotate
          These should exist in the tree but be ignored for now, since they are not applicable to maven at all:
        • (4) java-environment
        • (5) postgres-config

        (2) Taking jetty-runner out of framework

        I do not think that Jetty is part of your framework, you create war files and give the option for an easy start using Jetty. But maybe I am wrong.

        I set the jetty example and runner up so that they do not have explicit dependencies on any individual connectors, and thus they're built as part of the framework, which they DO have a dependency on. A case could be made for having these be separated into their own module-level component, in which case they'd also be their own maven deliverable.

        (3) Introducing a "src" directory under each of the framework components

        At the moment when running ant. You get a lot of folders of which it is not always easy to understand whether they are original source folders or not. That is why maven comes with a clear separation of src, generated-source and target for other generated content. To my opinion this makes it easier to see what is under version control and what is not.

        Check the maven page for more explanation.
        http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html

        I will read the page. It seems to me that we'd need to agree what the maven deliverables would be before we can decide where the "src" directory goes. If the framework is a component all by itself (and I think it should be), then naturally the structure would be "modules/framework/src/..." instead. Does maven allow multiple jars in a deliverable? That would be a necessary condition.

        (4) Moving the tests so far away from the code they are related to

        I am not sure if I was clear enough on this. In the original code base a test folder is available next to modules. For unit tests I would keep them as close as possible to the source code. Therefore we have the src/main and src/test in the same module. The integration tests are another beast. Usually a lot of environmental setup needs to be done, they take longer, and you might want to store them in a different folder so you can run them all at once. Another option would be to add them next to the unit tests in a different folder [src/main, src/test/ and src/integration-test] or use a different naming scheme. **Test.java and **IntegrationTest.java That way you can folder them out as well and use the maven lifecycle to decide whether to run unit test or both unit and integration tests.

        As of right now, there are three kinds of tests in the system: Unit tests (which are checked in in the module they are to test), integration unit tests (which are checked in at the "modules" level), and full integration tests that are a legacy of the LCF code grant (which are checked in in the "tests" directory above the modules level). The full integration tests are not executed but were meant to furnish the rudiments of a test plan,as well as useful bits for manipulating repositories themselves during test processes, and thus must be considered "reference material" at this time. The "documentation" directory above the modules level has a similar history.

        I put the integration unit tests under "modules" because the main build script was checked in at the "modules" level, and it needs to run those tests. If we move the mail build script higher up, the integration unit tests can move up also.

        As for the part where we need ant to make certain deliveries, we can still do that. Than you have two options, or run the ant command to make these deliverables yourself, or add them to the package lifecycle stage of maven. It is not hard to call ant from maven. You could even create the mother of all ant build file that calls maven to build the framework and the conenctors.

        Let's settle the basic questions first, and then revisit the best way to do the maven build.

        Show
        Karl Wright added a comment - As a response to the remark from Karl (1) Breaking up modules and putting pieces of that all over the place I do not think they are all over the place, maybe I am thinking wrong about the modules part, but for me modules is not really clear. At the moment we have documentation, modules and tests. I suggest a slightly more separated mode with: documentation, integration-tests, framework, connectors and environment. The only change is to move some stuff from modules into a new part environment en move the other parts of modules one level up. Each thing under modules is something you'd want to build separately, which is why I chose the arrangement in the first place. If I were deploying these on a debian system, each would be its own package. That is, each connector would necessarily be its own package, as would mod-authz-annotate, and java-environment. Indeed, java-environment was originally a debian package that was part of the LCF software grant and has not been modified even to build, because it in effect represented a Debian java deployment framework rather than actual code. Same thing with postgres-config, except that was for postgresql configuration under Debian. Furthermore, mod-authz-annotate is C, and probably cannot be built under maven (or do I have that wrong?) Therefore, for a maven build we should plan on building the following as SEPARATE maven deliverables/targets: (1) Each connector (2) The framework If there is a way to build C stuff under maven, then this too should be a maven deliverable/target: (3) mod-authz-annotate These should exist in the tree but be ignored for now, since they are not applicable to maven at all: (4) java-environment (5) postgres-config (2) Taking jetty-runner out of framework I do not think that Jetty is part of your framework, you create war files and give the option for an easy start using Jetty. But maybe I am wrong. I set the jetty example and runner up so that they do not have explicit dependencies on any individual connectors, and thus they're built as part of the framework, which they DO have a dependency on. A case could be made for having these be separated into their own module-level component, in which case they'd also be their own maven deliverable. (3) Introducing a "src" directory under each of the framework components At the moment when running ant. You get a lot of folders of which it is not always easy to understand whether they are original source folders or not. That is why maven comes with a clear separation of src, generated-source and target for other generated content. To my opinion this makes it easier to see what is under version control and what is not. Check the maven page for more explanation. http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html I will read the page. It seems to me that we'd need to agree what the maven deliverables would be before we can decide where the "src" directory goes. If the framework is a component all by itself (and I think it should be), then naturally the structure would be "modules/framework/src/..." instead. Does maven allow multiple jars in a deliverable? That would be a necessary condition. (4) Moving the tests so far away from the code they are related to I am not sure if I was clear enough on this. In the original code base a test folder is available next to modules. For unit tests I would keep them as close as possible to the source code. Therefore we have the src/main and src/test in the same module. The integration tests are another beast. Usually a lot of environmental setup needs to be done, they take longer, and you might want to store them in a different folder so you can run them all at once. Another option would be to add them next to the unit tests in a different folder [src/main, src/test/ and src/integration-test] or use a different naming scheme. **Test.java and **IntegrationTest.java That way you can folder them out as well and use the maven lifecycle to decide whether to run unit test or both unit and integration tests. As of right now, there are three kinds of tests in the system: Unit tests (which are checked in in the module they are to test), integration unit tests (which are checked in at the "modules" level), and full integration tests that are a legacy of the LCF code grant (which are checked in in the "tests" directory above the modules level). The full integration tests are not executed but were meant to furnish the rudiments of a test plan,as well as useful bits for manipulating repositories themselves during test processes, and thus must be considered "reference material" at this time. The "documentation" directory above the modules level has a similar history. I put the integration unit tests under "modules" because the main build script was checked in at the "modules" level, and it needs to run those tests. If we move the mail build script higher up, the integration unit tests can move up also. As for the part where we need ant to make certain deliveries, we can still do that. Than you have two options, or run the ant command to make these deliverables yourself, or add them to the package lifecycle stage of maven. It is not hard to call ant from maven. You could even create the mother of all ant build file that calls maven to build the framework and the conenctors. Let's settle the basic questions first, and then revisit the best way to do the maven build.
        Hide
        Jettro Coenradie added a comment -

        As a response to the remark from Karl
        (1) Breaking up modules and putting pieces of that all over the place
        I do not think they are all over the place, maybe I am thinking wrong about the modules part, but for me modules is not really clear. At the moment we have documentation, modules and tests. I suggest a slightly more separated mode with: documentation, integration-tests, framework, connectors and environment. The only change is to move some stuff from modules into a new part environment en move the other parts of modules one level up.

        (2) Taking jetty-runner out of framework
        I do not think that Jetty is part of your framework, you create war files and give the option for an easy start using Jetty. But maybe I am wrong.

        (3) Introducing a "src" directory under each of the framework components
        At the moment when running ant. You get a lot of folders of which it is not always easy to understand whether they are original source folders or not. That is why maven comes with a clear separation of src, generated-source and target for other generated content. To my opinion this makes it easier to see what is under version control and what is not.

        Check the maven page for more explanation.
        http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html

        (4) Moving the tests so far away from the code they are related to
        I am not sure if I was clear enough on this. In the original code base a test folder is available next to modules. For unit tests I would keep them as close as possible to the source code. Therefore we have the src/main and src/test in the same module. The integration tests are another beast. Usually a lot of environmental setup needs to be done, they take longer, and you might want to store them in a different folder so you can run them all at once. Another option would be to add them next to the unit tests in a different folder [src/main, src/test/ and src/integration-test] or use a different naming scheme. **Test.java and **IntegrationTest.java That way you can folder them out as well and use the maven lifecycle to decide whether to run unit test or both unit and integration tests.

        As for the part where we need ant to make certain deliveries, we can still do that. Than you have two options, or run the ant command to make these deliverables yourself, or add them to the package lifecycle stage of maven. It is not hard to call ant from maven. You could even create the mother of all ant build file that calls maven to build the framework and the conenctors.

        Show
        Jettro Coenradie added a comment - As a response to the remark from Karl (1) Breaking up modules and putting pieces of that all over the place I do not think they are all over the place, maybe I am thinking wrong about the modules part, but for me modules is not really clear. At the moment we have documentation, modules and tests. I suggest a slightly more separated mode with: documentation, integration-tests, framework, connectors and environment. The only change is to move some stuff from modules into a new part environment en move the other parts of modules one level up. (2) Taking jetty-runner out of framework I do not think that Jetty is part of your framework, you create war files and give the option for an easy start using Jetty. But maybe I am wrong. (3) Introducing a "src" directory under each of the framework components At the moment when running ant. You get a lot of folders of which it is not always easy to understand whether they are original source folders or not. That is why maven comes with a clear separation of src, generated-source and target for other generated content. To my opinion this makes it easier to see what is under version control and what is not. Check the maven page for more explanation. http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html (4) Moving the tests so far away from the code they are related to I am not sure if I was clear enough on this. In the original code base a test folder is available next to modules. For unit tests I would keep them as close as possible to the source code. Therefore we have the src/main and src/test in the same module. The integration tests are another beast. Usually a lot of environmental setup needs to be done, they take longer, and you might want to store them in a different folder so you can run them all at once. Another option would be to add them next to the unit tests in a different folder [src/main, src/test/ and src/integration-test] or use a different naming scheme. **Test.java and **IntegrationTest.java That way you can folder them out as well and use the maven lifecycle to decide whether to run unit test or both unit and integration tests. As for the part where we need ant to make certain deliveries, we can still do that. Than you have two options, or run the ant command to make these deliverables yourself, or add them to the package lifecycle stage of maven. It is not hard to call ant from maven. You could even create the mother of all ant build file that calls maven to build the framework and the conenctors.
        Hide
        Karl Wright added a comment -

        Re: build preferences

        Continuing to have an ant build is actually pretty important for some modes of delivery. I'm specifically thinking of debian and Ubuntu packaging here. Maven does not work well with these packaging schemes because it's too all-encompassing. We therefore need a way of doing builds locally, without pulling things down from a mirror.

        My original thought was that we'd have multiple layers - ant being the most basic, with a maven wrapper available to pull down what the ant build needed, and have the maven build call ant underneath. I don't know how realistic that is, but it does solve all the problems if it can be done that way.

        Show
        Karl Wright added a comment - Re: build preferences Continuing to have an ant build is actually pretty important for some modes of delivery. I'm specifically thinking of debian and Ubuntu packaging here. Maven does not work well with these packaging schemes because it's too all-encompassing. We therefore need a way of doing builds locally, without pulling things down from a mirror. My original thought was that we'd have multiple layers - ant being the most basic, with a maven wrapper available to pull down what the ant build needed, and have the maven build call ant underneath. I don't know how realistic that is, but it does solve all the problems if it can be done that way.
        Hide
        Karl Wright added a comment -

        This proposed change has a number of features I don't understand the reasons for:

        (1) Breaking up modules and putting pieces of that all over the place
        (2) Taking jetty-runner out of framework
        (3) Introducing a "src" directory under each of the framework components
        (4) Moving the tests so far away from the code they are related to

        Can you describe your logic for this reorganization?

        Show
        Karl Wright added a comment - This proposed change has a number of features I don't understand the reasons for: (1) Breaking up modules and putting pieces of that all over the place (2) Taking jetty-runner out of framework (3) Introducing a "src" directory under each of the framework components (4) Moving the tests so far away from the code they are related to Can you describe your logic for this reorganization?
        Hide
        Jettro Coenradie added a comment -

        idea of the directory structure

        Show
        Jettro Coenradie added a comment - idea of the directory structure

          People

          • Assignee:
            Karl Wright
            Reporter:
            Jettro Coenradie
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development