Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0, Trunk
    • Component/s: Build
    • Labels:
      None

      Description

      see the vote on the developer list.

      This is the first step: if we stop shipping a war then we are free to do anything we want.

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          32d 4h 58m 1 Robert Muir 08/Jun/13 20:08
          Resolved Resolved Reopened Reopened
          530d 19h 43m 2 Uwe Schindler 29/Nov/14 17:14
          Reopened Reopened Resolved Resolved
          12d 1h 6m 2 Uwe Schindler 03/Dec/14 15:58
          Resolved Resolved Closed Closed
          81d 13h 3m 1 Anshum Gupta 23/Feb/15 05:02
          Anshum Gupta made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Anshum Gupta added a comment -

          Bulk close after 5.0 release.

          Show
          Anshum Gupta added a comment - Bulk close after 5.0 release.
          Hide
          ASF GitHub Bot added a comment -

          Github user andyetitmoves closed the pull request at:

          https://github.com/apache/lucene-solr/pull/112

          Show
          ASF GitHub Bot added a comment - Github user andyetitmoves closed the pull request at: https://github.com/apache/lucene-solr/pull/112
          Hide
          Noble Paul added a comment -

          We can stop using HTTP only after we move to a no .war situation. After that we can probably use SPDY or netty sockets as alternative end-points. But this has to be first step

          Show
          Noble Paul added a comment - We can stop using HTTP only after we move to a no .war situation. After that we can probably use SPDY or netty sockets as alternative end-points. But this has to be first step
          Hide
          Mark Bennett added a comment -

          I realize that whether or not we ship a .war is not tightly coupled to whether or not we use HTTP to communicate, but I believe Elasticsearch uses some other type of socket (or maybe it's not even a socket?) to communicate between processes, supposedly it's much more efficient / lower latency.

          Has that also been discussed for Solr 5?

          Show
          Mark Bennett added a comment - I realize that whether or not we ship a .war is not tightly coupled to whether or not we use HTTP to communicate, but I believe Elasticsearch uses some other type of socket (or maybe it's not even a socket?) to communicate between processes, supposedly it's much more efficient / lower latency. Has that also been discussed for Solr 5?
          Hide
          ASF GitHub Bot added a comment -

          Github user andyetitmoves closed the pull request at:

          https://github.com/apache/lucene-solr/pull/107

          Show
          ASF GitHub Bot added a comment - Github user andyetitmoves closed the pull request at: https://github.com/apache/lucene-solr/pull/107
          Hide
          Alexandre Rafalovitch added a comment -

          The top level README.txt (in 5.x) still has the following text:

          dist/solr-XX.war
          The Apache Solr Application. Deploy this WAR file to any servlet
          container to run Apache Solr.

          Should probably be deleted or at least pointed at the new location for WAR file with warning it is NOT official.

          Show
          Alexandre Rafalovitch added a comment - The top level README.txt (in 5.x) still has the following text: dist/solr-XX.war The Apache Solr Application. Deploy this WAR file to any servlet container to run Apache Solr. Should probably be deleted or at least pointed at the new location for WAR file with warning it is NOT official.
          Hide
          Jayson Minard added a comment -

          in distribution solr-webapp dir still exists, is that intentional? webapps contains the WAR but this old directory its stuck in the distribution. Should be removed yes?

          Show
          Jayson Minard added a comment - in distribution solr-webapp dir still exists, is that intentional? webapps contains the WAR but this old directory its stuck in the distribution. Should be removed yes?
          Hide
          Jayson Minard added a comment -

          I've added two tickets for Solr-Undertow to support this change:

          Bootstrap WAR download
          https://github.com/bremeld/solr-undertow/issues/11

          Solr 5.0 support (documentation, location from which to bootstrap WAR)
          https://github.com/bremeld/solr-undertow/issues/12

          Show
          Jayson Minard added a comment - I've added two tickets for Solr-Undertow to support this change: Bootstrap WAR download https://github.com/bremeld/solr-undertow/issues/11 Solr 5.0 support (documentation, location from which to bootstrap WAR) https://github.com/bremeld/solr-undertow/issues/12
          Uwe Schindler made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Uwe Schindler added a comment -

          Uwe Schindler, I think if you want to discuss further changes, we should do it in a new issue and leave this one at this.

          OK

          Show
          Uwe Schindler added a comment - Uwe Schindler, I think if you want to discuss further changes, we should do it in a new issue and leave this one at this. OK
          Hide
          Mark Miller added a comment -

          so final understanding: WAR gone from maven, WAR gone from dist, but will remain in server/web-apps

          That's how I intend to leave this issue.

          Uwe Schindler, I think if you want to discuss further changes, we should do it in a new issue and leave this one at this.

          Show
          Mark Miller added a comment - so final understanding: WAR gone from maven, WAR gone from dist, but will remain in server/web-apps That's how I intend to leave this issue. Uwe Schindler , I think if you want to discuss further changes, we should do it in a new issue and leave this one at this.
          Hide
          ASF subversion and git services added a comment -

          Commit 1642880 from Mark Miller in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1642880 ]

          SOLR-4792: Remove WAR file from dist directory.

          Show
          ASF subversion and git services added a comment - Commit 1642880 from Mark Miller in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1642880 ] SOLR-4792 : Remove WAR file from dist directory.
          Hide
          ASF subversion and git services added a comment -

          Commit 1642879 from Mark Miller in branch 'dev/trunk'
          [ https://svn.apache.org/r1642879 ]

          SOLR-4792: Remove WAR file from dist directory.

          Show
          ASF subversion and git services added a comment - Commit 1642879 from Mark Miller in branch 'dev/trunk' [ https://svn.apache.org/r1642879 ] SOLR-4792 : Remove WAR file from dist directory.
          Hide
          Jayson Minard added a comment -

          Mark:

          so final understanding: WAR gone from maven, WAR gone from dist, but will remain in server/web-apps
          correct?

          Show
          Jayson Minard added a comment - Mark: so final understanding: WAR gone from maven, WAR gone from dist, but will remain in server/web-apps correct?
          Hide
          Mark Miller added a comment -

          My point is - we have had a discussion, we have had a vote, the WAR is gone.

          If someone wants to re hash things but is not willing to dig up the previous discussion and vote, that is up to them, but I don't have al lot to comment on then. Anyone can add any links to this issue.

          I'm not looking to have this fight yet again, and so I'm just going to make sure official WAR support is removed for 5x and go back to my regularly scheduled programming.

          Show
          Mark Miller added a comment - My point is - we have had a discussion, we have had a vote, the WAR is gone. If someone wants to re hash things but is not willing to dig up the previous discussion and vote, that is up to them, but I don't have al lot to comment on then. Anyone can add any links to this issue. I'm not looking to have this fight yet again, and so I'm just going to make sure official WAR support is removed for 5x and go back to my regularly scheduled programming.
          Hide
          Yonik Seeley added a comment -

          Sorry for the somewhat off-topic... here's what I tried:

          /opt/code/lusolr5/solr$ cp -rp server s1
          /opt/code/lusolr5/solr$ cd s1
          /opt/code/lusolr5/solr/s1$ ../bin/solr start -e techproducts -d .
          ERROR: start.jar file not found in /opt/code/lusolr5/solr/.!
          Please check your -d parameter to set the correct Solr server directory.
          /opt/code/lusolr5/solr/s1$ ../bin/solr start -e techproducts -d `pwd`
          Waiting to see Solr listening on port 8983 [/]  
          Started Solr server on port 8983 (pid=97723). Happy searching!
          [...]
          /opt/code/lusolr5/solr/s1$ find . -name data
          /opt/code/lusolr5/solr/s1$ find .. -name data
          ../server/solr/techproducts/data
          /opt/code/lusolr5/solr/s1$ 
          

          So I guess my mistake was thinking that the examples like "techproducts" were portable.

          Oh, and I do like how some commands are displayed, like:

          Creating new core 'techproducts' using command:
          http://localhost:8983/solr/admin/cores?action=CREATE&name=techproducts&instanceDir=techproducts
          
          Show
          Yonik Seeley added a comment - Sorry for the somewhat off-topic... here's what I tried: /opt/code/lusolr5/solr$ cp -rp server s1 /opt/code/lusolr5/solr$ cd s1 /opt/code/lusolr5/solr/s1$ ../bin/solr start -e techproducts -d . ERROR: start.jar file not found in /opt/code/lusolr5/solr/.! Please check your -d parameter to set the correct Solr server directory. /opt/code/lusolr5/solr/s1$ ../bin/solr start -e techproducts -d `pwd` Waiting to see Solr listening on port 8983 [/] Started Solr server on port 8983 (pid=97723). Happy searching! [...] /opt/code/lusolr5/solr/s1$ find . -name data /opt/code/lusolr5/solr/s1$ find .. -name data ../server/solr/techproducts/data /opt/code/lusolr5/solr/s1$ So I guess my mistake was thinking that the examples like "techproducts" were portable. Oh, and I do like how some commands are displayed, like: Creating new core 'techproducts' using command: http: //localhost:8983/solr/admin/cores?action=CREATE&name=techproducts&instanceDir=techproducts
          Hide
          Jayson Minard added a comment -

          ok, so final understanding: WAR gone from maven, WAR gone from dist, but will remain in server/web-apps

          correct?

          Show
          Jayson Minard added a comment - ok, so final understanding: WAR gone from maven, WAR gone from dist, but will remain in server/web-apps correct?
          Jayson Minard made changes -
          Comment [ Last note, because I'm probably boring people...

          Keep it in Maven does not put it into the distribution but means that Solr doesn't have to be rebuilt just to create a WAR. if not in the .tar.gz and not publicized, but still available in Maven then other's can link to it, document it, include it in other projects as a dependency, etc. So if it is in server/webapps but not in /dist, then why not maven. ]
          Hide
          Shawn Heisey added a comment -

          My opinion right now: As long as the example works and it's not supremely painful to adapt for production, we're probably good.

          My production install currently uses bits pulled from an older 4.x example – jetty and a war. Jetty in my install has been upgraded to an 8.x release higher than what is currently in Solr, but otherwise it's virtually identical. I built my own init script for CentOS 6.

          I'd like an easy upgrade path beyond the 4.9.1 that I'm currently using, but I'm not afraid of a little work. Sticking close to the example is important so that there aren't too many unusual bits to explain when I need help.

          Show
          Shawn Heisey added a comment - My opinion right now: As long as the example works and it's not supremely painful to adapt for production, we're probably good. My production install currently uses bits pulled from an older 4.x example – jetty and a war. Jetty in my install has been upgraded to an 8.x release higher than what is currently in Solr, but otherwise it's virtually identical. I built my own init script for CentOS 6. I'd like an easy upgrade path beyond the 4.9.1 that I'm currently using, but I'm not afraid of a little work. Sticking close to the example is important so that there aren't too many unusual bits to explain when I need help.
          Hide
          Jayson Minard added a comment -

          We have already discussed and had a vote. This issue is the result of that. The vote and previous discussions are available in the archives.

          Well, the result of the vote did not end up in this issue. Only the "break Solr deployment for all users" part seemed to have been recorded. And then acted upon. Great. The archives are helpful for fun reading but don't fix the problem.

          5 kind of came out of nowhere, and so this seemed the easiest transition path.

          easiest for whom?

          No, it won't be in maven - there will be no official WAR support.

          translation: "make love, not WARs"

          How about, remove the WAR when something worthwhile has replaced the functionality that people use in the app servers in which they host the WAR.

          Even ElasticSearch provides Transport Wares so that people can use WAR deployment, because in some places, it is just the rule of law. And you can convince the laws to change, when you have something sufficiently manageable as an alternative. Here, that is lacking. People trial this side by side with ES, and one will look better.

          I guess a follow-on list of activities is: someone create a solr-war-packaging project outside the code base so that it is "not official" (I now have https://github.com/bremeld/solr-in-a-war so will get on it based on current trunk). and maybe someone write the JIRA issues for making the Solr runner into a real project. Since that doesn't exist yet, I'll continue Solr-undertow to help in the meantime. But would be nice if it was in the same direction as what was intended rather than duplicate effort.

          Show
          Jayson Minard added a comment - We have already discussed and had a vote. This issue is the result of that. The vote and previous discussions are available in the archives. Well, the result of the vote did not end up in this issue. Only the "break Solr deployment for all users" part seemed to have been recorded. And then acted upon. Great. The archives are helpful for fun reading but don't fix the problem. 5 kind of came out of nowhere, and so this seemed the easiest transition path. easiest for whom? No, it won't be in maven - there will be no official WAR support. translation: "make love, not WARs" How about, remove the WAR when something worthwhile has replaced the functionality that people use in the app servers in which they host the WAR. Even ElasticSearch provides Transport Wares so that people can use WAR deployment, because in some places, it is just the rule of law. And you can convince the laws to change, when you have something sufficiently manageable as an alternative. Here, that is lacking. People trial this side by side with ES, and one will look better. I guess a follow-on list of activities is: someone create a solr-war-packaging project outside the code base so that it is "not official" (I now have https://github.com/bremeld/solr-in-a-war so will get on it based on current trunk). and maybe someone write the JIRA issues for making the Solr runner into a real project. Since that doesn't exist yet, I'll continue Solr-undertow to help in the meantime. But would be nice if it was in the same direction as what was intended rather than duplicate effort.
          Hide
          Steve Molloy added a comment -

          We have already discussed and had a vote.

          I didn't want to imply it wasn't discussed or that there wasn't any valid reason, I already said I agree with the change and we have moved away from using the war directly as soon as the decision was made. But still, linking this ticket to what it's actually blocking would make it clear why it is needed and why it's worth breaking some current integrations for people who check Jira but don't read the threads on the mailing list.
          Anyway, just wanted to help as we've been ready for the change for a while on our side.

          Show
          Steve Molloy added a comment - We have already discussed and had a vote. I didn't want to imply it wasn't discussed or that there wasn't any valid reason, I already said I agree with the change and we have moved away from using the war directly as soon as the decision was made. But still, linking this ticket to what it's actually blocking would make it clear why it is needed and why it's worth breaking some current integrations for people who check Jira but don't read the threads on the mailing list. Anyway, just wanted to help as we've been ready for the change for a while on our side.
          Hide
          Mark Miller added a comment -

          No, it won't be in maven - there will be no official WAR support.

          Show
          Mark Miller added a comment - No, it won't be in maven - there will be no official WAR support.
          Hide
          Alexandre Rafalovitch added a comment -

          So, to be clear on the possible compromise.

          Do not ship war in 5.0 itself (as part of download). But make it available as a maven package still for those who want to bundle it themselves in a different way (Tomcat, O/S packages, etc). Maybe mark it deprecated somewhere in the WAR (if possible).

          That would work the way I see the world and the messaging around Solr.

          Show
          Alexandre Rafalovitch added a comment - So, to be clear on the possible compromise. Do not ship war in 5.0 itself (as part of download). But make it available as a maven package still for those who want to bundle it themselves in a different way (Tomcat, O/S packages, etc). Maybe mark it deprecated somewhere in the WAR (if possible). That would work the way I see the world and the messaging around Solr.
          Hide
          Mark Miller added a comment -

          Perhaps a solution is: Remove war from /dist but keep it in /server/webapps. Then we don't commit to continue disting a war in 5.x, but downstream distros and products currently bundling Solr with their app, will have a quick way of using 5.x through an (unsupported) war, but at the same time get a sense of urge that Solr will default to another network layer very soon.

          This is what I initially wanted to do for 5 and still think may be the best path. 5 kind of came out of nowhere, and so this seemed the easiest transition path.

          Show
          Mark Miller added a comment - Perhaps a solution is: Remove war from /dist but keep it in /server/webapps. Then we don't commit to continue disting a war in 5.x, but downstream distros and products currently bundling Solr with their app, will have a quick way of using 5.x through an (unsupported) war, but at the same time get a sense of urge that Solr will default to another network layer very soon. This is what I initially wanted to do for 5 and still think may be the best path. 5 kind of came out of nowhere, and so this seemed the easiest transition path.
          Hide
          Mark Miller added a comment -

          I think all that is missing on this ticket is a link to the stuff that removing the WAR is expected to fix.

          We have already discussed and had a vote. This issue is the result of that. The vote and previous discussions are available in the archives.

          Show
          Mark Miller added a comment - I think all that is missing on this ticket is a link to the stuff that removing the WAR is expected to fix. We have already discussed and had a vote. This issue is the result of that. The vote and previous discussions are available in the archives.
          Hide
          Timothy Potter added a comment -

          I've been unable to get the latter to do some simple things like start up two different servers on different ports

          Yonik Seeley Maybe related to SOLR-6726 which has issues depending on which ports are chosen? If you want to start multiple servers, then there has to be multiple server directories or at the least, multiple solr.solr.home directories. Does the following work for you?

          cd solr
          cp -r server server2
          bin/solr start
          bin/solr start -p 8984 -d server2
          
          Show
          Timothy Potter added a comment - I've been unable to get the latter to do some simple things like start up two different servers on different ports Yonik Seeley Maybe related to SOLR-6726 which has issues depending on which ports are chosen? If you want to start multiple servers, then there has to be multiple server directories or at the least, multiple solr.solr.home directories. Does the following work for you? cd solr cp -r server server2 bin/solr start bin/solr start -p 8984 -d server2
          Hide
          Gopal Patwa added a comment -

          +1 to keep war in maven repo if possible, so user like us can easily migrate to solr 5.0, we deploy solr as war since company provide infrastructure as tomcat for deployment. It will take some time to change to other deployment model.

          Show
          Gopal Patwa added a comment - +1 to keep war in maven repo if possible, so user like us can easily migrate to solr 5.0, we deploy solr as war since company provide infrastructure as tomcat for deployment. It will take some time to change to other deployment model.
          Hide
          Steve Molloy added a comment -

          I think all that is missing on this ticket is a link to the stuff that removing the WAR is expected to fix. I mean I think I agree with the approach, but only because I can kind ofo guess at what issues we're trying to solve by reading different discussions and such. There has been talks about deadlocks, connections, and all sorts of optimizations that are currently not possible because of the fact that Solr is a webapp shipped as a WAR that must work on any webapp container. So, where are the entries for these optimizations that need to be made? If you have half a dozen of those that are all depending on not being a war that can run on any container, then people will stop asking why we're removing a war and we can get back to actually implementing improvements.

          Show
          Steve Molloy added a comment - I think all that is missing on this ticket is a link to the stuff that removing the WAR is expected to fix. I mean I think I agree with the approach, but only because I can kind ofo guess at what issues we're trying to solve by reading different discussions and such. There has been talks about deadlocks, connections, and all sorts of optimizations that are currently not possible because of the fact that Solr is a webapp shipped as a WAR that must work on any webapp container. So, where are the entries for these optimizations that need to be made? If you have half a dozen of those that are all depending on not being a war that can run on any container, then people will stop asking why we're removing a war and we can get back to actually implementing improvements.
          Hide
          Jayson Minard added a comment -

          Remove war from /dist but keep it in /server/webapps.

          Can we keep it (the WAR) in the maven produced artifacts and repo? that is the easiest place to pull it by other tools, either at build time, runtime or for users.

          Show
          Jayson Minard added a comment - Remove war from /dist but keep it in /server/webapps. Can we keep it (the WAR) in the maven produced artifacts and repo? that is the easiest place to pull it by other tools, either at build time, runtime or for users.
          Hide
          Ramkumar Aiyengar added a comment -

          Remove war from /dist but keep it in /server/webapps

          That's what the change above does. The original patch does it for the tarball, and the PR above (#112) does it for the build as well..

          Re: your question, one of the reasons for this change is so that we aren't constrained by the lowest denominator in terms of transport, that prevents us from implementing correctness properly (preventing distributed deadlocks), and hinders a lot of optimizations. Probably we can still come up with a suitable transport interface to allow for plugins, but even then a plugin to allow a servlet container like tomcat is going to be hard as presumably the interface is going to have demands the servlet interface can't satisfy.

          Show
          Ramkumar Aiyengar added a comment - Remove war from /dist but keep it in /server/webapps That's what the change above does. The original patch does it for the tarball, and the PR above (#112) does it for the build as well.. Re: your question, one of the reasons for this change is so that we aren't constrained by the lowest denominator in terms of transport, that prevents us from implementing correctness properly (preventing distributed deadlocks), and hinders a lot of optimizations. Probably we can still come up with a suitable transport interface to allow for plugins, but even then a plugin to allow a servlet container like tomcat is going to be hard as presumably the interface is going to have demands the servlet interface can't satisfy.
          Hide
          Jan Høydahl added a comment -

          Perhaps a solution is: Remove war from /dist but keep it in /server/webapps. Then we don't commit to continue disting a war in 5.x, but downstream distros and products currently bundling Solr with their app, will have a quick way of using 5.x through an (unsupported) war, but at the same time get a sense of urge that Solr will default to another network layer very soon.

          Let's start to formulating the release text for 5.0, it will probably help making decisions that will benefit users!

          Question: Even if we bundle Solr with Netty or similar in the future, is should not be a goal in itself to avoid people packaging Solr as a webapp to deploy in Tomcat? Solr should be transport agnostic, and various contribs/plugins could exist to support different ones.

          Show
          Jan Høydahl added a comment - Perhaps a solution is: Remove war from /dist but keep it in /server/webapps. Then we don't commit to continue disting a war in 5.x, but downstream distros and products currently bundling Solr with their app, will have a quick way of using 5.x through an (unsupported) war, but at the same time get a sense of urge that Solr will default to another network layer very soon. Let's start to formulating the release text for 5.0, it will probably help making decisions that will benefit users! Question: Even if we bundle Solr with Netty or similar in the future, is should not be a goal in itself to avoid people packaging Solr as a webapp to deploy in Tomcat? Solr should be transport agnostic, and various contribs/plugins could exist to support different ones.
          Jayson Minard made changes -
          Comment [ Besides being Kotlin, what would you want to see different? ]
          Hide
          Jayson Minard added a comment - - edited

          But if we remove it in 5, we can fix the rest of the usability issues in 5.1-5.3.

          I feel a half broken Solr 5.0 with expectations to fix later leaves a bad impression. Similar to collection api's missing in Solr 4.4 and added incrementally through Solr 4.10 leaving different functionality in each release for things that should have been present in 4.4. Some users of Solr are slow to upgrade and many have large important things missing because it is hard to find a 'complete' release. I prefer not to propagate thus behaviour into Solr 5.x. It is a big reason ElasticSearch gains ground.

          Show
          Jayson Minard added a comment - - edited But if we remove it in 5, we can fix the rest of the usability issues in 5.1-5.3. I feel a half broken Solr 5.0 with expectations to fix later leaves a bad impression. Similar to collection api's missing in Solr 4.4 and added incrementally through Solr 4.10 leaving different functionality in each release for things that should have been present in 4.4. Some users of Solr are slow to upgrade and many have large important things missing because it is hard to find a 'complete' release. I prefer not to propagate thus behaviour into Solr 5.x. It is a big reason ElasticSearch gains ground.
          Hide
          Jayson Minard added a comment - - edited

          The solr-undertow is not in Java, right? I think I saw a Kotlin reference in the docs.

          It is a stand alone component outside the main build and nothing depends on it directly. Kotlin is fully interoperable in both directions with Java with a tiny runtime. Stable. Performant. Uses undertow.io (written in Java). Easy to read. And is better than start.sh which is not in Java nor the JVM. It is a < 4MB component.

          Show
          Jayson Minard added a comment - - edited The solr-undertow is not in Java, right? I think I saw a Kotlin reference in the docs. It is a stand alone component outside the main build and nothing depends on it directly. Kotlin is fully interoperable in both directions with Java with a tiny runtime. Stable. Performant. Uses undertow.io (written in Java). Easy to read. And is better than start.sh which is not in Java nor the JVM. It is a < 4MB component.
          Hide
          Alexandre Rafalovitch added a comment -

          I don't understand why this issue exists in absence of the fully baked answer.

          I understand why this issue exists. If we don't remove war file in 5, it will be there until 6. Which could be very long time.

          But if we remove it in 5, we can fix the rest of the usability issues in 5.1-5.3. Given that the timing of 5 is driven quite heavily by Lucene 5, it may be the best of both worlds.

          At the same time, it would make sense to have the consequences/messaging clear. So, explicitly not sell Solr 5 to the package managers, etc. Tell them to stay with Solr 4 for now. Which sounds fine, but has secondary consequences.

          For example, people are starting to push schemaless Solr (to compete on messaging with Elasticsearch I guess). But the schemaless version of Solr in 4 is not a full solution (e.g. no easy way to create new type definitions). In 5, it will be better. So, not pushing people to 5, means also not pushing schemaless Solr so much.

          P.s. The solr-undertow is not in Java, right? I think I saw a Kotlin reference in the docs. I am not sure what the implications of that would be.

          Show
          Alexandre Rafalovitch added a comment - I don't understand why this issue exists in absence of the fully baked answer. I understand why this issue exists. If we don't remove war file in 5, it will be there until 6. Which could be very long time. But if we remove it in 5, we can fix the rest of the usability issues in 5.1-5.3. Given that the timing of 5 is driven quite heavily by Lucene 5, it may be the best of both worlds. At the same time, it would make sense to have the consequences/messaging clear. So, explicitly not sell Solr 5 to the package managers, etc. Tell them to stay with Solr 4 for now. Which sounds fine, but has secondary consequences. For example, people are starting to push schemaless Solr (to compete on messaging with Elasticsearch I guess). But the schemaless version of Solr in 4 is not a full solution (e.g. no easy way to create new type definitions). In 5, it will be better. So, not pushing people to 5, means also not pushing schemaless Solr so much. P.s. The solr-undertow is not in Java, right? I think I saw a Kotlin reference in the docs. I am not sure what the implications of that would be.
          Hide
          Jayson Minard added a comment -

          I see no value in removing war file. It is a documentation problem for deployment but us relied upon for users and isn't important to delete.

          Add the new 'ideal world' and overlap time to give a chance for devops to change, then delete. But probably is not important to delete once a better answer is present and the focus of documentation.

          I don't understand why this issue exists in absence of the fully baked answer. Or at least look at the existing implantation that solved this already.

          Raro

          Show
          Jayson Minard added a comment - I see no value in removing war file. It is a documentation problem for deployment but us relied upon for users and isn't important to delete. Add the new 'ideal world' and overlap time to give a chance for devops to change, then delete. But probably is not important to delete once a better answer is present and the focus of documentation. I don't understand why this issue exists in absence of the fully baked answer. Or at least look at the existing implantation that solved this already. Raro
          Hide
          Jayson Minard added a comment -

          Note: Solr-undertow switches to blocking i/o when using servlet but also supports non blocking i/o for other resources such as the Web resources or future elements that are non blocking. And undertow is much easier library than Netty for a base. More functionality, easier API, high performance. Basis for Wildly the JBoss replacement. Modern.

          Show
          Jayson Minard added a comment - Note: Solr-undertow switches to blocking i/o when using servlet but also supports non blocking i/o for other resources such as the Web resources or future elements that are non blocking. And undertow is much easier library than Netty for a base. More functionality, easier API, high performance. Basis for Wildly the JBoss replacement. Modern.
          Hide
          Jayson Minard added a comment -

          I would be willing to shore up. My Solr-undertow to use as a base. It is simpler, easy to configure, in production at two deployments of high scale (1500-1600 qps) using less threads and memory than jetty, out performed efforts of cloudera stack and did not need to break previous deployments by a war to accomplish this.

          My problem with this issue, to repeat what I said before, and others recently said today... Is that it is half the story, half baked. Without knowing what to do, it breaks things for no user benefit.

          Do you want to review Solr-undertow and provide feedback and I can merge it over. Without constraints of old params it can be even cleaner.

          It is further along than the startup scripts which really are not enough. And it has more thought into it than this issue appears to have.

          Show
          Jayson Minard added a comment - I would be willing to shore up. My Solr-undertow to use as a base. It is simpler, easy to configure, in production at two deployments of high scale (1500-1600 qps) using less threads and memory than jetty, out performed efforts of cloudera stack and did not need to break previous deployments by a war to accomplish this. My problem with this issue, to repeat what I said before, and others recently said today... Is that it is half the story, half baked. Without knowing what to do, it breaks things for no user benefit. Do you want to review Solr-undertow and provide feedback and I can merge it over. Without constraints of old params it can be even cleaner. It is further along than the startup scripts which really are not enough. And it has more thought into it than this issue appears to have.
          Hide
          Mark Miller added a comment -

          I've been unable to get the latter to do some simple things like start

          This is why I've not volunteered to much more for 5.0 myself. As it is coming right up, it's not a lot of time to make any large changes and verify that we are not missing any toes. I've got enough that I want to fix for 5 - I'd like to get the SolrCloud dev scripts working again, among other things.

          If we are going very far down this rabbit hole, it's going to take the effort of more than a few people to get it done properly and fully vetted in the time frame that has been proposed.

          I'm happy to simply say that we don't support being a WAR for 5 - if you count on the WAR, it's like using internal APIs. If others have the time to do more, I'm all for it, but there is precious little time left.

          Keep in mind that we won't really have parity of features either - I still think something like a request filter plugin point is pretty important, among other config complications. It might make a lot of sense to keep the webapp but not count in it for 5.x and then do a more complete switch in 6.x - once we have the proper time to digest and discuss it all.

          Show
          Mark Miller added a comment - I've been unable to get the latter to do some simple things like start This is why I've not volunteered to much more for 5.0 myself. As it is coming right up, it's not a lot of time to make any large changes and verify that we are not missing any toes. I've got enough that I want to fix for 5 - I'd like to get the SolrCloud dev scripts working again, among other things. If we are going very far down this rabbit hole, it's going to take the effort of more than a few people to get it done properly and fully vetted in the time frame that has been proposed. I'm happy to simply say that we don't support being a WAR for 5 - if you count on the WAR, it's like using internal APIs. If others have the time to do more, I'm all for it, but there is precious little time left. Keep in mind that we won't really have parity of features either - I still think something like a request filter plugin point is pretty important, among other config complications. It might make a lot of sense to keep the webapp but not count in it for 5.x and then do a more complete switch in 6.x - once we have the proper time to digest and discuss it all.
          Hide
          Yonik Seeley added a comment -

          The startup script work is already done with SOLR-3617, now that that's done, this just gets rid of the existing war interface.

          Those seem mostly independent though...
          "java -jar start.jar" is a startup mechanism, and so is "bin/solr start". Frankly, I've been unable to get the latter to do some simple things like start up two different servers on different ports, so I've reverted to using the former startup method.

          Show
          Yonik Seeley added a comment - The startup script work is already done with SOLR-3617 , now that that's done, this just gets rid of the existing war interface. Those seem mostly independent though... "java -jar start.jar" is a startup mechanism, and so is "bin/solr start". Frankly, I've been unable to get the latter to do some simple things like start up two different servers on different ports, so I've reverted to using the former startup method.
          Hide
          Alexandre Rafalovitch added a comment -

          ElasticSearch for that matter

          The problem is that Solr ships as a kitchen sink while Elasticsearch ships as a bare (production?) deployment. Just see the slides 4-5 of http://www.slideshare.net/arafalov/solr-vs-elasticsearch-case-by-case . The difference is stark.

          So, with the war file, people would go and create various packages around solr.war but without all the extra packages (e.g. Japanese, Polish, hadoop, etc). Now, it is not clear what the - user - story for production deployment will be now.

          This is not a story that can be resolved JIRA by JIRA. It's a usability impact that spans multiple issues and needs to be thought through on that level (this JIRA is not even linked into higher-level item...). And the impact assessment possibly includes looking at the things currently outside of Solr such as Linux packages, Puppet deployment modules, etc. If this change breaks all of them without any easy fix - the impact is quite high indeed.

          But how and by whom this needs to be discussed? I have no idea. I am just doing the canary in the coal mine job righ tnow.

          Show
          Alexandre Rafalovitch added a comment - ElasticSearch for that matter The problem is that Solr ships as a kitchen sink while Elasticsearch ships as a bare (production?) deployment. Just see the slides 4-5 of http://www.slideshare.net/arafalov/solr-vs-elasticsearch-case-by-case . The difference is stark. So, with the war file, people would go and create various packages around solr.war but without all the extra packages (e.g. Japanese, Polish, hadoop, etc). Now, it is not clear what the - user - story for production deployment will be now. This is not a story that can be resolved JIRA by JIRA. It's a usability impact that spans multiple issues and needs to be thought through on that level (this JIRA is not even linked into higher-level item...). And the impact assessment possibly includes looking at the things currently outside of Solr such as Linux packages, Puppet deployment modules, etc. If this change breaks all of them without any easy fix - the impact is quite high indeed. But how and by whom this needs to be discussed? I have no idea. I am just doing the canary in the coal mine job righ tnow.
          Hide
          Ramkumar Aiyengar added a comment -

          a more clear description of what Solr is expected to look like - from a Solr user perspective - once the infamous war is no longer "shipped". I mean, the phrase "we are free to do anything we want" may mean something to some of the more elite devs here, but show a little sympathy to the rest of the Solr community!

          From the user's perspective, Solr's only interface will be the bin/solr startup script, and I don't think that would change with stuff we want to do once we stop shipping the war. This is probably the model with most servers out there, like database servers, or ElasticSearch for that matter. I don't think there's any duplication of effort here, embedded Jetty or Netty are good alternatives we might resort to – but that's all up to discussion still, for now, we just want to hide us using a war as an implementation detail.

          The startup script work is already done with SOLR-3617, now that that's done, this just gets rid of the existing war interface.

          Show
          Ramkumar Aiyengar added a comment - a more clear description of what Solr is expected to look like - from a Solr user perspective - once the infamous war is no longer "shipped". I mean, the phrase "we are free to do anything we want" may mean something to some of the more elite devs here, but show a little sympathy to the rest of the Solr community! From the user's perspective, Solr's only interface will be the bin/solr startup script, and I don't think that would change with stuff we want to do once we stop shipping the war. This is probably the model with most servers out there, like database servers, or ElasticSearch for that matter. I don't think there's any duplication of effort here, embedded Jetty or Netty are good alternatives we might resort to – but that's all up to discussion still, for now, we just want to hide us using a war as an implementation detail. The startup script work is already done with SOLR-3617 , now that that's done, this just gets rid of the existing war interface.
          Hide
          Mark Miller added a comment -

          Thats an easy change.

          +1 - I think that is all where we want to end up as well. If you can help get it done by the 5 release, that would be great.

          Show
          Mark Miller added a comment - Thats an easy change. +1 - I think that is all where we want to end up as well. If you can help get it done by the 5 release, that would be great.
          Hide
          Jack Krupansky added a comment -

          As I just noted on the Solr user list, it would be helpful if people could provide a reference to some existing "server" products that they are attempting to model Solr 5.0 after - or provide a rationale as to why no existing server products provide a model worthy of adopting for Solr. I mean, are we trying too reinvent the wheel here, or what?!

          So, which existing Apache "server" product is Solr 5.0 most closely trying to emulate in terms of overall operation as a "server" and "web service"?

          I'd request that the description of this Jira be redone to provide a more clear description of what Solr is expected to look like - from a Solr user perspective - once the infamous war is no longer "shipped". I mean, the phrase "we are free to do anything we want" may mean something to some of the more elite devs here, but show a little sympathy to the rest of the Solr community!

          Show
          Jack Krupansky added a comment - As I just noted on the Solr user list, it would be helpful if people could provide a reference to some existing "server" products that they are attempting to model Solr 5.0 after - or provide a rationale as to why no existing server products provide a model worthy of adopting for Solr. I mean, are we trying too reinvent the wheel here, or what?! So, which existing Apache "server" product is Solr 5.0 most closely trying to emulate in terms of overall operation as a "server" and "web service"? I'd request that the description of this Jira be redone to provide a more clear description of what Solr is expected to look like - from a Solr user perspective - once the infamous war is no longer "shipped". I mean, the phrase "we are free to do anything we want" may mean something to some of the more elite devs here, but show a little sympathy to the rest of the Solr community!
          Hide
          Shawn Heisey added a comment -

          Some very smart people are steering this boat. I don't know where we're going, so it's time for me to take a back seat and wait to see where we end up.

          Show
          Shawn Heisey added a comment - Some very smart people are steering this boat. I don't know where we're going, so it's time for me to take a back seat and wait to see where we end up.
          Hide
          ASF subversion and git services added a comment -

          Commit 1642465 from Uwe Schindler in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1642465 ]

          Merged revision(s) 1642464 from lucene/dev/trunk:
          SOLR-4792: remove empty webapp folder in dev-tools/maven

          Show
          ASF subversion and git services added a comment - Commit 1642465 from Uwe Schindler in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1642465 ] Merged revision(s) 1642464 from lucene/dev/trunk: SOLR-4792 : remove empty webapp folder in dev-tools/maven
          Uwe Schindler made changes -
          Fix Version/s 5.0 [ 12327845 ]
          Hide
          ASF subversion and git services added a comment -

          Commit 1642464 from Uwe Schindler in branch 'dev/trunk'
          [ https://svn.apache.org/r1642464 ]

          SOLR-4792: remove empty webapp folder in dev-tools/maven

          Show
          ASF subversion and git services added a comment - Commit 1642464 from Uwe Schindler in branch 'dev/trunk' [ https://svn.apache.org/r1642464 ] SOLR-4792 : remove empty webapp folder in dev-tools/maven
          Hide
          Uwe Schindler added a comment - - edited

          I would really want to make the WAR file disappear completely. Its more or lessa matter of classpath. I would prefer to have a similar setup like in the test cases: Just start up JETTY with out own Main.class and configure a servlet context there, without an web application.

          I can help to do that. I fact the classpath separation done by the webapp container also brings more issues than it helps. We are already working on cleanup on SolrResourceLoader, so we should remove all classpath abstraction and let run Solr in its root application classloader, just starting a Jetty instance as a connector not start itsself inside Jetty. Thats an easy change. In the future we can then freely go the route to switch to Netty and remove the ServletFilter completely. With a WAR file in 5.0 we are hard wired to the webapp infrastructure until 6.0.

          Show
          Uwe Schindler added a comment - - edited I would really want to make the WAR file disappear completely. Its more or lessa matter of classpath. I would prefer to have a similar setup like in the test cases: Just start up JETTY with out own Main.class and configure a servlet context there, without an web application. I can help to do that. I fact the classpath separation done by the webapp container also brings more issues than it helps. We are already working on cleanup on SolrResourceLoader, so we should remove all classpath abstraction and let run Solr in its root application classloader, just starting a Jetty instance as a connector not start itsself inside Jetty. Thats an easy change. In the future we can then freely go the route to switch to Netty and remove the ServletFilter completely. With a WAR file in 5.0 we are hard wired to the webapp infrastructure until 6.0.
          Hide
          Uwe Schindler added a comment - - edited

          We know that? That's why this issue was reopened and Ram did a backport. Please see the comments above.

          Sorry, just noticed its gone - was a little bit behind. In trunk and 5.x we still have an empty webapp folder in dev-tools/maven. We should remove it!

          In any case we should keep the issue open until all instances of the WAR file are gone! This is really important to prevent people from asking questions all the time.

          Show
          Uwe Schindler added a comment - - edited We know that? That's why this issue was reopened and Ram did a backport. Please see the comments above. Sorry, just noticed its gone - was a little bit behind. In trunk and 5.x we still have an empty webapp folder in dev-tools/maven. We should remove it! In any case we should keep the issue open until all instances of the WAR file are gone! This is really important to prevent people from asking questions all the time.
          Hide
          ASF GitHub Bot added a comment -

          GitHub user andyetitmoves opened a pull request:

          https://github.com/apache/lucene-solr/pull/112

          Remove copy of .war in dist, rename example to server

          Patch for SOLR-4792 to move the war out of dist.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/bloomberg/lucene-solr trunk-war-clean

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/lucene-solr/pull/112.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #112


          commit 03f1a1d82c633d95c43a64ab26a2c32f94eccb68
          Author: Ramkumar Aiyengar <andyetitmoves@gmail.com>
          Date: 2014-11-29T13:40:19Z

          Remove copy of .war in dist, rename example to server


          Show
          ASF GitHub Bot added a comment - GitHub user andyetitmoves opened a pull request: https://github.com/apache/lucene-solr/pull/112 Remove copy of .war in dist, rename example to server Patch for SOLR-4792 to move the war out of dist. You can merge this pull request into a Git repository by running: $ git pull https://github.com/bloomberg/lucene-solr trunk-war-clean Alternatively you can review and apply these changes as the patch at: https://github.com/apache/lucene-solr/pull/112.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #112 commit 03f1a1d82c633d95c43a64ab26a2c32f94eccb68 Author: Ramkumar Aiyengar <andyetitmoves@gmail.com> Date: 2014-11-29T13:40:19Z Remove copy of .war in dist, rename example to server
          Hide
          Ramkumar Aiyengar added a comment - - edited

          I removed the war file from dist already, was ending up with a weird exception when testing with the start script which might be unrelated. Let me upload this now..

          This still keeps the war but it is not in dist but in the final webapps folder used by the start script.

          I also renamed the 'example' target to 'server' here, further in the direction of there being one target to build the server and run it. (have kept the example as an alias for now).

          Show
          Ramkumar Aiyengar added a comment - - edited I removed the war file from dist already, was ending up with a weird exception when testing with the start script which might be unrelated. Let me upload this now.. This still keeps the war but it is not in dist but in the final webapps folder used by the start script. I also renamed the 'example' target to 'server' here, further in the direction of there being one target to build the server and run it. (have kept the example as an alias for now).
          Hide
          Mark Miller added a comment -

          I'm working on a phone now, but my guess is that we may still need to tweak something to keep the war out of the dist folder

          Show
          Mark Miller added a comment - I'm working on a phone now, but my guess is that we may still need to tweak something to keep the war out of the dist folder
          Hide
          Mark Miller added a comment -

          We know that? That's why this issue was reopened and Ram did a backport. Please see the comments above.

          Show
          Mark Miller added a comment - We know that? That's why this issue was reopened and Ram did a backport. Please see the comments above.
          Uwe Schindler made changes -
          Fix Version/s 5.0 [ 12327845 ]
          Hide
          Uwe Schindler added a comment -

          FYI: Robert did not backport this to 5.x. It is only in trunk! This why the WAR is still in 5.0!

          Show
          Uwe Schindler added a comment - FYI: Robert did not backport this to 5.x. It is only in trunk! This why the WAR is still in 5.0!
          Hide
          Mark Miller added a comment -

          There has been a lot of historical discussion on all of those topics. At this point, this is happening regardless of what else happens. But more will happen over time and patches are welcome.

          Show
          Mark Miller added a comment - There has been a lot of historical discussion on all of those topics. At this point, this is happening regardless of what else happens. But more will happen over time and patches are welcome.
          Hide
          Jayson Minard added a comment -

          "Yup. Solr 5 is not a WAR. It's a search server application."

          or it is still what is was before, just the fact that it is servlet based is hidden now, and without the obvious option of alternative deployments.

          Regardless, I'll update solr-undertow (which already makes it a search server application by using the WAR outside of an application server) to work with a distribution archive instead of the packaged WAR. The WAR had less files than a full distribution which also includes examples so this makes Solr bigger.

          Would be nice if you do this that you make a clean distribution as well without any samples and just minimal placeholder files.

          Also with this change are the parameters for things like jetty.port, solr.home, solr.data going to be cleaned up and use something like a configuration file? or made more uniform and obvious in some way?

          This was something else I cleaned up in https://github.com/bremeld/solr-undertow using typesafe config. Maybe merge some of those ideas in and I can drop the separate project. But then again rate limiters to protect the server and other features might be missing...

          Also, we should review the thread and other settings that were previously recommended by Jetty if it is continued to be used, because they were excessive (10,000 will exceed most file limits set per user on boxes by default, and is unnecessarily high). Settings like this create a chance for the server to fail under high load rather than just slow down, it is more likely to crash.

          Also, multiple ports. One for internal communication, another for external, maybe another for updates different from queries, for admin. Helps to make sure that under load (eating the thread pool) a server still responds for other needs.

          If you take on responsibility to provide Solr as a server, it needs to start adding features around these other issues. Otherwise people will be configuring something in front of Solr to proxy and protect it, and then you might as well have stayed in a WAR so they could do it within the same process.

          Show
          Jayson Minard added a comment - "Yup. Solr 5 is not a WAR. It's a search server application." or it is still what is was before, just the fact that it is servlet based is hidden now, and without the obvious option of alternative deployments. Regardless, I'll update solr-undertow (which already makes it a search server application by using the WAR outside of an application server) to work with a distribution archive instead of the packaged WAR. The WAR had less files than a full distribution which also includes examples so this makes Solr bigger. Would be nice if you do this that you make a clean distribution as well without any samples and just minimal placeholder files. Also with this change are the parameters for things like jetty.port, solr.home, solr.data going to be cleaned up and use something like a configuration file? or made more uniform and obvious in some way? This was something else I cleaned up in https://github.com/bremeld/solr-undertow using typesafe config. Maybe merge some of those ideas in and I can drop the separate project. But then again rate limiters to protect the server and other features might be missing... Also, we should review the thread and other settings that were previously recommended by Jetty if it is continued to be used, because they were excessive (10,000 will exceed most file limits set per user on boxes by default, and is unnecessarily high). Settings like this create a chance for the server to fail under high load rather than just slow down, it is more likely to crash. Also, multiple ports. One for internal communication, another for external, maybe another for updates different from queries, for admin. Helps to make sure that under load (eating the thread pool) a server still responds for other needs. If you take on responsibility to provide Solr as a server, it needs to start adding features around these other issues. Otherwise people will be configuring something in front of Solr to proxy and protect it, and then you might as well have stayed in a WAR so they could do it within the same process.
          Hide
          Mark Miller added a comment -

          I have no issue with shipping with the war already expanded - I just see it as an optimization so not high on my list, but +1 if someone else wants to do the work.

          Show
          Mark Miller added a comment - I have no issue with shipping with the war already expanded - I just see it as an optimization so not high on my list, but +1 if someone else wants to do the work.
          Uwe Schindler made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hide
          Uwe Schindler added a comment -

          Reopen issue to also remove the obsolete WAR file that brings confusion, too.

          Show
          Uwe Schindler added a comment - Reopen issue to also remove the obsolete WAR file that brings confusion, too.
          Hide
          Uwe Schindler added a comment -

          We don't need to ship a war, even inside the current server dir. Personally, I would also change this! This is an overhead not really needed and would also make the distribution smaller, if we would remove it! Just build the classpath correctly, so server starts correctly. Currently, the curring jetty unpacks the webapp's WAR file to a temporary folder over an over. Why do this. Just put everything into a lib folder as done currently already and include it into the startup script's classpath. The web.xml and some other files is only there to configure the context correctly.

          Show
          Uwe Schindler added a comment - We don't need to ship a war, even inside the current server dir. Personally, I would also change this! This is an overhead not really needed and would also make the distribution smaller, if we would remove it! Just build the classpath correctly, so server starts correctly. Currently, the curring jetty unpacks the webapp's WAR file to a temporary folder over an over. Why do this. Just put everything into a lib folder as done currently already and include it into the startup script's classpath. The web.xml and some other files is only there to configure the context correctly.
          Hide
          Mark Miller added a comment -

          Yup. Solr 5 is not a WAR. It's a search server application.

          Show
          Mark Miller added a comment - Yup. Solr 5 is not a WAR. It's a search server application.
          Hide
          Alexandre Rafalovitch added a comment -

          So, running Solr under Tomcat is now officially a dead option. And the users who want to do that need to use HDS distribution instead?

          Just to be clear. Because we are going to have to deal with user questions on this over and over again (I predict). Both educating them on mental change and on what they are supposed to do now with their Tomcat installs. So, this needs to be a prominent explanation as part of the Reference Guide, etc.

          Show
          Alexandre Rafalovitch added a comment - So, running Solr under Tomcat is now officially a dead option. And the users who want to do that need to use HDS distribution instead? Just to be clear. Because we are going to have to deal with user questions on this over and over again (I predict). Both educating them on mental change and on what they are supposed to do now with their Tomcat installs. So, this needs to be a prominent explanation as part of the Reference Guide, etc.
          Hide
          Mark Miller added a comment -

          To be clear - Solr is still built upon a war - we still have to ship a war for Solr to work. The user must get a war. This is more of a mental change. We are no longer a war as far as the user is concerned. If you count on that, you are doing something that is possible but not approved. The war is no longer an artifact we publish to maven nor do we have to worry about people running the war in other containers.

          Show
          Mark Miller added a comment - To be clear - Solr is still built upon a war - we still have to ship a war for Solr to work. The user must get a war. This is more of a mental change. We are no longer a war as far as the user is concerned. If you count on that, you are doing something that is possible but not approved. The war is no longer an artifact we publish to maven nor do we have to worry about people running the war in other containers.
          Hide
          Alexandre Rafalovitch added a comment -

          I can still see solr.war under server/webapps directory inside the latest branch_5x built distribution archive (whether tgz or zip).

          Show
          Alexandre Rafalovitch added a comment - I can still see solr.war under server/webapps directory inside the latest branch_5x built distribution archive (whether tgz or zip).
          Hide
          Ramkumar Aiyengar added a comment -

          dist still creates a .war which then gets copied to solr/webapps in the example target. It just doesn't get copied when the tarball is created (i.e. doesn't get 'shipped').

          Side note: I don't see a reason any longer why we output this to dist instead of solr/webapps directly..

          Show
          Ramkumar Aiyengar added a comment - dist still creates a .war which then gets copied to solr/webapps in the example target. It just doesn't get copied when the tarball is created (i.e. doesn't get 'shipped'). Side note: I don't see a reason any longer why we output this to dist instead of solr/webapps directly..
          Hide
          Shawn Heisey added a comment -

          I must admit to being confused. The committed patch seems to indicate that "ant dist" will not create a .war file, but the dist target still created the war on an updated checkout. I checked out branch_5x fresh and wiped the ivy cache, then tried the build again. It created the war again. Do I have the wrong idea?

          Show
          Shawn Heisey added a comment - I must admit to being confused. The committed patch seems to indicate that "ant dist" will not create a .war file, but the dist target still created the war on an updated checkout. I checked out branch_5x fresh and wiped the ivy cache, then tried the build again. It created the war again. Do I have the wrong idea?
          Mark Miller made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Mark Miller added a comment -

          Thanks Ram!

          Show
          Mark Miller added a comment - Thanks Ram!
          Hide
          ASF subversion and git services added a comment -

          Commit 1641993 from Mark Miller in branch 'dev/trunk'
          [ https://svn.apache.org/r1641993 ]

          SOLR-4792: Move CHANGES entry from 6 to 5.

          Show
          ASF subversion and git services added a comment - Commit 1641993 from Mark Miller in branch 'dev/trunk' [ https://svn.apache.org/r1641993 ] SOLR-4792 : Move CHANGES entry from 6 to 5.
          Hide
          ASF subversion and git services added a comment -

          Commit 1641990 from Mark Miller in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1641990 ]

          SOLR-4792: Stop shipping a .war.

          Show
          ASF subversion and git services added a comment - Commit 1641990 from Mark Miller in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1641990 ] SOLR-4792 : Stop shipping a .war.
          Hide
          Ramkumar Aiyengar added a comment -

          Here's the commit on trunk backported to branch_5x. The bin/solr seems to be working on a package created with that (since the war is still deployed to server/webapps/solr.war is still shipped).

          If there are other changes needed, happy to take a shot at it..

          Show
          Ramkumar Aiyengar added a comment - Here's the commit on trunk backported to branch_5x . The bin/solr seems to be working on a package created with that (since the war is still deployed to server/webapps/solr.war is still shipped). If there are other changes needed, happy to take a shot at it..
          Hide
          ASF GitHub Bot added a comment -

          GitHub user andyetitmoves opened a pull request:

          https://github.com/apache/lucene-solr/pull/107

          SOLR-4792: stop shipping a war in 5.0

          Patch for SOLR-4792.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/bloomberg/lucene-solr branch_5x-stop-ship-war

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/lucene-solr/pull/107.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #107


          commit 398f74fcf1a5e92d166a6517452f16a040ccb501
          Author: Robert Muir <rmuir@apache.org>
          Date: 2013-06-08T19:08:17Z

          SOLR-4792: stop shipping a war in 5.0


          Show
          ASF GitHub Bot added a comment - GitHub user andyetitmoves opened a pull request: https://github.com/apache/lucene-solr/pull/107 SOLR-4792 : stop shipping a war in 5.0 Patch for SOLR-4792 . You can merge this pull request into a Git repository by running: $ git pull https://github.com/bloomberg/lucene-solr branch_5x-stop-ship-war Alternatively you can review and apply these changes as the patch at: https://github.com/apache/lucene-solr/pull/107.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #107 commit 398f74fcf1a5e92d166a6517452f16a040ccb501 Author: Robert Muir <rmuir@apache.org> Date: 2013-06-08T19:08:17Z SOLR-4792 : stop shipping a war in 5.0
          Mark Miller made changes -
          Summary stop shipping a war in trunk (6.0) stop shipping a war in 5.0
          Hide
          Mark Miller added a comment -

          Ramkumar Aiyengar brought this up at Lucene Revolution - 5.x could be a long line and this is a fairly simple change internally - it just takes a volunteer - let's do it in 5.x as originally planned and voted on.

          Show
          Mark Miller added a comment - Ramkumar Aiyengar brought this up at Lucene Revolution - 5.x could be a long line and this is a fairly simple change internally - it just takes a volunteer - let's do it in 5.x as originally planned and voted on.
          Mark Miller made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Mark Miller made changes -
          Assignee Robert Muir [ rcmuir ] Mark Miller [ markrmiller@gmail.com ]
          Fix Version/s 5.0 [ 12327845 ]
          Hide
          Mark Miller added a comment -

          The issue title is wrong ... we WILL be shipping a .war in 5.x versions.

          I think we just need to backport this to 5.x. No reason we have to wait until 6x.

          Show
          Mark Miller added a comment - The issue title is wrong ... we WILL be shipping a .war in 5.x versions. I think we just need to backport this to 5.x. No reason we have to wait until 6x.
          Shawn Heisey made changes -
          Link This issue relates to SOLR-6733 [ SOLR-6733 ]
          Hide
          Shawn Heisey added a comment -

          I've been looking for another issue to record some ideas, since this issue has a very narrow focus, and it's resolved. Noble Paul mentioned that he would open an issue to create a standalone app, but I can't seem to find one. I'm willing to create the issue if it doesn't exist.

          Show
          Shawn Heisey added a comment - I've been looking for another issue to record some ideas, since this issue has a very narrow focus, and it's resolved. Noble Paul mentioned that he would open an issue to create a standalone app, but I can't seem to find one. I'm willing to create the issue if it doesn't exist.
          Hide
          Jayson Minard added a comment -

          For 6.0 can we please create issues for the things we DO WANT instead of just "no war" ... as you say, the title here is bad and it isn't an actionable list of issues to reach the goals from the mailing list vote.

          Show
          Jayson Minard added a comment - For 6.0 can we please create issues for the things we DO WANT instead of just "no war" ... as you say, the title here is bad and it isn't an actionable list of issues to reach the goals from the mailing list vote.
          Hide
          Jayson Minard added a comment -

          I agree with the comments from the mailing list, and they are not mutually
          exclusive with a WAR. I'm not sure we need a WAR, but you can always adapt
          to one (as an example, ElasticSearch for example has
          elasticsearch-transport-wares to do this)

          App servers don't prevent better ways of doing configuration, and usually
          everything needs at least a starting point to find a configuration whether
          it is a war or a command-line tool. It can be available for both.

          Not that I'm all pro WAR anyway since I obviously moved away from it
          myself, but there are already solutions that make it easier while it is
          still a WAR. That is just a bundling of everything needed to run...

          Either way, it should be easier to configure and use.

          On Wed, Nov 12, 2014 at 12:14 AM, Shawn Heisey (JIRA) <jira@apache.org>

          Show
          Jayson Minard added a comment - I agree with the comments from the mailing list, and they are not mutually exclusive with a WAR. I'm not sure we need a WAR, but you can always adapt to one (as an example, ElasticSearch for example has elasticsearch-transport-wares to do this) App servers don't prevent better ways of doing configuration, and usually everything needs at least a starting point to find a configuration whether it is a war or a command-line tool. It can be available for both. Not that I'm all pro WAR anyway since I obviously moved away from it myself, but there are already solutions that make it easier while it is still a WAR. That is just a bundling of everything needed to run... Either way, it should be easier to configure and use. On Wed, Nov 12, 2014 at 12:14 AM, Shawn Heisey (JIRA) <jira@apache.org>
          Hide
          Jayson Minard added a comment -

          I agree with the comments from the mailing list, and they are not mutually exclusive with a WAR. I'm not sure we need a WAR, but you can always adapt to one (as an example, ElasticSearch for example has elasticsearch-transport-wares to do this)

          App servers don't prevent better ways of doing configuration, and usually everything needs at least a starting point to find a configuration whether it is a war or a command-line tool. It can be available for both.

          Not that I'm all pro WAR anyway since I obviously moved away from it myself, but there are already solutions that make it easier while it is still a WAR. That is just a bundling of everything needed to run...

          Either way, it should be easier to configure and use.

          Show
          Jayson Minard added a comment - I agree with the comments from the mailing list, and they are not mutually exclusive with a WAR. I'm not sure we need a WAR, but you can always adapt to one (as an example, ElasticSearch for example has elasticsearch-transport-wares to do this) App servers don't prevent better ways of doing configuration, and usually everything needs at least a starting point to find a configuration whether it is a war or a command-line tool. It can be available for both. Not that I'm all pro WAR anyway since I obviously moved away from it myself, but there are already solutions that make it easier while it is still a WAR. That is just a bundling of everything needed to run... Either way, it should be easier to configure and use.
          Shawn Heisey made changes -
          Summary stop shipping a war in 5.0 stop shipping a war in trunk (6.0)
          Hide
          Shawn Heisey added a comment -

          Jayson Minard, the reason we want to move away from the war/servlet deployment has little to do with documentation, or performance. Jetty and Tomcat are not very hard to use, and performance is quite good.

          Here's an incomplete list of some very good reasons for turning Solr into a standalone application rather than a webapp:

          http://mail-archives.apache.org/mod_mbox/lucene-dev/201305.mbox/%3C807476C6-E4C3-4E7E-9F67-2BECB63990DE%40gmail.com%3E

          The issue title is wrong ... we WILL be shipping a .war in 5.x versions. I will update it to 6.0.

          Show
          Shawn Heisey added a comment - Jayson Minard , the reason we want to move away from the war/servlet deployment has little to do with documentation, or performance. Jetty and Tomcat are not very hard to use, and performance is quite good. Here's an incomplete list of some very good reasons for turning Solr into a standalone application rather than a webapp: http://mail-archives.apache.org/mod_mbox/lucene-dev/201305.mbox/%3C807476C6-E4C3-4E7E-9F67-2BECB63990DE%40gmail.com%3E The issue title is wrong ... we WILL be shipping a .war in 5.x versions. I will update it to 6.0.
          Hide
          Jayson Minard added a comment - - edited

          This is unnecessary change. A WAR can be shipped and still be used in embedded runner while still retaining the ability to be configured and used in an application server.

          See: https://github.com/bremeld/solr-undertow

          Uses Undertow.io to create an easy to configure Solr (and SolrCloud) server by extracting the WAR to create the web site and the classpath then executes Solr embedded with a high performance server. And yet still retains the ability to add additional functionality such as request rate limiting (released), auth, ssl, other port configures, and other features in the future due to capabilities of Undertow.

          I think the problem with WAR in the past is that there is little reasonable documentation for current Jetty, current Tomcat, current Wildfly for Solr, and most old resources on the internet are very old and dated on the topics for Solr. A small Doc effort, and killing these old outdated resources would go a long way to fixing that issue.

          Regardless, there are other options that keep a WAR while providing a simple to configure and start server that is as fast or faster than Jetty. (Solr-Undertow uses far fewer threads for the same throughput for one, and its configuration is more sane than the default Jetty configuration has been in the past)

          I am the author of Solr-Undertow but also a user of Solr with experience in almost every app server and deployment model possible; so this small project was a way to fix the problem while retaining war deployment.

          WAR deployment could use a simpler model and less environment variables. Maybe one -D system property to identify a configuration file (URL, that can be downloaded) would be a better model and easier to add to most Application server startups.

          Show
          Jayson Minard added a comment - - edited This is unnecessary change. A WAR can be shipped and still be used in embedded runner while still retaining the ability to be configured and used in an application server. See: https://github.com/bremeld/solr-undertow Uses Undertow.io to create an easy to configure Solr (and SolrCloud) server by extracting the WAR to create the web site and the classpath then executes Solr embedded with a high performance server. And yet still retains the ability to add additional functionality such as request rate limiting (released), auth, ssl, other port configures, and other features in the future due to capabilities of Undertow. I think the problem with WAR in the past is that there is little reasonable documentation for current Jetty, current Tomcat, current Wildfly for Solr, and most old resources on the internet are very old and dated on the topics for Solr. A small Doc effort, and killing these old outdated resources would go a long way to fixing that issue. Regardless, there are other options that keep a WAR while providing a simple to configure and start server that is as fast or faster than Jetty. (Solr-Undertow uses far fewer threads for the same throughput for one, and its configuration is more sane than the default Jetty configuration has been in the past) I am the author of Solr-Undertow but also a user of Solr with experience in almost every app server and deployment model possible; so this small project was a way to fix the problem while retaining war deployment. WAR deployment could use a simpler model and less environment variables. Maybe one -D system property to identify a configuration file (URL, that can be downloaded) would be a better model and easier to add to most Application server startups.
          Hide
          Jan Høydahl added a comment -

          Now that 5.0 -> Trunk, I guess this will happen in 6.x so we should change the title to avoid confusion, perhaps also in CHANGES.txt?

          Show
          Jan Høydahl added a comment - Now that 5.0 -> Trunk, I guess this will happen in 6.x so we should change the title to avoid confusion, perhaps also in CHANGES.txt?
          Show
          Hoss Man added a comment - for easier linking... first message in above mentioned thread... https://mail-archives.apache.org/mod_mbox/lucene-dev/201305.mbox/%3CCAOdYfZU245s6u8ZMwBBHSTkneLVckggxVwCb-eUygoWhtCOiMA%40mail.gmail.com%3E miller's very good list of things currently hamstrung by shipping as a war... https://mail-archives.apache.org/mod_mbox/lucene-dev/201305.mbox/%3C807476C6-E4C3-4E7E-9F67-2BECB63990DE@gmail.com%3E
          Hide
          Noble Paul added a comment -

          Thanks Shawn for pointing me to the list. Seriously, I was sleeping at the wheel

          Mark Miller nicely captured everything I have to say on this subject and I have very little to add . I always wanted Solr to be a standalone app

          +1

          Show
          Noble Paul added a comment - Thanks Shawn for pointing me to the list. Seriously, I was sleeping at the wheel Mark Miller nicely captured everything I have to say on this subject and I have very little to add . I always wanted Solr to be a standalone app +1
          Hide
          Shawn Heisey added a comment -

          Alejandro Abdelnur, thank you for taking a look and giving your input.

          The issue description says to see the vote on the developer list. If you were not a subscriber to the dev list in early May of this year when that vote happened, then here is where you can look at it:

          http://mail-archives.apache.org/mod_mbox/lucene-dev/201305.mbox/browser

          Make sure you are on the Thread view. Click "Next" to go to page 2. Search the page for "webapp" and then you can see/read the many emails with the subject "VOTE: solr no longer webapp". Looking back through the thread, I don't recall seeing a -1 vote from any Lucene/Solr committers. The few -1 votes came from users/contributors.

          It is completely expected that this change will be unpopular with some experienced users. I was initially very skeptical. Mark Miller put forth a rather large list of compelling reasons for switching, so I changed my vote from -0 to +1. Many of good reasons seem like they are developer-only things ... but when the developers find themselves seriously constrained, the users will not see rapid development of the features that they want, so reducing developer pain is good for end users.

          Since that vote, at least once a week I seem to come across a problem that would not exist (or would be very easy to fix) if Solr were a self-contained application. I can't think of any examples right now.

          Show
          Shawn Heisey added a comment - Alejandro Abdelnur , thank you for taking a look and giving your input. The issue description says to see the vote on the developer list. If you were not a subscriber to the dev list in early May of this year when that vote happened, then here is where you can look at it: http://mail-archives.apache.org/mod_mbox/lucene-dev/201305.mbox/browser Make sure you are on the Thread view. Click "Next" to go to page 2. Search the page for "webapp" and then you can see/read the many emails with the subject "VOTE: solr no longer webapp". Looking back through the thread, I don't recall seeing a -1 vote from any Lucene/Solr committers. The few -1 votes came from users/contributors. It is completely expected that this change will be unpopular with some experienced users. I was initially very skeptical. Mark Miller put forth a rather large list of compelling reasons for switching, so I changed my vote from -0 to +1. Many of good reasons seem like they are developer-only things ... but when the developers find themselves seriously constrained, the users will not see rapid development of the features that they want, so reducing developer pain is good for end users. Since that vote, at least once a week I seem to come across a problem that would not exist (or would be very easy to fix) if Solr were a self-contained application. I can't think of any examples right now.
          Hide
          Noble Paul added a comment -

          W1 gives the flexibility of choosing a servlet container, and upgrading it independently of the application

          it gives flexibility as well as it adds complexity to deployment. Every shop has a preference on the web container they use and this leads to an explosion of the permutations possible. This leads to confusion on how to upgrade things, how to add libs to classpath etc. When someone posts a question as to how to do 'x' in appserver 'y' we will be scrambling to get a copy of that appserver and reproducing the situation

          W3 leaves completely out of scope webserver configuration from the application.

          This is a valid point. But the Solr is not a regular webserver. It just needs a subset of properties which webservers normally expose. We probably will only have a handful of configurable properties for Solr (from the app/webserver side)

          Also, depending how the webapp components are declared (servlets, filters) things can get messy.

          We always don't need a webserver. The entire play web framework is written without a servlet container. Even Solr does not need a servlet container. It is happy as long as it can expose itself through an HTTP interface. I personally believe Servlet Container is an overkill for Solr.

          AFAIK ElasticSearch does not ship as a .war . Not shipping a .war adds to a lot of simplicity .

          Show
          Noble Paul added a comment - W1 gives the flexibility of choosing a servlet container, and upgrading it independently of the application it gives flexibility as well as it adds complexity to deployment. Every shop has a preference on the web container they use and this leads to an explosion of the permutations possible. This leads to confusion on how to upgrade things, how to add libs to classpath etc. When someone posts a question as to how to do 'x' in appserver 'y' we will be scrambling to get a copy of that appserver and reproducing the situation W3 leaves completely out of scope webserver configuration from the application. This is a valid point. But the Solr is not a regular webserver. It just needs a subset of properties which webservers normally expose. We probably will only have a handful of configurable properties for Solr (from the app/webserver side) Also, depending how the webapp components are declared (servlets, filters) things can get messy. We always don't need a webserver. The entire play web framework is written without a servlet container. Even Solr does not need a servlet container. It is happy as long as it can expose itself through an HTTP interface. I personally believe Servlet Container is an overkill for Solr. AFAIK ElasticSearch does not ship as a .war . Not shipping a .war adds to a lot of simplicity .
          Hide
          Alejandro Abdelnur added a comment -

          I'd like to provide unsolicited feedback on going from a WAR to an embedded deployment model. The following is based on my experience with Oozie (WAR) and Hadoop (embedded).

          WAR model:

          • W1. it runs in any servlet container
          • W2. it requires bundling servlet container to run out of the box
          • W3. configuration of the webserver is independent of the application configuration

          Embedded model:

          • E1. it runs in a bundled servlet container
          • E2. it bundles servlet container container code
          • E3. the hosting application must configure the webserver

          W1 gives the flexibility of choosing a servlet container, and upgrading it independently of the application. E1 requires a new release of the application.

          W2 makes the binary packaging of the application fatter and more complex. E2 streamlines the binary and simplifies the packaging.

          W3 leaves completely out of scope webserver configuration from the application. For example: memory, threadpool serving incoming HTTP connections, security configuration (HTTPS). E3 requires the application to take care of all configuration of the 'webserver'.

          Also, depending how the webapp components are declared (servlets, filters) things can get messy. For example, how Hadoop HttpServer class registers servlets and filters programmatically is a big mess. If you go this path, I would strongly suggest keeping web.xml around as the place where you define your webapp components (embedded Jetty can load that).

          I don't know the trigger motivation from moving away from a WAR, but in my experience the WAR model has always worked well.

          Hope this helps one way or the other.

          Show
          Alejandro Abdelnur added a comment - I'd like to provide unsolicited feedback on going from a WAR to an embedded deployment model. The following is based on my experience with Oozie (WAR) and Hadoop (embedded). WAR model: W1. it runs in any servlet container W2. it requires bundling servlet container to run out of the box W3. configuration of the webserver is independent of the application configuration Embedded model: E1. it runs in a bundled servlet container E2. it bundles servlet container container code E3. the hosting application must configure the webserver W1 gives the flexibility of choosing a servlet container, and upgrading it independently of the application. E1 requires a new release of the application. W2 makes the binary packaging of the application fatter and more complex. E2 streamlines the binary and simplifies the packaging. W3 leaves completely out of scope webserver configuration from the application. For example: memory, threadpool serving incoming HTTP connections, security configuration (HTTPS). E3 requires the application to take care of all configuration of the 'webserver'. Also, depending how the webapp components are declared (servlets, filters) things can get messy. For example, how Hadoop HttpServer class registers servlets and filters programmatically is a big mess. If you go this path, I would strongly suggest keeping web.xml around as the place where you define your webapp components (embedded Jetty can load that). I don't know the trigger motivation from moving away from a WAR, but in my experience the WAR model has always worked well. Hope this helps one way or the other.
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] sarowe
          http://svn.apache.org/viewvc?view=revision&revision=1491970

          SOLR-4792: Skip target '-validate-maven-dependencies' in solr/webapp/

          Show
          Commit Tag Bot added a comment - [trunk commit] sarowe http://svn.apache.org/viewvc?view=revision&revision=1491970 SOLR-4792 : Skip target '-validate-maven-dependencies' in solr/webapp/
          Hide
          Steve Rowe added a comment - - edited

          ant validate-maven-dependencies is failing now (and has failed a couple of Jenkins runs), because the -validate-maven-dependencies target in solr/build.xml calls -validate-maven-dependencies in solr/webapp, but there is no longer a pom.xml file for that dir, so the target, and the build, fails.

          When I add the following to solr/webapp/build.xml, the build succeeds:

            <!-- nothing to do -->
            <target name="-validate-maven-dependencies"/>
          

          Committing shortly.

          Show
          Steve Rowe added a comment - - edited ant validate-maven-dependencies is failing now (and has failed a couple of Jenkins runs), because the -validate-maven-dependencies target in solr/build.xml calls -validate-maven-dependencies in solr/webapp , but there is no longer a pom.xml file for that dir, so the target, and the build, fails. When I add the following to solr/webapp/build.xml , the build succeeds: <!-- nothing to do --> <target name= "-validate-maven-dependencies" /> Committing shortly.
          Hide
          Noble Paul added a comment -

          I think people should create new JIRA issues for other ideas around this change.

          I shall open an issue for the Solr as a standalone java app.

          Show
          Noble Paul added a comment - I think people should create new JIRA issues for other ideas around this change. I shall open an issue for the Solr as a standalone java app.
          Hide
          Mark Miller added a comment -

          This issue is simply about not shipping a war anymore - I think people should create new JIRA issues for other ideas around this change.

          Show
          Mark Miller added a comment - This issue is simply about not shipping a war anymore - I think people should create new JIRA issues for other ideas around this change.
          Hide
          Noble Paul added a comment -

          I'm thinking that more and more users will move to solrcloud and the conf dir is not even required for cloud. As you suggested, the solr conf can live at the instanceDir wherever it is configured in the solr.solr.home attribute . but the rest can be same

          Solr shold startup an embedded jetty and manage the lifecycle completely.

          Or, alternately

          I suggest that we move to something like netty for http and gain the performance benefits also and we probably will be able to offer async request like elastic provides

          Show
          Noble Paul added a comment - I'm thinking that more and more users will move to solrcloud and the conf dir is not even required for cloud. As you suggested, the solr conf can live at the instanceDir wherever it is configured in the solr.solr.home attribute . but the rest can be same Solr shold startup an embedded jetty and manage the lifecycle completely. Or, alternately I suggest that we move to something like netty for http and gain the performance benefits also and we probably will be able to offer async request like elastic provides
          Hide
          Shawn Heisey added a comment -
          solr.x.x
             |-----lib
             |-----conf (solr.properties, solrconfig.xml, schema.xml ,etc)
             |-----bin (start.sh, stop.sh, start-cloud.sh etc)
             |-----data (configure a different value from the conf dir)
          

          This layout looks pretty good for a setup that uses config sets. It is also a good layout for the single-core mode that was common back in the 1.x days but still supported in 4.x. IMHO we should drop support for that mode in 5.0.

          For multicore with discovery as it exists in 4.4, it's not a good layout. You'd want to have a directory for what is currently called solr.solr.home, which would contain solr.xml. Under that you would have directories for each core, each of which would have data and possibly conf.

          We also have to figure out where to put bits like example and the source code.

          Show
          Shawn Heisey added a comment - solr.x.x |-----lib |-----conf (solr.properties, solrconfig.xml, schema.xml ,etc) |-----bin (start.sh, stop.sh, start-cloud.sh etc) |-----data (configure a different value from the conf dir) This layout looks pretty good for a setup that uses config sets. It is also a good layout for the single-core mode that was common back in the 1.x days but still supported in 4.x. IMHO we should drop support for that mode in 5.0. For multicore with discovery as it exists in 4.4, it's not a good layout. You'd want to have a directory for what is currently called solr.solr.home, which would contain solr.xml. Under that you would have directories for each core, each of which would have data and possibly conf. We also have to figure out where to put bits like example and the source code.
          Hide
          Noble Paul added a comment - - edited

          What will be the directory structure in the new ditribution?
          What will be the start command? How will you configure the web server properties (thread count, http properties etc)

          I personally believe we should make our distro like any other servers where the distro is a tar/zip file on extracting we should get something like

          solr.x.x
             |-----lib
             |-----conf (solr.properties, solrconfig.xml, schema.xml ,etc)
             |-----bin (start.sh, stop.sh, start-cloud.sh etc)
             |-----data (configure a different value from the conf dir)
          
          Show
          Noble Paul added a comment - - edited What will be the directory structure in the new ditribution? What will be the start command? How will you configure the web server properties (thread count, http properties etc) I personally believe we should make our distro like any other servers where the distro is a tar/zip file on extracting we should get something like solr.x.x |-----lib |-----conf (solr.properties, solrconfig.xml, schema.xml ,etc) |-----bin (start.sh, stop.sh, start-cloud.sh etc) |-----data (configure a different value from the conf dir)
          Robert Muir made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Robert Muir made changes -
          Link This issue incorporates SOLR-3890 [ SOLR-3890 ]
          Hide
          Robert Muir added a comment -

          Uwe: those ideas sound great!

          I was afraid to try to do anything sophisticated here: because i dont want to break the example. But i definitely wanted any .war to ONLY be under example/

          Show
          Robert Muir added a comment - Uwe: those ideas sound great! I was afraid to try to do anything sophisticated here: because i dont want to break the example. But i definitely wanted any .war to ONLY be under example/
          Hide
          Uwe Schindler added a comment - - edited

          As a second setp, we should remove the war also from the example folder. The jetty in the example folder can use the classpath directly and put the JAR files into the webapp. No need to ship a war file, because the Jetty container will unpack it while starting for the first time (to /tmp), which is horrible by itsself. If we ship a sole "application" we should place all files that are needed to run by solr in the jetty classpath. In a later step we can remove the "webapp" by itsself and provide a separate "Lucene" start.jar that starts up the whole application with jetty (only embedded as HTTP layer, no longer using webapp contexts), configured as needed.

          Show
          Uwe Schindler added a comment - - edited As a second setp, we should remove the war also from the example folder. The jetty in the example folder can use the classpath directly and put the JAR files into the webapp. No need to ship a war file, because the Jetty container will unpack it while starting for the first time (to /tmp), which is horrible by itsself. If we ship a sole "application" we should place all files that are needed to run by solr in the jetty classpath. In a later step we can remove the "webapp" by itsself and provide a separate "Lucene" start.jar that starts up the whole application with jetty (only embedded as HTTP layer, no longer using webapp contexts), configured as needed.
          Hide
          Robert Muir added a comment -

          This also saves ~ 20MBytes from the binary distribution. Previously it contained the .war twice: once in dist/ and once in example/

          Show
          Robert Muir added a comment - This also saves ~ 20MBytes from the binary distribution. Previously it contained the .war twice: once in dist/ and once in example/
          Hide
          Mark Miller added a comment -

          Thanks for starting this!

          Show
          Mark Miller added a comment - Thanks for starting this!
          Robert Muir made changes -
          Field Original Value New Value
          Attachment SOLR-4792.patch [ 12582101 ]
          Hide
          Robert Muir added a comment -

          First stab at a patch. Its very simple.

          • remove .war from dist/ in the binary distribution (its still in example/, was duplicated before)
          • remove .war maven artifact
          • small doc changes removing warness.
          Show
          Robert Muir added a comment - First stab at a patch. Its very simple. remove .war from dist/ in the binary distribution (its still in example/, was duplicated before) remove .war maven artifact small doc changes removing warness.
          Robert Muir created issue -

            People

            • Assignee:
              Mark Miller
              Reporter:
              Robert Muir
            • Votes:
              0 Vote for this issue
              Watchers:
              30 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development