Solr
  1. Solr
  2. SOLR-1602

Refactor SOLR package structure to include o.a.solr.response and move QueryResponseWriters in there

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.2, 1.3, 1.4
    • Fix Version/s: 3.1, 4.0-ALPHA
    • Component/s: Response Writers
    • Labels:
      None
    • Environment:

      independent of environment (code structure)

      Description

      Currently all o.a.solr.request.QueryResponseWriter implementations are curiously located in the o.a.solr.request package. Not only is this package getting big (30+ classes), a lot of them are misplaced. There should be a first-class o.a.solr.response package, and the response related classes should be given a home there. Patch forthcoming.

      1. SOLR-1602.Mattmann.112509_02.patch.txt
        229 kB
        Chris A. Mattmann
      2. SOLR-1602.Mattmann.112509.patch.txt
        225 kB
        Chris A. Mattmann
      3. SOLR-1602.Mattmann.final.050810.patch.txt
        10 kB
        Chris A. Mattmann
      4. SOLR-1602.Mattmann.wrapup.031010.patch.txt
        16 kB
        Chris A. Mattmann
      5. upgrade_solr_config
        1 kB
        Chris A. Mattmann

        Activity

        Hide
        Chris A. Mattmann added a comment -
        • moves all writers from o.a.solr.request into o.a.solr.response
        • moves SolrQueryResponse into o.a.solr.response
        • updates unit tests and rest of code to have correct references
        • updates embedded solr code inside webapp to have correct references
        Show
        Chris A. Mattmann added a comment - moves all writers from o.a.solr.request into o.a.solr.response moves SolrQueryResponse into o.a.solr.response updates unit tests and rest of code to have correct references updates embedded solr code inside webapp to have correct references
        Hide
        Chris A. Mattmann added a comment -

        This patch also:

        • creates an o.a.solr.response test package, and moves response writer related (and response related) tests into that package
        • updates the solrconfig.xml file to ref the proper package structure
        Show
        Chris A. Mattmann added a comment - This patch also: creates an o.a.solr.response test package, and moves response writer related (and response related) tests into that package updates the solrconfig.xml file to ref the proper package structure
        Hide
        Chris A. Mattmann added a comment -
        • newest patch includes updates to all contrib classes that ensure they work with the new package structure
        Show
        Chris A. Mattmann added a comment - newest patch includes updates to all contrib classes that ensure they work with the new package structure
        Hide
        patrick o'leary added a comment -

        total +1

        Makes it a pain to have to use find to code, and response seems like the right place for the writers.

        Show
        patrick o'leary added a comment - total +1 Makes it a pain to have to use find to code, and response seems like the right place for the writers.
        Hide
        Ryan McKinley added a comment -

        Sounds fine... except for the back compatibility issues – especially for people upgrading with the same solrconfig.xml

        When we moved all the handlers to a new package o.a.solr.handler, it left a bunch of deprecated calsses in o.a.solr.request:

        @Deprecated 
        public class StandardRequestHandler extends org.apache.solr.handler.StandardRequestHandler {
         // Don't use this class
        }
        

        Also, if we make a 'response' package, seems SolrQueryResponse.java should go there.

        Show
        Ryan McKinley added a comment - Sounds fine... except for the back compatibility issues – especially for people upgrading with the same solrconfig.xml When we moved all the handlers to a new package o.a.solr.handler, it left a bunch of deprecated calsses in o.a.solr.request: @Deprecated public class StandardRequestHandler extends org.apache.solr.handler.StandardRequestHandler { // Don't use this class } Also, if we make a 'response' package, seems SolrQueryResponse.java should go there.
        Hide
        Chris A. Mattmann added a comment - - edited

        Hi Ryan:

        Thanks.

        Sounds fine... except for the back compatibility issues - especially for people upgrading with the same solrconfig.xml

        There are 8 response writers defined by default in solrconfig.xml. That doesn't seem too unwieldy a job for find/replace as far as configuration upgrades for someone moving from SOLR x.y to SOLR 1.5.

        As for code, yes, unfortunately users will have to recompile their response writers and update a few package names/imports per response writer which they'd probably want/have to do anyways on an upgrade regardless.

        When we moved all the handlers to a new package o.a.solr.handler, it left a bunch of deprecated calsses in o.a.solr.request.

        You could certainly do this, and I've done it before in other projects. The tradeoff is, what type of message from the Java compiler do you want to notify you as a consumer of the SOLR java classes:

        1. A deprecation (that could easily get swallowed if someone compiles with deprecation notifications off)
        2. A compiler error, forcing the user to perform the small amount of legwork to update package refs

        I'm a fan of number 2, and I'd venture to guess the work wouldn't be too bad in this case since most of the ReponseWriters aren't friendly to user extension or sub-classing.

        Also, if we make a 'response' package, seems SolrQueryResponse.java should go there.

        Agreed, the patch I attached should move it there.

        Show
        Chris A. Mattmann added a comment - - edited Hi Ryan: Thanks. Sounds fine... except for the back compatibility issues - especially for people upgrading with the same solrconfig.xml There are 8 response writers defined by default in solrconfig.xml. That doesn't seem too unwieldy a job for find/replace as far as configuration upgrades for someone moving from SOLR x.y to SOLR 1.5. As for code, yes, unfortunately users will have to recompile their response writers and update a few package names/imports per response writer which they'd probably want/have to do anyways on an upgrade regardless. When we moved all the handlers to a new package o.a.solr.handler, it left a bunch of deprecated calsses in o.a.solr.request. You could certainly do this, and I've done it before in other projects. The tradeoff is, what type of message from the Java compiler do you want to notify you as a consumer of the SOLR java classes: A deprecation (that could easily get swallowed if someone compiles with deprecation notifications off) A compiler error, forcing the user to perform the small amount of legwork to update package refs I'm a fan of number 2, and I'd venture to guess the work wouldn't be too bad in this case since most of the ReponseWriters aren't friendly to user extension or sub-classing. Also, if we make a 'response' package, seems SolrQueryResponse.java should go there. Agreed, the patch I attached should move it there.
        Hide
        Ryan McKinley added a comment -
        I'm a fan of number 2, and I'd venture to guess the work wouldn't be too bad in this case since most of the ReponseWriters aren't friendly to user extension or sub-classing.

        +1

        but since this is a breaking change – that would need explicitly called out in CHANGES.txt, we should get pretty wide consensus before moving forward...

        Show
        Ryan McKinley added a comment - I'm a fan of number 2, and I'd venture to guess the work wouldn't be too bad in this case since most of the ReponseWriters aren't friendly to user extension or sub-classing. +1 but since this is a breaking change – that would need explicitly called out in CHANGES.txt, we should get pretty wide consensus before moving forward...
        Hide
        Chris A. Mattmann added a comment -

        Hi Ryan:

        but since this is a breaking change - that would need explicitly called out in CHANGES.txt, we should get pretty wide consensus before moving forward...

        +1, I'll call a vote.

        Cheers,
        Chris

        Show
        Chris A. Mattmann added a comment - Hi Ryan: but since this is a breaking change - that would need explicitly called out in CHANGES.txt, we should get pretty wide consensus before moving forward... +1, I'll call a vote. Cheers, Chris
        Hide
        Noble Paul added a comment -

        the patch does not apply. iis it not updated to trunk?

        Show
        Noble Paul added a comment - the patch does not apply. iis it not updated to trunk?
        Hide
        Chris A. Mattmann added a comment -

        Hi Noble,

        the patch does not apply. iis it not updated to trunk?

        Not sure, what message you getting? I'll take a look. I posted it originally in November so it's entirely possible it's out of date.

        Cheers,
        Chris

        Show
        Chris A. Mattmann added a comment - Hi Noble, the patch does not apply. iis it not updated to trunk? Not sure, what message you getting? I'll take a look. I posted it originally in November so it's entirely possible it's out of date. Cheers, Chris
        Hide
        Ryan McKinley added a comment -
        the patch does not apply. iis it not updated to trunk?

        I've never had good luck with patches for moving files. Even if it applies correctly, if you commit the patch, the svn history does not show that the file was moved. (unless this has been fixed in the last year since i looked)

        For refactoring like this, whoever commits this will probably need to make the changes directly.

        Show
        Ryan McKinley added a comment - the patch does not apply. iis it not updated to trunk? I've never had good luck with patches for moving files. Even if it applies correctly, if you commit the patch, the svn history does not show that the file was moved. (unless this has been fixed in the last year since i looked) For refactoring like this, whoever commits this will probably need to make the changes directly.
        Hide
        Chris A. Mattmann added a comment -

        Hey Ryan,

        You are exactly right. When I created the patch in Eclipse, it negated to do the add +++ part of the patch b/c it saw the files as getting moved. Note that's why the patch doesn't apply – it tries to make changes to the response writer class packages based on the path org/apache/solr/response/<WriterName>.java, which doesn't exist yet.

        To apply this patch:

        1. first move the response writers into a new folder (response). Also copy SolrQueryResponse there.
        2. apply the patch

        Cheers,
        Chris

        Show
        Chris A. Mattmann added a comment - Hey Ryan, You are exactly right. When I created the patch in Eclipse, it negated to do the add +++ part of the patch b/c it saw the files as getting moved. Note that's why the patch doesn't apply – it tries to make changes to the response writer class packages based on the path org/apache/solr/response/<WriterName>.java, which doesn't exist yet. To apply this patch: 1. first move the response writers into a new folder (response). Also copy SolrQueryResponse there. 2. apply the patch Cheers, Chris
        Hide
        Hoss Man added a comment -

        I've got no strong opinions about moving the code (it would probably be easier to understand if we changed it, but so many people use IDEs these days that i odn't know if it matters) but if we do change it i'd prefer to go the deprecation route just out of consistency with how we've dealt with the RequestHandlers and other utilities classes in the past – it's relatively trivial to do, so there's very little down side – and while it's true someone w/ deprecation warnings turned off probably won't notice – that's kind of the point behind doing deprecations, you get them if you want, you ignore them if you don't – but things don't break.

        additionally: the config file issue should not be downplayed. yes it would be a trivial search/replace, but that's precisely the reason why it would aggravate users: because it would be such a trivial change w/o any obvious benefit to the users.

        (novice developers tend to be much more forgiving of eccentricities in the code base then novice users are of upgrade incompatibilities)

        Show
        Hoss Man added a comment - I've got no strong opinions about moving the code (it would probably be easier to understand if we changed it, but so many people use IDEs these days that i odn't know if it matters) but if we do change it i'd prefer to go the deprecation route just out of consistency with how we've dealt with the RequestHandlers and other utilities classes in the past – it's relatively trivial to do, so there's very little down side – and while it's true someone w/ deprecation warnings turned off probably won't notice – that's kind of the point behind doing deprecations, you get them if you want, you ignore them if you don't – but things don't break. additionally: the config file issue should not be downplayed. yes it would be a trivial search/replace, but that's precisely the reason why it would aggravate users: because it would be such a trivial change w/o any obvious benefit to the users. (novice developers tend to be much more forgiving of eccentricities in the code base then novice users are of upgrade incompatibilities)
        Hide
        Chris A. Mattmann added a comment -

        I've got no strong opinions about moving the code (it would probably be easier to understand if we changed it, but so many people use IDEs these days that i odn't know if it matters) but if we do change it i'd prefer to go the deprecation route just out of consistency with how we've dealt with the RequestHandlers and other utilities classes in the past - it's relatively trivial to do, so there's very little down side - and while it's true someone w/ deprecation warnings turned off probably won't notice - that's kind of the point behind doing deprecations, you get them if you want, you ignore them if you don't - but things don't break.

        Thanks for the comments Hoss. As for deprecations I'm totally for them, when the intention is to limit the impact on classes and code that has infected a code base and the sheer impact of changing all of the package imports etc. is so burndensome that you want to give someone some time to do it, while still moving forward with the refactoring. I don't think that's the case here. ResponseWriters aren't extensible as we've went back and forth all the time about. I doubt many people have extended them. As far as writing their own, the code is likely in their own package structure. So, I think in this case, despite being different than what you guys have done before (which is a con), the pro is the change is so minute and likely of little impact, we want the compiler to throw an error or 2, so the user can fix those one or 2 and be set for all future releases.

        additionally: the config file issue should not be downplayed. yes it would be a trivial search/replace, but that's precisely the reason why it would aggravate users: because it would be such a trivial change w/o any obvious benefit to the users.

        (novice developers tend to be much more forgiving of eccentricities in the code base then novice users are of upgrade incompatibilities)

        I'm just not seeing this. I'm a user of plenty of software and a developer of the same. If someone told me I'd have to do a find/replace on a config file to take advantage of a new version of software, and that find replace would have the impact of 7 or 8 lines which I probably haven't touched in the config file anyways I really wouldn't care (in other words the benefits far outweigh the negatives). Though this is a generalization, I'd say on average, customers for software I've developed over the last 10 years really haven't either.

        If it makes you feel better I could strip out and upload a small patch that only changes the sorlconfig.xml file part of this issue, and then in CHANGES.txt we could reference this issue and tell the users, OK here's a quick way to upgrade an existing deployment:

        patch -p0 < curl "https://issues.apache.org/jira/secure/attachment/12426188/SOLR-1602.configonly.patch.txt"

        or something like that?

        Oh, also I opened up a vote thread for this. If you get a chance could you vote on it? Thanks a lot Hoss.

        Show
        Chris A. Mattmann added a comment - I've got no strong opinions about moving the code (it would probably be easier to understand if we changed it, but so many people use IDEs these days that i odn't know if it matters) but if we do change it i'd prefer to go the deprecation route just out of consistency with how we've dealt with the RequestHandlers and other utilities classes in the past - it's relatively trivial to do, so there's very little down side - and while it's true someone w/ deprecation warnings turned off probably won't notice - that's kind of the point behind doing deprecations, you get them if you want, you ignore them if you don't - but things don't break. Thanks for the comments Hoss. As for deprecations I'm totally for them, when the intention is to limit the impact on classes and code that has infected a code base and the sheer impact of changing all of the package imports etc. is so burndensome that you want to give someone some time to do it, while still moving forward with the refactoring. I don't think that's the case here. ResponseWriters aren't extensible as we've went back and forth all the time about. I doubt many people have extended them. As far as writing their own, the code is likely in their own package structure. So, I think in this case, despite being different than what you guys have done before (which is a con), the pro is the change is so minute and likely of little impact, we want the compiler to throw an error or 2, so the user can fix those one or 2 and be set for all future releases. additionally: the config file issue should not be downplayed. yes it would be a trivial search/replace, but that's precisely the reason why it would aggravate users: because it would be such a trivial change w/o any obvious benefit to the users. (novice developers tend to be much more forgiving of eccentricities in the code base then novice users are of upgrade incompatibilities) I'm just not seeing this. I'm a user of plenty of software and a developer of the same. If someone told me I'd have to do a find/replace on a config file to take advantage of a new version of software, and that find replace would have the impact of 7 or 8 lines which I probably haven't touched in the config file anyways I really wouldn't care (in other words the benefits far outweigh the negatives). Though this is a generalization, I'd say on average, customers for software I've developed over the last 10 years really haven't either. If it makes you feel better I could strip out and upload a small patch that only changes the sorlconfig.xml file part of this issue, and then in CHANGES.txt we could reference this issue and tell the users, OK here's a quick way to upgrade an existing deployment: patch -p0 < curl "https://issues.apache.org/jira/secure/attachment/12426188/SOLR-1602.configonly.patch.txt" or something like that? Oh, also I opened up a vote thread for this. If you get a chance could you vote on it? Thanks a lot Hoss.
        Hide
        Erik Hatcher added a comment -

        Chris - I'm with Hoss on this one. -1 to simply moving without keeping the old classes and deprecating them.

        Your rationalization only makes sense for developers that are compiling things... consider the case where someone has enabled the Velocity response writer (since it isn't yet enabled by default) and then it gets moved. Their upgrade is going to break for no good reason.

        We must maintain the current classes and deprecate them in order to move them.

        Show
        Erik Hatcher added a comment - Chris - I'm with Hoss on this one. -1 to simply moving without keeping the old classes and deprecating them. Your rationalization only makes sense for developers that are compiling things... consider the case where someone has enabled the Velocity response writer (since it isn't yet enabled by default) and then it gets moved. Their upgrade is going to break for no good reason. We must maintain the current classes and deprecate them in order to move them.
        Hide
        Chris A. Mattmann added a comment - - edited

        Hi Erik,

        Thanks for your feedback. My comments below:

        Your rationalization only makes sense for developers that are compiling things... consider the case where someone has enabled the Velocity response writer (since it isn't yet enabled by default) and then it gets moved. Their upgrade is going to break for no good reason.

        I'm sorry but I guess I feel it's more important to be strict and informative, rather than loose and accommodating on these types of changes. Let's elucidate a real scenario.

        1. I'm a SOLR 1.4 user.
        2. I'm using the Velocity Response Writer, which means, I've gone to the trouble of making a small edit to my solrconfig.xml configuration, and added a queryResponseWriter by enabling it with a few lines of XML in my configuration.
        3. I decide I'm going to use SOLR 1.5.
        (corollary) SOLR-1602 goes in as I've proposed, with no deprecations
        4. I upgrade to SOLR 1.5.

        • user A is the type that doesn't bother reading CHANGES.txt. Ideally this isn't the user you want to design for, but you as a development team are willing to accommodate these types of users, even though it's highly encouraged that they read CHANGES.txt on any upgrades or when they download the software.
        • user B is the type that believes it makes sense to read CHANGES.txt first. (clap clap)
        • let's take the example of user A: in this case, the guy goes to run a query and ask for a response with wt=veolcity and some selected template. The query breaks, with a SOLR error, with an informative message to the likes of: response writer o.a.solr.request.VelocityResponseWriter doesn't exist. I'm annoyed, but I say, hmm why would this not work anymore. Let me take a look to see what response writers are present on the classpath with a jar tvf (if I'm not savvy enough to do this then we've got a whole other problem on our hands). Even if this approach doesn't pan out, I'd argue user A is still savvy enough at that point to, read_the_instruction_manual. In doing so, he finds the answer to his question. It's reactive rather than proactive, but we still arrive at a solution with a small amount of acceptable effort. (BTW in this case, user A is annoyed for what he believes to be "no good reason", but I can tell you from experience in government, research/academia and industry, this happens all the time. As developers, and good designers there is a good reason [better organization for a small user A annoyance cost]).
        • let's take the example of user B. It's a trivial subcase of user A's scenario, to the point where the user reads CHANGES.txt, notes the information in there, and proactively updates a few lines of XML in his configuration file

        So, anyways, I hope I've made my point and I don't really ever seem to get a lot of support on these types of issues, regardless. As a software architect and someone that teaches a graduate course in this stuff, I can tell you putting things titled ResponseWriter in a request package is not the way to go, regardless if SOLR has made 20 releases so far or 1 (software architecture recovery, static analysis, and a host of other issues are affected by this. Imagine in 10 years a company has a SOLR deployment long after SOLR ++ or some-other-search-service +++ 2.0 has come out. Someone has to come in and recover the architecture of a SOLR system so that they can understand how the changes they'd like to make to evolve the system will impact the so-called "load bearing walls". These types of issues, if caught early on, help reduce this effort and ultimately reduce cost.)

        Cheers,
        Chris

        Show
        Chris A. Mattmann added a comment - - edited Hi Erik, Thanks for your feedback. My comments below: Your rationalization only makes sense for developers that are compiling things... consider the case where someone has enabled the Velocity response writer (since it isn't yet enabled by default) and then it gets moved. Their upgrade is going to break for no good reason. I'm sorry but I guess I feel it's more important to be strict and informative, rather than loose and accommodating on these types of changes. Let's elucidate a real scenario. 1. I'm a SOLR 1.4 user. 2. I'm using the Velocity Response Writer, which means, I've gone to the trouble of making a small edit to my solrconfig.xml configuration, and added a queryResponseWriter by enabling it with a few lines of XML in my configuration. 3. I decide I'm going to use SOLR 1.5. (corollary) SOLR-1602 goes in as I've proposed, with no deprecations 4. I upgrade to SOLR 1.5. user A is the type that doesn't bother reading CHANGES.txt. Ideally this isn't the user you want to design for, but you as a development team are willing to accommodate these types of users, even though it's highly encouraged that they read CHANGES.txt on any upgrades or when they download the software. user B is the type that believes it makes sense to read CHANGES.txt first. (clap clap) let's take the example of user A: in this case, the guy goes to run a query and ask for a response with wt=veolcity and some selected template. The query breaks, with a SOLR error, with an informative message to the likes of: response writer o.a.solr.request.VelocityResponseWriter doesn't exist. I'm annoyed, but I say, hmm why would this not work anymore. Let me take a look to see what response writers are present on the classpath with a jar tvf (if I'm not savvy enough to do this then we've got a whole other problem on our hands). Even if this approach doesn't pan out, I'd argue user A is still savvy enough at that point to, read_the_instruction_manual . In doing so, he finds the answer to his question. It's reactive rather than proactive, but we still arrive at a solution with a small amount of acceptable effort. (BTW in this case, user A is annoyed for what he believes to be "no good reason", but I can tell you from experience in government, research/academia and industry, this happens all the time. As developers, and good designers there is a good reason [better organization for a small user A annoyance cost] ). let's take the example of user B. It's a trivial subcase of user A's scenario, to the point where the user reads CHANGES.txt, notes the information in there, and proactively updates a few lines of XML in his configuration file So, anyways, I hope I've made my point and I don't really ever seem to get a lot of support on these types of issues, regardless. As a software architect and someone that teaches a graduate course in this stuff, I can tell you putting things titled ResponseWriter in a request package is not the way to go, regardless if SOLR has made 20 releases so far or 1 (software architecture recovery, static analysis, and a host of other issues are affected by this. Imagine in 10 years a company has a SOLR deployment long after SOLR ++ or some-other-search-service +++ 2.0 has come out. Someone has to come in and recover the architecture of a SOLR system so that they can understand how the changes they'd like to make to evolve the system will impact the so-called "load bearing walls". These types of issues, if caught early on, help reduce this effort and ultimately reduce cost.) Cheers, Chris
        Hide
        Chris A. Mattmann added a comment -

        Furthermore, to add one small corollary to my last point: the reason I'm pushing so strongly for no deprecations is that I'd like to kill 2 birds with one stone: address those such as user A or B above (only configuration updates), as well as addressing those users who are compiling and building a SOLR upgrade too.

        Anyhoo, Happy New Years, everyone! I'll be back around tomorrow.

        Show
        Chris A. Mattmann added a comment - Furthermore, to add one small corollary to my last point: the reason I'm pushing so strongly for no deprecations is that I'd like to kill 2 birds with one stone: address those such as user A or B above (only configuration updates), as well as addressing those users who are compiling and building a SOLR upgrade too. Anyhoo, Happy New Years, everyone! I'll be back around tomorrow.
        Hide
        Erik Hatcher added a comment -

        Chris - you sure do go to an extreme of lots of typing to try to make your case when you've got two committers (and likely a user community consensus) saying deprecation is required. It's easy enough, and really makes a big difference to the sysadmin that'll be the one to deploy a new version of Solr.

        Consider the dramatic changes that go on with the Lucene index format itself, and it remains backwards compatible for a version or more. What if we told users they had to reindex everything? And you're proposing we make life more difficult for end users for an internal package move?! You're not killing two birds with one stone, you're killing user A's valuable time.

        I'll even add that for this to be commit-ready that a log message should be made in the deprecated classes to alert the user of the move.

        Show
        Erik Hatcher added a comment - Chris - you sure do go to an extreme of lots of typing to try to make your case when you've got two committers (and likely a user community consensus) saying deprecation is required. It's easy enough, and really makes a big difference to the sysadmin that'll be the one to deploy a new version of Solr. Consider the dramatic changes that go on with the Lucene index format itself, and it remains backwards compatible for a version or more. What if we told users they had to reindex everything? And you're proposing we make life more difficult for end users for an internal package move?! You're not killing two birds with one stone, you're killing user A's valuable time. I'll even add that for this to be commit-ready that a log message should be made in the deprecated classes to alert the user of the move.
        Hide
        Chris A. Mattmann added a comment - - edited

        Hi Erik,

        I'm sorry that you feel that way. I had one committer (Ryan) who voted on the related thread I posted on this ( http://mail-archives.apache.org/mod_mbox/lucene-solr-dev/200912.mbox/%3CC760CB08.8B0A%25Chris.A.Mattmann@jpl.nasa.gov%3E ) and agreed with the issue as I've proposed even (with no deprecations). In terms of speaking for your user community that's a pretty lofty statement assuming that the view of 2 committers is that of your user community. This hasn't been my Apache experience, nor my experience developing software in general. BTW, besides being a developer, I'm also one of SOLR's users as well (as I'm sure you are too), so we're really wearing two hats here. I may be a different type of user than the one you're targeting, but I'm a user nonetheless and that should speak for something.

        As for the Lucene index example, in terms of going to extreme, that's an extreme example, right? We're not talking about a data file format here. We're talking about a package move of classes that are really in the wrong package, of which there are about < 10 of those classes in use right now (and by in use, we're really talking about configuration because there's not that many people that are compiling against them I would venture to guess, and if they are, I stand firm it's better to be verbose in those situations). That's a big difference in terms of indexes that can grow big, and that need to be long lived and maintained than a solrconfig.xml file.

        As far as a log message in the deprecated classes, that's an interesting case. I'm assuming that this would catch users of SOLR that upgraded to 1.5 but that didn't upgrade their solrconfig.xml, right (assuming that class deprecations go in)? The only problem I see with that is that Joe user isn't always savvy enough to go into a log file and if he is, then he's likely already savvy enough to have read CHANGES.txt, right? In other words, why wait until runtime to catch something that could have been caught apriori?

        Like I said above, we could provide a script along with this patch (attached to this issue) that users are required to run to upgrade their solrconfig.xml files when upgrading to 1.5. This is pretty much what a lot of other Apache projects, e.g., Hadoop, HBase do, in the form of telling users that they need to run DFS upgrade on any Hadoop upgrade as a matter of principle, see: http://hadoop.apache.org/common/docs/r0.17.2/hdfs_user_guide.html#Upgrade+and+Rollback.

        Cheers,
        Chris

        Show
        Chris A. Mattmann added a comment - - edited Hi Erik, I'm sorry that you feel that way. I had one committer (Ryan) who voted on the related thread I posted on this ( http://mail-archives.apache.org/mod_mbox/lucene-solr-dev/200912.mbox/%3CC760CB08.8B0A%25Chris.A.Mattmann@jpl.nasa.gov%3E ) and agreed with the issue as I've proposed even (with no deprecations). In terms of speaking for your user community that's a pretty lofty statement assuming that the view of 2 committers is that of your user community. This hasn't been my Apache experience, nor my experience developing software in general. BTW, besides being a developer, I'm also one of SOLR's users as well (as I'm sure you are too), so we're really wearing two hats here. I may be a different type of user than the one you're targeting, but I'm a user nonetheless and that should speak for something. As for the Lucene index example, in terms of going to extreme, that's an extreme example, right? We're not talking about a data file format here. We're talking about a package move of classes that are really in the wrong package, of which there are about < 10 of those classes in use right now (and by in use, we're really talking about configuration because there's not that many people that are compiling against them I would venture to guess, and if they are, I stand firm it's better to be verbose in those situations). That's a big difference in terms of indexes that can grow big, and that need to be long lived and maintained than a solrconfig.xml file. As far as a log message in the deprecated classes, that's an interesting case. I'm assuming that this would catch users of SOLR that upgraded to 1.5 but that didn't upgrade their solrconfig.xml, right (assuming that class deprecations go in)? The only problem I see with that is that Joe user isn't always savvy enough to go into a log file and if he is, then he's likely already savvy enough to have read CHANGES.txt, right? In other words, why wait until runtime to catch something that could have been caught apriori? Like I said above, we could provide a script along with this patch (attached to this issue) that users are required to run to upgrade their solrconfig.xml files when upgrading to 1.5. This is pretty much what a lot of other Apache projects, e.g., Hadoop, HBase do, in the form of telling users that they need to run DFS upgrade on any Hadoop upgrade as a matter of principle, see: http://hadoop.apache.org/common/docs/r0.17.2/hdfs_user_guide.html#Upgrade+and+Rollback . Cheers, Chris
        Hide
        Noble Paul added a comment -

        hi
        I believe this issue is blown out of proportions.
        Chris, let us accept that your opinions differ here. you think properly organized code is important and slight inconvenience to users is OK.

        I side with Erik here. Users generally copy their existing config files and just expect it to run between point releases. We can make exceptions, but I don't think this deserves it.

        Show
        Noble Paul added a comment - hi I believe this issue is blown out of proportions. Chris, let us accept that your opinions differ here. you think properly organized code is important and slight inconvenience to users is OK. I side with Erik here. Users generally copy their existing config files and just expect it to run between point releases. We can make exceptions, but I don't think this deserves it.
        Hide
        Chris A. Mattmann added a comment -

        Hi Noble,

        I'm not sure anything has been blown out of proportions – this is a discussion related to an issue and a great example of the open source world and communication. As far as my opinion differing, I'm fine with accepting it differs from yours, Erik's and Hoss's, but that it doesn't seem to from Ryan's.

        In terms of copying files between releases, how much more arduous would it be to do a copy and execute a upgrade_solr_cofig 1 line bash script I could attach to this issue to upgrade the SOLR config file?

        Cheers,
        Chris

        Show
        Chris A. Mattmann added a comment - Hi Noble, I'm not sure anything has been blown out of proportions – this is a discussion related to an issue and a great example of the open source world and communication. As far as my opinion differing, I'm fine with accepting it differs from yours, Erik's and Hoss's, but that it doesn't seem to from Ryan's. In terms of copying files between releases, how much more arduous would it be to do a copy and execute a upgrade_solr_cofig 1 line bash script I could attach to this issue to upgrade the SOLR config file? Cheers, Chris
        Hide
        Noble Paul added a comment -

        how much more arduous would it be to do a copy and execute a upgrade_solr_cofig 1 line bash script I could attach to this issue to upgrade the SOLR config file?

        my question is .WHY? to change a few files from one package to another? The user may not even care if the entire all classes are put into one package .

        Show
        Noble Paul added a comment - how much more arduous would it be to do a copy and execute a upgrade_solr_cofig 1 line bash script I could attach to this issue to upgrade the SOLR config file? my question is .WHY? to change a few files from one package to another? The user may not even care if the entire all classes are put into one package .
        Hide
        Chris A. Mattmann added a comment -
        • the SOLR config upgrade script I mentioned.
        Show
        Chris A. Mattmann added a comment - the SOLR config upgrade script I mentioned.
        Hide
        Chris A. Mattmann added a comment - - edited

        my question is .WHY? to change a few files from one package to another? The user may not even care if the entire all classes are put into one package .

        My question is since when are code-level and architecture-level design decisions for a software system directly dictated by the end users of your application? At NASA, I don't consult end users of the software I develop regarding the organization of the software's code structure. End-user concerns are mostly with functional properties (performability, scalability, etc...) with a few NFP's sprinkled in (extensibility, ease of use, configurability, etc.), and not design decisions. While it is true that these concerns have a great impact on the software system's architecture, the notion of code-level organization is a bit orthogonal. However, that doesn't mean that design decisions (or improvements) shouldn't be made. Modern distributed software has many stakeholders, including end-users, software developers, architects, managers, etc. This is clearly addressing a concern of a few of those stakeholders (architects, developers [you could argue either end of this], managers [who may market the software and where clean organization is something they'd like to add as a selling point], but end-users are not really one of them in this case I'll agree).

        Ultimately because of this, the goal is to reap the benefits of good design and code organization within acceptable bounds to the user. That's really the discussion point at hand. I believe acceptable bounds to be:

        1. Remove the classes from the package structure right now to deal with this move swiftly and reduce confusion (i.e., why are there 2 copies of the same class, in different packages?) Admittedly, this is an architect/developer/manager issue more so than an end-user one.
        2. Provide a simple, 1 line upgrade script that end-users can upgrade their solrconfig.xml file with. That way when and if they employ the approach you mentioned Noble (i.e., copying their old configuration), there is minute additional effort to upgrade the config.

        I've done the deprecation route on one of my projects at NASA 7-8 years ago in a highly similar situation. In fact, it was way more extreme. We moved most of our core Java classes (100s of them) from a top-level package to another top-level package. In doing so, we kept replicas of the Java classes in the old package structure using the extends structure with deprecations as shown by Ryan above. We saw 0 measurable benefit of doing this, and we had 100s-1000s of end users of the software across many NASA centers and external organizations. And to me, it was just an organizational mess that caused confusion among new developers we hired to evolve the code. So, I just don't see the benefit. From a code management point of view it added extra work to the developer/architect/manager stakeholders more than anything else.

        Cheers,
        Chris

        Show
        Chris A. Mattmann added a comment - - edited my question is .WHY? to change a few files from one package to another? The user may not even care if the entire all classes are put into one package . My question is since when are code-level and architecture-level design decisions for a software system directly dictated by the end users of your application? At NASA, I don't consult end users of the software I develop regarding the organization of the software's code structure. End-user concerns are mostly with functional properties (performability, scalability, etc...) with a few NFP's sprinkled in (extensibility, ease of use, configurability, etc.), and not design decisions. While it is true that these concerns have a great impact on the software system's architecture, the notion of code-level organization is a bit orthogonal. However, that doesn't mean that design decisions (or improvements) shouldn't be made. Modern distributed software has many stakeholders, including end-users, software developers, architects, managers, etc. This is clearly addressing a concern of a few of those stakeholders (architects, developers [you could argue either end of this] , managers [who may market the software and where clean organization is something they'd like to add as a selling point] , but end-users are not really one of them in this case I'll agree). Ultimately because of this, the goal is to reap the benefits of good design and code organization within acceptable bounds to the user. That's really the discussion point at hand. I believe acceptable bounds to be: Remove the classes from the package structure right now to deal with this move swiftly and reduce confusion (i.e., why are there 2 copies of the same class, in different packages?) Admittedly, this is an architect/developer/manager issue more so than an end-user one. Provide a simple, 1 line upgrade script that end-users can upgrade their solrconfig.xml file with. That way when and if they employ the approach you mentioned Noble (i.e., copying their old configuration), there is minute additional effort to upgrade the config. I've done the deprecation route on one of my projects at NASA 7-8 years ago in a highly similar situation. In fact, it was way more extreme. We moved most of our core Java classes (100s of them) from a top-level package to another top-level package. In doing so, we kept replicas of the Java classes in the old package structure using the extends structure with deprecations as shown by Ryan above. We saw 0 measurable benefit of doing this, and we had 100s-1000s of end users of the software across many NASA centers and external organizations. And to me, it was just an organizational mess that caused confusion among new developers we hired to evolve the code. So, I just don't see the benefit. From a code management point of view it added extra work to the developer/architect/manager stakeholders more than anything else. Cheers, Chris
        Hide
        Noble Paul added a comment -

        when are code-level and architecture-level design decisions for a software system directly dictated by the end users of your application?

        If I am going to add a new feature, I would not.

        We really don't know the impact. But , we are just being careful.

        Show
        Noble Paul added a comment - when are code-level and architecture-level design decisions for a software system directly dictated by the end users of your application? If I am going to add a new feature, I would not. We really don't know the impact. But , we are just being careful.
        Hide
        patrick o'leary added a comment -

        I've been writing plugin's for solr for a couple of years, and I have seen several situations when we broke backwards compatibility, and configuration for absolutely no reasonable reason except for refactoring for the sake of refactoring.

        One that springs to mind is updateRequestProcessor going to updateRequestProcessorChain.
        Along with many more that over the years have caused me and anyone else who consumes solr to realize that upgrades
        cost time and money.

        Why should this be different?
        This at least, cleans things up, the deprecations strategy used in the past have caused more headache than anything,
        we deprecate, leave a class in place, sometimes even empty out the class so does nothing....
        and force folks to use 'find' and 'grep' as a way to get solr working.
        Look at SOLR-489, as a prime example, that it's just not been a good practice in the past.
        To me that's not a helpful way to refactor.

        Show
        patrick o'leary added a comment - I've been writing plugin's for solr for a couple of years, and I have seen several situations when we broke backwards compatibility, and configuration for absolutely no reasonable reason except for refactoring for the sake of refactoring. One that springs to mind is updateRequestProcessor going to updateRequestProcessorChain. Along with many more that over the years have caused me and anyone else who consumes solr to realize that upgrades cost time and money. Why should this be different? This at least, cleans things up, the deprecations strategy used in the past have caused more headache than anything, we deprecate, leave a class in place, sometimes even empty out the class so does nothing.... and force folks to use 'find' and 'grep' as a way to get solr working. Look at SOLR-489 , as a prime example, that it's just not been a good practice in the past. To me that's not a helpful way to refactor.
        Hide
        Uri Boness added a comment -

        I think it is very important to understand all sides here.

        I fully and totally support Chris's attempts to clean up the code base which rightfully involves moving classes from one package to another. I think in some cases such cleanups need to come at the cost of user comfort as eventually they, as users, also gain from it as the system as a whole becomes more robust, extensible and maintainable. The good thing is that besides the deprecation issue I believe there is a consensus about the required changes. So thumbs up Chris!!!

        To deprecate or not to deprecate, that is the question. In a widely used library/framework/system with a large (or fast growing) install/user base such as the Solr community, the common practice is not to just break BWC without giving the users some grace period in which they can adjust their deployments to the new changes. Sometimes, it's absolutely necessary (such in the cases of bug fixes) but when it's not, in general it can create the opposite effect than you want with the community - instead having the community appreciate your improvements and see Solr as an "improving with time" product, they turn and see Solr as an inconsistent and sometime even unreliable product. So from my experience with delivering goods for the users, especially in the open source world (and I do have quite a bit of experience in that respect with the Spring Framework) you always need to strive to 100% BWC in theory and ~95% BWC in practice (never less than 90% though). If you stick to that, I believe changes will be widely accepted as improvements rather than harassments.

        But there's a catch here!!!! In order to stick to these numbers, you have to adhere to two important conditions:

        1. You need to have a rather solid architecture and code base to start with. If you don't, then naturally in the beginning you can expect many more extreme/major changes which lead to quite a few BWC breaks (it will gradually be reduced as the architecture/codebase improves). Whether Solr answers this condition is open for debate... there are a lot of solid parts in Solr and quite a few parts where a complete rewrite is appropriate.

        2. You need to have a steady and short release cycles. This is one thing that Solr lacks... big time. In a 1 year release cycle, deprecating code means that for the next year (in some cases two years), the code base will be messy with deprecated classes all over the place. In that respect, I can definitely understand Chris's objection for deprecation as the cleanup tasks that he's implementing may end up creating more mess (at least for a long while) than you had before the cleanup all together. I believe that moving to shorter release cycles (including bug-fix releases) will greatly help promoting deprecation in general.

        (NOTE: just a small note about the first condition. One thing to take into account is that every piece of software reaches a point in time where it needs to be completely re-written or at least go through a major refactoring/re-architecturing phase. This can be caused by many factors, let it be new technologies that are introduced, or simply limitations of the architecture that were not foreseen. It's very important to understand and admit to this fact - even from the user point of view it's acceptable. What is not acceptable, if it happens too many times and too frequent)

        Bottom line, it's always a conflict between the user point of view and the developer point of view. And there needs to be a balance and understanding of both sides. Each side needs to understand and give in to some extend to create this balance. But to make it happen, the culture, environment and well defined policies need to be in place. Arguing endlessly who's right here will never bring to a good outcome, simply because both sides are right and wrong at the same time, if you treat it as a black or white issue you'll end up loosing something - either the user trust or a better software. How about creating a proper release plan for the upcoming year, say a release every two months? Chris, if you have such a release schedule, will you feel more comfortable with deprecation?

        Show
        Uri Boness added a comment - I think it is very important to understand all sides here. I fully and totally support Chris's attempts to clean up the code base which rightfully involves moving classes from one package to another. I think in some cases such cleanups need to come at the cost of user comfort as eventually they, as users, also gain from it as the system as a whole becomes more robust, extensible and maintainable. The good thing is that besides the deprecation issue I believe there is a consensus about the required changes. So thumbs up Chris!!! To deprecate or not to deprecate, that is the question. In a widely used library/framework/system with a large (or fast growing) install/user base such as the Solr community, the common practice is not to just break BWC without giving the users some grace period in which they can adjust their deployments to the new changes. Sometimes, it's absolutely necessary (such in the cases of bug fixes) but when it's not, in general it can create the opposite effect than you want with the community - instead having the community appreciate your improvements and see Solr as an "improving with time" product, they turn and see Solr as an inconsistent and sometime even unreliable product. So from my experience with delivering goods for the users, especially in the open source world (and I do have quite a bit of experience in that respect with the Spring Framework) you always need to strive to 100% BWC in theory and ~95% BWC in practice (never less than 90% though). If you stick to that, I believe changes will be widely accepted as improvements rather than harassments. But there's a catch here!!!! In order to stick to these numbers, you have to adhere to two important conditions: 1. You need to have a rather solid architecture and code base to start with. If you don't, then naturally in the beginning you can expect many more extreme/major changes which lead to quite a few BWC breaks (it will gradually be reduced as the architecture/codebase improves). Whether Solr answers this condition is open for debate... there are a lot of solid parts in Solr and quite a few parts where a complete rewrite is appropriate. 2. You need to have a steady and short release cycles. This is one thing that Solr lacks... big time. In a 1 year release cycle, deprecating code means that for the next year (in some cases two years), the code base will be messy with deprecated classes all over the place. In that respect, I can definitely understand Chris's objection for deprecation as the cleanup tasks that he's implementing may end up creating more mess (at least for a long while) than you had before the cleanup all together. I believe that moving to shorter release cycles (including bug-fix releases) will greatly help promoting deprecation in general. (NOTE: just a small note about the first condition. One thing to take into account is that every piece of software reaches a point in time where it needs to be completely re-written or at least go through a major refactoring/re-architecturing phase. This can be caused by many factors, let it be new technologies that are introduced, or simply limitations of the architecture that were not foreseen. It's very important to understand and admit to this fact - even from the user point of view it's acceptable. What is not acceptable, if it happens too many times and too frequent) Bottom line, it's always a conflict between the user point of view and the developer point of view. And there needs to be a balance and understanding of both sides. Each side needs to understand and give in to some extend to create this balance. But to make it happen, the culture, environment and well defined policies need to be in place. Arguing endlessly who's right here will never bring to a good outcome, simply because both sides are right and wrong at the same time, if you treat it as a black or white issue you'll end up loosing something - either the user trust or a better software. How about creating a proper release plan for the upcoming year, say a release every two months? Chris, if you have such a release schedule, will you feel more comfortable with deprecation?
        Hide
        patrick o'leary added a comment -

        1) I agree that Solr needs shorter release cycles, and let me add to that, a firm road map, that is more community driven.

        • Tasks are tackled by trying to solve the hardest parts, rather than iterative development.
        • Users don't see benefits, just patches, most of which won't apply after a few weeks or months.

        2) Solr is still young

        • it's only been around 4 years.
        • It can afford changes, this one is minor, it's config driven, solrconfig (e.g. where only experts dare) rather than schema-
          <queryResponseWriter name="xml" class="org.apache.solr.response.XMLResponseWriter" default="true"/>
           <queryResponseWriter name="json" class="org.apache.solr.response.JSONResponseWriter"/>
           <queryResponseWriter name="python" class="org.apache.solr.response.PythonResponseWriter"/>
           <queryResponseWriter name="ruby" class="org.apache.solr.response.RubyResponseWriter"/>
           <queryResponseWriter name="php" class="org.apache.solr.response.PHPResponseWriter"/>
           <queryResponseWriter name="phps" class="org.apache.solr.response.PHPSerializedResponseWriter"/>
           <queryResponseWriter name="xslt" class="org.apache.solr.response.XSLTResponseWriter">
          

        3) We still don't identify who consumers of Solr are

        • The end user, where search quality, and performance makes a difference
        • The implementer, who downloads, installs, updates solr
        • The experts, who customize solr
        • The extender, who develop using Solr as a framework (embedded, and as a webapp)

        This change affects the last 2, experts, and extenders, and in a positive way.
        The last two classes of users are continuously being affected by development, just think of
        packages being moved around in solr/common, solr/java, solr/solrj etc..

        One compromise would be to have releases as sprints, say a minor release every quarter, and major every 1 to 2 years?
        Where you can deprecate something with the resolution of eliminating it before 2 minor releases. (6 months)

        Show
        patrick o'leary added a comment - 1) I agree that Solr needs shorter release cycles, and let me add to that, a firm road map, that is more community driven. Tasks are tackled by trying to solve the hardest parts, rather than iterative development. Users don't see benefits, just patches, most of which won't apply after a few weeks or months. 2) Solr is still young it's only been around 4 years. It can afford changes, this one is minor, it's config driven, solrconfig (e.g. where only experts dare) rather than schema- <queryResponseWriter name= "xml" class= "org.apache.solr.response.XMLResponseWriter" default = " true " /> <queryResponseWriter name= "json" class= "org.apache.solr.response.JSONResponseWriter" /> <queryResponseWriter name= "python" class= "org.apache.solr.response.PythonResponseWriter" /> <queryResponseWriter name= "ruby" class= "org.apache.solr.response.RubyResponseWriter" /> <queryResponseWriter name= "php" class= "org.apache.solr.response.PHPResponseWriter" /> <queryResponseWriter name= "phps" class= "org.apache.solr.response.PHPSerializedResponseWriter" /> <queryResponseWriter name= "xslt" class= "org.apache.solr.response.XSLTResponseWriter" > 3) We still don't identify who consumers of Solr are The end user, where search quality, and performance makes a difference The implementer, who downloads, installs, updates solr The experts, who customize solr The extender, who develop using Solr as a framework (embedded, and as a webapp) This change affects the last 2, experts, and extenders, and in a positive way. The last two classes of users are continuously being affected by development, just think of packages being moved around in solr/common, solr/java, solr/solrj etc.. One compromise would be to have releases as sprints, say a minor release every quarter, and major every 1 to 2 years? Where you can deprecate something with the resolution of eliminating it before 2 minor releases. (6 months)
        Hide
        Hoss Man added a comment -

        Uh ... ok.

        First off: Broad discussions beyond the scope of this particular Jira issue really don't feel appropriate in the comments – discussions about Solr's release cycles, general architecture, roadmaps, targeted user base, etc. should be on the mailing list where they are more visible.

        Second: In trying to catch up with all of the new comments in this issue since the last time i looked at it, i find myself with very little substantive comments to add beyond what i already said. i started trying to address some individual points people have made, but that just felt tedious, so i'll just make a few comments in general. Hopefully I'll say the same thing in a different way that will make more sense...

        There are very few instances in Solr's history where we have ever (knowningly) broken backwards compatibility at the "user level" ( config files, request params, etc...) – in all of those cases, we tried to provide a simple method for users who depended on the legacy behavior to restore it with a simple configure change which was advertised in the upgrade instructions. In every one of those situations (that i can remember) we still had a few users who got confused/frustrated at the change in behavior. I don't really have the energy/inclination to go find documentation on all of those situations, but here's two examples that are fresh in my mind because they just happened last month when people tried to upgrade to 1.4 and didn't notice the change regarding the legacy ";" sort syntax...

        http://old.nabble.com/Upgrade-from-1.2-to-1.4-to26829388.html#a26829388
        http://phatness.com/2009/11/sorting-by-date-with-solr/

        I'm not suggesting that we should always strive for 100% back compat in the configs and the request params and that people should expect identical configs to always work in every Solr release for hte next 74 years – but so far, the only times we've (knowingly) broken existing configs it was when the goal was to either improve the default behavior, or add additional functionality. I've got no problem telling 40% of our existing users "you need to add foo=bar to your config to have it keep working" if it means that the other 60% and every other new users gets something good out of deal....

        ....but i see no good reason to break things for a large percentage of our user base when there is no value add to them in return ... having a (subjectively) cleaner code base is not a justification for making an upgrade break unless users edit their configs, or run a script.

        If breaking things was the only way to fix some major bug, or add some cool feature, or saved us 100 man hours of development then i would be completley on board telling people "sorry, please change this one line." but when keeping things working takes such little effort, it would be completley irresponsible to fuck our users over like that – trivial shit like needing to make a nusance edit to a config file for no apparent reason is what drives users away and leaves a bad enough taste in their mouth that they actively trash talk a product/project as being "unstable"

        I honestly can't fathom how having ~10 less files in one directory is worth the amount of discussion we've already had on this – let alone the headaches of all the people this would screw over on upgrading. It's just such a fucking no brainer to me.

        Show
        Hoss Man added a comment - Uh ... ok. First off: Broad discussions beyond the scope of this particular Jira issue really don't feel appropriate in the comments – discussions about Solr's release cycles, general architecture, roadmaps, targeted user base, etc. should be on the mailing list where they are more visible. Second: In trying to catch up with all of the new comments in this issue since the last time i looked at it, i find myself with very little substantive comments to add beyond what i already said. i started trying to address some individual points people have made, but that just felt tedious, so i'll just make a few comments in general. Hopefully I'll say the same thing in a different way that will make more sense... There are very few instances in Solr's history where we have ever (knowningly) broken backwards compatibility at the "user level" ( config files, request params, etc...) – in all of those cases, we tried to provide a simple method for users who depended on the legacy behavior to restore it with a simple configure change which was advertised in the upgrade instructions. In every one of those situations (that i can remember) we still had a few users who got confused/frustrated at the change in behavior. I don't really have the energy/inclination to go find documentation on all of those situations, but here's two examples that are fresh in my mind because they just happened last month when people tried to upgrade to 1.4 and didn't notice the change regarding the legacy ";" sort syntax... http://old.nabble.com/Upgrade-from-1.2-to-1.4-to26829388.html#a26829388 http://phatness.com/2009/11/sorting-by-date-with-solr/ I'm not suggesting that we should always strive for 100% back compat in the configs and the request params and that people should expect identical configs to always work in every Solr release for hte next 74 years – but so far, the only times we've (knowingly) broken existing configs it was when the goal was to either improve the default behavior, or add additional functionality. I've got no problem telling 40% of our existing users "you need to add foo=bar to your config to have it keep working" if it means that the other 60% and every other new users gets something good out of deal.... ....but i see no good reason to break things for a large percentage of our user base when there is no value add to them in return ... having a (subjectively) cleaner code base is not a justification for making an upgrade break unless users edit their configs, or run a script. If breaking things was the only way to fix some major bug, or add some cool feature, or saved us 100 man hours of development then i would be completley on board telling people "sorry, please change this one line." but when keeping things working takes such little effort, it would be completley irresponsible to fuck our users over like that – trivial shit like needing to make a nusance edit to a config file for no apparent reason is what drives users away and leaves a bad enough taste in their mouth that they actively trash talk a product/project as being "unstable" I honestly can't fathom how having ~10 less files in one directory is worth the amount of discussion we've already had on this – let alone the headaches of all the people this would screw over on upgrading. It's just such a fucking no brainer to me.
        Hide
        Chris A. Mattmann added a comment -

        Hi Hoss:

        Comments below:

        in all of those cases, we tried to provide a simple method for users who depended on the legacy behavior to restore it with a simple configure change which was advertised in the upgrade instructions.

        This is precisely what I suggested doing.

        In every one of those situations (that i can remember) we still had a few users who got confused/frustrated at the change in behavior.

        The change I'm proposing doesn't change behavior. It changes configuration. The two are different.

        I'm not suggesting that we should always strive for 100% back compat in the configs and the request params and that people should expect identical configs to always work in every Solr release for hte next 74 years - but so far, the only times we've (knowingly) broken existing configs it was when the goal was to either improve the default behavior, or add additional functionality. I've got no problem telling 40% of our existing users "you need to add foo=bar to your config to have it keep working" if it means that the other 60% and every other new users gets something good out of deal....

        "good" is a qualitative term. I consider strong system design and organization something that SOLR should be good at. All of the other qualities or observable aspects of the system (performance, efficiency, scalability, etc.) are also important things – however, not to this issue.

        ....but i see no good reason to break things for a large percentage of our user base when there is no value add to them in return ... having a (subjectively) cleaner code base is not a justification for making an upgrade break unless users edit their configs, or run a script.

        There is value added in better system structure, architecture and code organization. There are plenty of books written about the benefits of this and I won't bother to go into detail. Further, "upgrade break" is sort of a loaded phrase. It's not a break. It's an upgrade. Break would imply something out of the norm, and I don't see any formal SOLR upgrade process that the approach proposed in this issue deviates from. You suggested yourself that the types of "upgrade" notations that this issue proposes in are advertised in CHANGES.txt. This issue is no different.

        If breaking things was the only way to fix some major bug, or add some cool feature, or saved us 100 man hours of development then i would be completley on board telling people "sorry, please change this one line."

        There are many phases to the software engineering lifecycle. Development is just one of them. This issue addresses time/cost savings in understandability of SOLR's code base. For the quantitative value, I'll omit guessing as to the general savings and just point to you to some reading on what bad code organization and system structure can cause in terms of cost when others try and understand your code [1] [2] [3].

        but when keeping things working takes such little effort, it would be completley irresponsible to ___ our users over like that - trivial ___ like needing to make a nusance edit to a config file for no apparent reason is what drives users away and leaves a bad enough taste in their mouth that they actively trash talk a product/project as being "unstable"

        I'm both a software developer and a user of SOLR, and the consistent resistance to any proposed refactoring is quite troubling. As Uri noted above, software requires refactoring, SOLR is no different. And as I noted from a code organization standpoint, placing classes named response in a package named request is not subjectively anything – it's poor design and it needs to be addressed. As for "no apparent reason" as I mentioned to Noble, end-users of a system don't dictate its code-level organization/design.

        I'm also of the opinion that there are many stakeholders of any distributed software system, SOLR being no different. Patrick iterated them above, as did I and as did Uri. There aren't just "users" of SOLR – there are end-users, system administrators, core developers, plugin writers, architects, managers, etc.

        Cheers,
        Chris

        [1] http://portal.acm.org/citation.cfm?id=592025
        [2] http://portal.acm.org/citation.cfm?id=302691
        [3] http://portal.acm.org/citation.cfm?id=50089

        Show
        Chris A. Mattmann added a comment - Hi Hoss: Comments below: in all of those cases, we tried to provide a simple method for users who depended on the legacy behavior to restore it with a simple configure change which was advertised in the upgrade instructions. This is precisely what I suggested doing. In every one of those situations (that i can remember) we still had a few users who got confused/frustrated at the change in behavior. The change I'm proposing doesn't change behavior. It changes configuration. The two are different. I'm not suggesting that we should always strive for 100% back compat in the configs and the request params and that people should expect identical configs to always work in every Solr release for hte next 74 years - but so far, the only times we've (knowingly) broken existing configs it was when the goal was to either improve the default behavior, or add additional functionality. I've got no problem telling 40% of our existing users "you need to add foo=bar to your config to have it keep working" if it means that the other 60% and every other new users gets something good out of deal.... "good" is a qualitative term. I consider strong system design and organization something that SOLR should be good at. All of the other qualities or observable aspects of the system (performance, efficiency, scalability, etc.) are also important things – however, not to this issue. ....but i see no good reason to break things for a large percentage of our user base when there is no value add to them in return ... having a (subjectively) cleaner code base is not a justification for making an upgrade break unless users edit their configs, or run a script. There is value added in better system structure, architecture and code organization. There are plenty of books written about the benefits of this and I won't bother to go into detail. Further, "upgrade break" is sort of a loaded phrase. It's not a break. It's an upgrade. Break would imply something out of the norm, and I don't see any formal SOLR upgrade process that the approach proposed in this issue deviates from. You suggested yourself that the types of "upgrade" notations that this issue proposes in are advertised in CHANGES.txt. This issue is no different. If breaking things was the only way to fix some major bug, or add some cool feature, or saved us 100 man hours of development then i would be completley on board telling people "sorry, please change this one line." There are many phases to the software engineering lifecycle. Development is just one of them. This issue addresses time/cost savings in understandability of SOLR's code base. For the quantitative value, I'll omit guessing as to the general savings and just point to you to some reading on what bad code organization and system structure can cause in terms of cost when others try and understand your code [1] [2] [3] . but when keeping things working takes such little effort, it would be completley irresponsible to ___ our users over like that - trivial ___ like needing to make a nusance edit to a config file for no apparent reason is what drives users away and leaves a bad enough taste in their mouth that they actively trash talk a product/project as being "unstable" I'm both a software developer and a user of SOLR, and the consistent resistance to any proposed refactoring is quite troubling. As Uri noted above, software requires refactoring, SOLR is no different. And as I noted from a code organization standpoint, placing classes named response in a package named request is not subjectively anything – it's poor design and it needs to be addressed. As for "no apparent reason" as I mentioned to Noble, end-users of a system don't dictate its code-level organization/design. I'm also of the opinion that there are many stakeholders of any distributed software system, SOLR being no different. Patrick iterated them above, as did I and as did Uri. There aren't just "users" of SOLR – there are end-users, system administrators, core developers, plugin writers, architects, managers, etc. Cheers, Chris [1] http://portal.acm.org/citation.cfm?id=592025 [2] http://portal.acm.org/citation.cfm?id=302691 [3] http://portal.acm.org/citation.cfm?id=50089
        Hide
        Shalin Shekhar Mangar added a comment -

        One that springs to mind is updateRequestProcessor going to updateRequestProcessorChain.

        Patrick, I think that change was made in trunk before update processors were ever released.

        I'm both a software developer and a user of SOLR, and the consistent resistance to any proposed refactoring is quite troubling.

        The resistance is not towards refactoring. We are arguing about compatibility, not refactoring.

        And as I noted from a code organization standpoint, placing classes named response in a package named request is not subjectively anything - it's poor design and it needs to be addressed.

        I bet 99% of the users do not care about a wrongly named package when everything works. But they care when things stop working. Code organization is secondary to usability. Let us not cause discomfort to our users for such a trivial issue.

        As for "no apparent reason" as I mentioned to Noble, end-users of a system don't dictate its code-level organization/design.

        End users do not dictate code level organization but they do have an influence when compatibility is involved. In this case, it is an inconvenience for many of them which can be avoided easily, so why not?

        I agree with Hoss. This is too much discussion over too small an issue. I think things are quite clear. Hoss, Erik, Noble and I all feel that breaking compatibility is not worth it. So lets do what needs to be done and get on with more important things.

        Show
        Shalin Shekhar Mangar added a comment - One that springs to mind is updateRequestProcessor going to updateRequestProcessorChain. Patrick, I think that change was made in trunk before update processors were ever released. I'm both a software developer and a user of SOLR, and the consistent resistance to any proposed refactoring is quite troubling. The resistance is not towards refactoring. We are arguing about compatibility, not refactoring. And as I noted from a code organization standpoint, placing classes named response in a package named request is not subjectively anything - it's poor design and it needs to be addressed. I bet 99% of the users do not care about a wrongly named package when everything works. But they care when things stop working. Code organization is secondary to usability. Let us not cause discomfort to our users for such a trivial issue. As for "no apparent reason" as I mentioned to Noble, end-users of a system don't dictate its code-level organization/design. End users do not dictate code level organization but they do have an influence when compatibility is involved. In this case, it is an inconvenience for many of them which can be avoided easily, so why not? I agree with Hoss. This is too much discussion over too small an issue. I think things are quite clear. Hoss, Erik, Noble and I all feel that breaking compatibility is not worth it. So lets do what needs to be done and get on with more important things.
        Hide
        Chris A. Mattmann added a comment -

        I bet 99% of the users do not care about a wrongly named package when everything works. But they care when things stop working. Code organization is secondary to usability. Let us not cause discomfort to our users for such a trivial issue.

        I wouldn't be so quick to bet what 99% of your "users" think without some solid evidence to back that up. And no, code organization is not second to usability to all parties – it depends on what hat you're using. To you it may be, but not to everyone.

        End users do not dictate code level organization but they do have an influence when compatibility is involved. In this case, it is an inconvenience for many of them which can be avoided easily, so why not?

        Because satisfying end-users is only looking at one of the stakeholders of the system and ignoring the others.

        I agree with Hoss. This is too much discussion over too small an issue. I think things are quite clear. Hoss, Erik, Noble and I all feel that breaking compatibility is not worth it. So lets do what needs to be done and get on with more important things.

        The issue is far from clear as far as I'm concerned and I'm the one who reported the issue. The comments on this issue from others are evidence of that. The issue will be satisfied when there is clear consensus from the parties involved.

        Show
        Chris A. Mattmann added a comment - I bet 99% of the users do not care about a wrongly named package when everything works. But they care when things stop working. Code organization is secondary to usability. Let us not cause discomfort to our users for such a trivial issue. I wouldn't be so quick to bet what 99% of your "users" think without some solid evidence to back that up. And no, code organization is not second to usability to all parties – it depends on what hat you're using. To you it may be, but not to everyone. End users do not dictate code level organization but they do have an influence when compatibility is involved. In this case, it is an inconvenience for many of them which can be avoided easily, so why not? Because satisfying end-users is only looking at one of the stakeholders of the system and ignoring the others. I agree with Hoss. This is too much discussion over too small an issue. I think things are quite clear. Hoss, Erik, Noble and I all feel that breaking compatibility is not worth it. So lets do what needs to be done and get on with more important things. The issue is far from clear as far as I'm concerned and I'm the one who reported the issue. The comments on this issue from others are evidence of that. The issue will be satisfied when there is clear consensus from the parties involved.
        Hide
        Ryan McKinley added a comment -

        In an effort to get some resolution here...

        There are only three options:
        A. Leave it as is
        B. Refactor with deprecations
        -C. Refactor without deprecations-

        C is out, so A and B are the only options worth discussing.

        The advantage of B is that the package:
        o.a.s.response would be clean and organized
        (but o.a.s.request would still have a bunch of deprecated files)

        The arguments for A amount to: "things are fine as they are", or "the confusion of changing is not worth whatever slight gain we would get"

        The strong resistance is to 'C', I suspect wider ambivalence towards 'B'

        Show
        Ryan McKinley added a comment - In an effort to get some resolution here... There are only three options: A. Leave it as is B. Refactor with deprecations - C. Refactor without deprecations - C is out, so A and B are the only options worth discussing. The advantage of B is that the package: o.a.s.response would be clean and organized (but o.a.s.request would still have a bunch of deprecated files) The arguments for A amount to: "things are fine as they are", or "the confusion of changing is not worth whatever slight gain we would get" The strong resistance is to 'C', I suspect wider ambivalence towards 'B'
        Hide
        Andrzej Bialecki added a comment -

        I'm in favor of B. This worked well in Hadoop (mapred -> mapreduce) where the list of deprecations was massive and API changes were not straightforward at all - still it was done to promote a better design and allow new functionality. Whole deprecated hierarchies live there for at least two major releases, and surely they were visible to thousands of Hadoop devs. The downside was occasional confusion, and of course the porting effort required to use the new API, but the upside was an excellent back-compat to keep serious users happy, and a clear message to all to get prepared for the switch.

        So IMHO having a bunch of deprecated classes for a while is not a big deal, if it gives us freedom to pursue a better design.

        Show
        Andrzej Bialecki added a comment - I'm in favor of B. This worked well in Hadoop (mapred -> mapreduce) where the list of deprecations was massive and API changes were not straightforward at all - still it was done to promote a better design and allow new functionality. Whole deprecated hierarchies live there for at least two major releases, and surely they were visible to thousands of Hadoop devs. The downside was occasional confusion, and of course the porting effort required to use the new API, but the upside was an excellent back-compat to keep serious users happy, and a clear message to all to get prepared for the switch. So IMHO having a bunch of deprecated classes for a while is not a big deal, if it gives us freedom to pursue a better design.
        Hide
        Chris A. Mattmann added a comment - - edited

        Hi All:

        In the interest of moving forward on this, I'll go with option B, as I think I've made my point (as have others) and this is something important to get wrapped up IMO. I'd like to have an understanding though that in the next release of SOLR (1.6?) the deprecated classes can go away. Or, as Uri and Patrick pointed out, if the release cycle picks up then at worst 1.7.

        Thanks to everyone for their thoughtful comments. I am going to throw up a new patch that:

        1. replaces the content of all o.a.s.request.*ResponseWriters with the extends notation that Ryan mentioned above. In addition, I'll throw in a log message (to satisfy Erik's concern) in the constructor of each of the deprecated classes stating that these classes are going away very soon, so please change references to o.a.s.response.*

        2. adds the old o.a.s.request.*ResponseWriters to o.a.s.response.*ResponseWriters
        3. merges with my existing patch which updates references everywhere else including solrconfig.xml.

        I think that should satisfy everyone. I'll throw up a patch hopefully by the end of the week.

        Cheers,
        Chris

        Show
        Chris A. Mattmann added a comment - - edited Hi All: In the interest of moving forward on this, I'll go with option B, as I think I've made my point (as have others) and this is something important to get wrapped up IMO. I'd like to have an understanding though that in the next release of SOLR (1.6?) the deprecated classes can go away. Or, as Uri and Patrick pointed out, if the release cycle picks up then at worst 1.7. Thanks to everyone for their thoughtful comments. I am going to throw up a new patch that: 1. replaces the content of all o.a.s.request.*ResponseWriters with the extends notation that Ryan mentioned above. In addition, I'll throw in a log message (to satisfy Erik's concern) in the constructor of each of the deprecated classes stating that these classes are going away very soon, so please change references to o.a.s.response.* 2. adds the old o.a.s.request.*ResponseWriters to o.a.s.response.*ResponseWriters 3. merges with my existing patch which updates references everywhere else including solrconfig.xml. I think that should satisfy everyone. I'll throw up a patch hopefully by the end of the week. Cheers, Chris
        Hide
        Ryan McKinley added a comment -

        Hey Chris-

        Don't worry about putting up a patch... the real work is just svn move and that does not work well with patches.

        Lets wait a few days and see if there are no objections, I will do the refactoring (and replacing) on the repos that would get committed.

        ryan

        Show
        Ryan McKinley added a comment - Hey Chris- Don't worry about putting up a patch... the real work is just svn move and that does not work well with patches. Lets wait a few days and see if there are no objections, I will do the refactoring (and replacing) on the repos that would get committed. ryan
        Hide
        Chris A. Mattmann added a comment -

        Hey Ryan:

        +1 on the SVN move to handle the package creation/class movement/deprecation classes. However, you'll still want to apply the existing patch I threw up (step 3 above on my prior comment) that updates the refs everywhere in the code/contrib and config to use the new pkg name.

        +1 on waiting the few days, thanks for your help and for offering to commit the solution.

        Cheers,
        Chris

        Show
        Chris A. Mattmann added a comment - Hey Ryan: +1 on the SVN move to handle the package creation/class movement/deprecation classes. However, you'll still want to apply the existing patch I threw up (step 3 above on my prior comment) that updates the refs everywhere in the code/contrib and config to use the new pkg name. +1 on waiting the few days, thanks for your help and for offering to commit the solution. Cheers, Chris
        Hide
        Noble Paul added a comment -

        instead of using the FQN of classes we should use the solr.<SimpleClassname> notation in solrconfig.xml. That would have enabled us moving packages easily

        Show
        Noble Paul added a comment - instead of using the FQN of classes we should use the solr.<SimpleClassname> notation in solrconfig.xml. That would have enabled us moving packages easily
        Hide
        Erik Hatcher added a comment -

        Noble - thanks for mentioning the solr.* trick. I thought of this the other day. It's kinda already done, no?

        from SolrResourceLoader
        static final String[] packages =

        {"","analysis.","schema.","handler.","search.","update.","core.","request.","update.processor.","util.", "spelling.", "handler.component.", "handler.dataimport."}

        ;

        So, with the response package registered in there, all would be fine. I still think the right thing to do with this one is simply to deprecate and sweep it up after a version released or so. Can't hurt really.

        This actually gets to some package design considerations. While it has been frustrating for some Lucene hackers to hit the wall on final/private classes in the core, it gave Lucene a lot of flexibility to refactor relentlessly without worrying about deprecating and leaving a mess. But if things are public actual API meant to be extended, deprecation is the kindest, most appropriate way forward.

        Show
        Erik Hatcher added a comment - Noble - thanks for mentioning the solr.* trick. I thought of this the other day. It's kinda already done, no? from SolrResourceLoader static final String[] packages = {"","analysis.","schema.","handler.","search.","update.","core.","request.","update.processor.","util.", "spelling.", "handler.component.", "handler.dataimport."} ; So, with the response package registered in there, all would be fine. I still think the right thing to do with this one is simply to deprecate and sweep it up after a version released or so. Can't hurt really. This actually gets to some package design considerations. While it has been frustrating for some Lucene hackers to hit the wall on final/private classes in the core, it gave Lucene a lot of flexibility to refactor relentlessly without worrying about deprecating and leaving a mess. But if things are public actual API meant to be extended, deprecation is the kindest, most appropriate way forward.
        Hide
        Noble Paul added a comment -

        I thought of this the other day. It's kinda already done, no?

        but in the solrconfig.xml all the responsewriters are specified with FQN . Going forward, we can avoid using FQN

        Show
        Noble Paul added a comment - I thought of this the other day. It's kinda already done, no? but in the solrconfig.xml all the responsewriters are specified with FQN . Going forward, we can avoid using FQN
        Hide
        Erik Hatcher added a comment -

        but in the solrconfig.xml all the responsewriters are specified with FQN . Going forward, we can avoid using FQN

        well, technically, in the solrconfig.xml that we distribute to the world, the one kitchen sink example solrconfig.xml, they're all commented out and pre-registered. so it's just an "example".

        Show
        Erik Hatcher added a comment - but in the solrconfig.xml all the responsewriters are specified with FQN . Going forward, we can avoid using FQN well, technically, in the solrconfig.xml that we distribute to the world, the one kitchen sink example solrconfig.xml, they're all commented out and pre-registered. so it's just an "example".
        Hide
        Hoss Man added a comment -

        instead of using the FQN of classes we should use the solr.<SimpleClassname> notation in solrconfig.xml. That would have enabled us moving packages easily

        I've been biting my tounge on that aspect of things. Once upon a time, in the early days of incubation, the example config didn't contain any FQNs, but there were some vocal people arguing that it was missleading to suggest that a "class" named "solr.Foo" existed (the implication being that it would confuse users who understood java and expected that to be a real class/package name.

        I'm more then happy to switch back to using "solr.Foo" everywhere in the example configs, but there have also been some threads out there in the past pointing out that using FQNs can speed up core initialization (when people are dynamicly creating lots of cores) so we shouldn't/can't ever assume that people are only using solr.Foo even if that's all we start using in the examples.

        And lastly...

        well, technically, in the solrconfig.xml that we distribute to the world, the one kitchen sink example solrconfig.xml, they're all commented out and pre-registered. so it's just an "example".

        ... most of them are commented out, not all.

        Besides which: even if it's just an "example" it would be pretty shitty to break that example in the very next release.

        Show
        Hoss Man added a comment - instead of using the FQN of classes we should use the solr.<SimpleClassname> notation in solrconfig.xml. That would have enabled us moving packages easily I've been biting my tounge on that aspect of things. Once upon a time, in the early days of incubation, the example config didn't contain any FQNs, but there were some vocal people arguing that it was missleading to suggest that a "class" named "solr.Foo" existed (the implication being that it would confuse users who understood java and expected that to be a real class/package name. I'm more then happy to switch back to using "solr.Foo" everywhere in the example configs, but there have also been some threads out there in the past pointing out that using FQNs can speed up core initialization (when people are dynamicly creating lots of cores) so we shouldn't/can't ever assume that people are only using solr.Foo even if that's all we start using in the examples. And lastly... well, technically, in the solrconfig.xml that we distribute to the world, the one kitchen sink example solrconfig.xml, they're all commented out and pre-registered. so it's just an "example". ... most of them are commented out, not all. Besides which: even if it's just an "example" it would be pretty shitty to break that example in the very next release.
        Hide
        Noble Paul added a comment -

        but there have also been some threads out there in the past pointing out that using FQNs can speed up core initialization

        This is resolved SOLR-921

        Show
        Noble Paul added a comment - but there have also been some threads out there in the past pointing out that using FQNs can speed up core initialization This is resolved SOLR-921
        Hide
        Ryan McKinley added a comment -
        .Besides which: even if it's just an "example" it would be pretty shitty to break that example in the very next release.

        Agreed – we will make sure old FQNs work (until the next major release), but moving forward, we should remove FQN from schema.xml so this is less of an issue in the future.

        Show
        Ryan McKinley added a comment - .Besides which: even if it's just an "example" it would be pretty shitty to break that example in the very next release. Agreed – we will make sure old FQNs work (until the next major release), but moving forward, we should remove FQN from schema.xml so this is less of an issue in the future.
        Hide
        Ryan McKinley added a comment -

        Nobel, this issue is assigned to you? Do you want to take care of it? If not I can...

        Patches won't work well since it will be a few steps in svn to make sure the history is maintained:
        1. svn move the files to a new location, update references etc
        2. commit
        3. add stub files in the location where the old files were
        4. commit

        Show
        Ryan McKinley added a comment - Nobel, this issue is assigned to you? Do you want to take care of it? If not I can... Patches won't work well since it will be a few steps in svn to make sure the history is maintained: 1. svn move the files to a new location, update references etc 2. commit 3. add stub files in the location where the old files were 4. commit
        Hide
        Ryan McKinley added a comment -

        I made a few changes to /trunk, but we are still missing:
        1. error messages
        2. CHANGES.txt
        3. FQN seems to be the only thing that worked easily. Need to look into it more to see what we can do with SolrResourceLoader

        Now that the main changes are in, attaching patces is useful

        Show
        Ryan McKinley added a comment - I made a few changes to /trunk, but we are still missing: 1. error messages 2. CHANGES.txt 3. FQN seems to be the only thing that worked easily. Need to look into it more to see what we can do with SolrResourceLoader Now that the main changes are in, attaching patces is useful
        Hide
        Noble Paul added a comment -

        FQN seems to be the only thing that worked easily.

        you missed the period after the package name . I just committed

        Show
        Noble Paul added a comment - FQN seems to be the only thing that worked easily. you missed the period after the package name . I just committed
        Hide
        Chris A. Mattmann added a comment -

        Hi Ryan:

        Thanks so much for moving on this!

        Patch forthcoming that addresses:

        1. error messages
        2. CHANGES.txt

        I think you handled 3, right?

        Thanks, again and I'll try and throw up the patch for 1/2 tomorrow morning pacific time.

        Cheers,
        Chris

        Show
        Chris A. Mattmann added a comment - Hi Ryan: Thanks so much for moving on this! Patch forthcoming that addresses: 1. error messages 2. CHANGES.txt I think you handled 3, right? Thanks, again and I'll try and throw up the patch for 1/2 tomorrow morning pacific time. Cheers, Chris
        Hide
        Erik Hatcher added a comment -

        I've encountered an issue with this refactoring, with some local toy request handlers I had laying around:

        //..
        import org.apache.solr.request.SolrQueryResponse;
        
        
        public class HighlightingHandler extends RequestHandlerBase {
          @Override
          public void handleRequestBody(SolrQueryRequest req, SolrQueryResponse rsp) throws Exception....
        
        // ...
        

        This class no longer compiled. Nor if I had a binary version of this and upgraded Solr would it have run.

        I'm just sayin.

        No big deal for me, as I'm refactoring my code. I guess it's the price you have to pay for change. But deprecating simply wasn't good enough to keep things that were working working.

        Show
        Erik Hatcher added a comment - I've encountered an issue with this refactoring, with some local toy request handlers I had laying around: //.. import org.apache.solr.request.SolrQueryResponse; public class HighlightingHandler extends RequestHandlerBase { @Override public void handleRequestBody(SolrQueryRequest req, SolrQueryResponse rsp) throws Exception.... // ... This class no longer compiled. Nor if I had a binary version of this and upgraded Solr would it have run. I'm just sayin. No big deal for me, as I'm refactoring my code. I guess it's the price you have to pay for change. But deprecating simply wasn't good enough to keep things that were working working.
        Hide
        Chris A. Mattmann added a comment -

        Hey Erik:

        You just forgot to press the easy button!

        http://bit.ly/zog3s

        Cheers,
        Chris

        Show
        Chris A. Mattmann added a comment - Hey Erik: You just forgot to press the easy button! http://bit.ly/zog3s Cheers, Chris
        Hide
        Chris A. Mattmann added a comment -
        • so I finally found that bit of time I was looking for to wrap this up. Ryan, this should take care of 1 and 2 that we were waiting on to close this out. My +1 to wrap it up.
        Show
        Chris A. Mattmann added a comment - so I finally found that bit of time I was looking for to wrap this up. Ryan, this should take care of 1 and 2 that we were waiting on to close this out. My +1 to wrap it up.
        Hide
        Hoss Man added a comment -

        Erik: can you elaborate on what compilation error you got? is the problem just that we don't have a deprecated version of RequestHandlerBase in o.a.s.request?

        Chris: your SOLR-1602.Mattmann.wrapup.031010.patch.txt patch has some weird stuff in it – in particular a whole lot of previously deprecated "*Params" classes now log messages saying that they are deprecated and to use the version in "org.apache.solr.response" but that's not where the valid versions of those classes are located.

        Show
        Hoss Man added a comment - Erik: can you elaborate on what compilation error you got? is the problem just that we don't have a deprecated version of RequestHandlerBase in o.a.s.request? Chris: your SOLR-1602 .Mattmann.wrapup.031010.patch.txt patch has some weird stuff in it – in particular a whole lot of previously deprecated "*Params" classes now log messages saying that they are deprecated and to use the version in "org.apache.solr.response" but that's not where the valid versions of those classes are located.
        Hide
        Chris A. Mattmann added a comment -

        Hoss: Ugh, think I just lost track of what classes I was throwing the message into. I'll take it out of the Params classes and only deal with those classes that are part of this patch.

        New patch forthcoming.

        Cheers,
        Chris

        Show
        Chris A. Mattmann added a comment - Hoss: Ugh, think I just lost track of what classes I was throwing the message into. I'll take it out of the Params classes and only deal with those classes that are part of this patch. New patch forthcoming. Cheers, Chris
        Hide
        Chris A. Mattmann added a comment -

        Hey Hoss:

        I went ahead and got some time for an updated patch. I think it should address your concerns.

        Thanks!

        Cheers,
        Chris

        Show
        Chris A. Mattmann added a comment - Hey Hoss: I went ahead and got some time for an updated patch. I think it should address your concerns. Thanks! Cheers, Chris
        Hide
        Hoss Man added a comment -

        Bulk updating 240 Solr issues to set the Fix Version to "next" per the process outlined in this email...

        http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E

        Selection criteria was "Unresolved" with a Fix Version of 1.5, 1.6, 3.1, or 4.0. email notifications were suppressed.

        A unique token for finding these 240 issues in the future: hossversioncleanup20100527

        Show
        Hoss Man added a comment - Bulk updating 240 Solr issues to set the Fix Version to "next" per the process outlined in this email... http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E Selection criteria was "Unresolved" with a Fix Version of 1.5, 1.6, 3.1, or 4.0. email notifications were suppressed. A unique token for finding these 240 issues in the future: hossversioncleanup20100527
        Hide
        Hoss Man added a comment -

        Thanks Chris, that looks good.

        Trunk...
        Committed revision 950835.

        branch 3x...
        Committed revision 950838.

        Show
        Hoss Man added a comment - Thanks Chris, that looks good. Trunk... Committed revision 950835. branch 3x... Committed revision 950838.
        Hide
        Grant Ingersoll added a comment -

        Bulk close for 3.1.0 release

        Show
        Grant Ingersoll added a comment - Bulk close for 3.1.0 release

          People

          • Assignee:
            Ryan McKinley
            Reporter:
            Chris A. Mattmann
          • Votes:
            2 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development