Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9657

Create a new TemplateUpdateRequestProcessorFactory

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.3, master (7.0)
    • Component/s: None
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      Unlike other URPs, this will operate on request parameters

      example:

      processor=Template&Template.field=fname:${somefield}some_string${someotherfield}
      

      The actual name of the class is TemplateUpdateProcessorFactory and it is possible to optionally drop the UpdateProcessorfactory part. The Template.field specifies a field name as well as a template. The Template.field parameter is multivalued , so , it is possible to add multiple fields or a multivalued field with same name

        Activity

        Hide
        dsmiley David Smiley added a comment - - edited

        This is nice; it'd come in handy for "lat,lon". Can you add at least a one-liner javadoc to the class? And I like that this can work off of Solr request parameters but why doesn't it also work like all the other ones work – by predefined configuration in solrconfig.xml? I wonder if it's feasible for the URP processing subsystem to be refactored such that all URPs could operate in both modes, similarly to how request handlers can be. It'd be great to not have this inconsistency.

        Show
        dsmiley David Smiley added a comment - - edited This is nice; it'd come in handy for "lat,lon". Can you add at least a one-liner javadoc to the class? And I like that this can work off of Solr request parameters but why doesn't it also work like all the other ones work – by predefined configuration in solrconfig.xml? I wonder if it's feasible for the URP processing subsystem to be refactored such that all URPs could operate in both modes, similarly to how request handlers can be. It'd be great to not have this inconsistency.
        Hide
        noble.paul Noble Paul added a comment -

        I wonder if it's feasible for the URP processing subsystem to be refactored such that all URPs could operate in both modes, similarly to how request handlers can be

        That's the plan

        Show
        noble.paul Noble Paul added a comment - I wonder if it's feasible for the URP processing subsystem to be refactored such that all URPs could operate in both modes, similarly to how request handlers can be That's the plan
        Hide
        ichattopadhyaya Ishan Chattopadhyaya added a comment -

        +1 to this URP (since it takes request parameters for fields).

        Show
        ichattopadhyaya Ishan Chattopadhyaya added a comment - +1 to this URP (since it takes request parameters for fields).
        Hide
        noble.paul Noble Paul added a comment -

        The plan is to have automatic request parameter support for all URPs (wherever possible)

        Show
        noble.paul Noble Paul added a comment - The plan is to have automatic request parameter support for all URPs (wherever possible)
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit c2e031add3d5db2c4e89a5a92afd7bb8cc1f481f in lucene-solr's branch refs/heads/master from Noble Paul
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c2e031a ]

        SOLR-9657: New TemplateUpdateProcessorFactory added

        Show
        jira-bot ASF subversion and git services added a comment - Commit c2e031add3d5db2c4e89a5a92afd7bb8cc1f481f in lucene-solr's branch refs/heads/master from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c2e031a ] SOLR-9657 : New TemplateUpdateProcessorFactory added
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 2fc29c1cd5320322a434ca7f820b69d80536aa5a in lucene-solr's branch refs/heads/branch_6x from Noble Paul
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2fc29c1 ]

        SOLR-9657: New TemplateUpdateProcessorFactory added

        Show
        jira-bot ASF subversion and git services added a comment - Commit 2fc29c1cd5320322a434ca7f820b69d80536aa5a in lucene-solr's branch refs/heads/branch_6x from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2fc29c1 ] SOLR-9657 : New TemplateUpdateProcessorFactory added
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 180efba9b59190e6b44b1a2428e40617c52bb74c in lucene-solr's branch refs/heads/branch_6x from Noble Paul
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=180efba ]

        SOLR-9657: New TemplateUpdateProcessorFactory added

        Show
        jira-bot ASF subversion and git services added a comment - Commit 180efba9b59190e6b44b1a2428e40617c52bb74c in lucene-solr's branch refs/heads/branch_6x from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=180efba ] SOLR-9657 : New TemplateUpdateProcessorFactory added
        Hide
        noble.paul Noble Paul added a comment -

        Is there DIH contrib dependency here?

        The code is moved from DIH to core so that the same code can be used in both places. So, the answer is No.

        Show
        noble.paul Noble Paul added a comment - Is there DIH contrib dependency here? The code is moved from DIH to core so that the same code can be used in both places. So, the answer is No.
        Hide
        dsmiley David Smiley added a comment -

        Noble Paul did you disagree with my feedback on adding at least a one-liner class javadoc? It feels rude to me to have simultaneously committed it without it and not responding.

        Show
        dsmiley David Smiley added a comment - Noble Paul did you disagree with my feedback on adding at least a one-liner class javadoc? It feels rude to me to have simultaneously committed it without it and not responding.
        Hide
        noble.paul Noble Paul added a comment -

        Sorry about that David Smiley , I should have responded to your comment and I missed it. I'm fine with adding javadocs. Just that it was not clear as to where you wanted it.

        Is it to help a user who would use this URP? How many users of Solr are java users and are likely to use javadocs. Most users rely on the the ref guide to learn how to use a component. Honestly, a one liner is not really useful. You really need a few examples at least to learn to use it.

        Is it to help the Solr developer? I agree with you.

        That being said . I'm not yet resolving this. There was a concern raised by Erik hatcher about the variable naming

        Show
        noble.paul Noble Paul added a comment - Sorry about that David Smiley , I should have responded to your comment and I missed it. I'm fine with adding javadocs. Just that it was not clear as to where you wanted it. Is it to help a user who would use this URP? How many users of Solr are java users and are likely to use javadocs. Most users rely on the the ref guide to learn how to use a component. Honestly, a one liner is not really useful. You really need a few examples at least to learn to use it. Is it to help the Solr developer? I agree with you. That being said . I'm not yet resolving this. There was a concern raised by Erik hatcher about the variable naming
        Hide
        dsmiley David Smiley added a comment -

        Fair point RE refguide. Nevertheless, IMO as a matter of universal good developer practices I think every top-level non-test class ought to have at least one sentence said about it, aimed at any other developer (whoever they may be) stumbling over it – either via looking at the actual source, or via docs pop-up in their IDE. IMO it's a shame the Solr codebase has gotten to the point that so little has such a bare minimum. For this one I'll write it for you; feel free to edit: "Adds new fields to documents based on a template pattern specified via template.field request parameters (multi-valued)."

        Alexandre Rafalovitch do you think this is at all useful? I know you work on adding docs to URPs 'n things.

        Show
        dsmiley David Smiley added a comment - Fair point RE refguide. Nevertheless, IMO as a matter of universal good developer practices I think every top-level non-test class ought to have at least one sentence said about it , aimed at any other developer (whoever they may be) stumbling over it – either via looking at the actual source, or via docs pop-up in their IDE. IMO it's a shame the Solr codebase has gotten to the point that so little has such a bare minimum. For this one I'll write it for you; feel free to edit: "Adds new fields to documents based on a template pattern specified via template.field request parameters (multi-valued)." Alexandre Rafalovitch do you think this is at all useful? I know you work on adding docs to URPs 'n things.
        Hide
        arafalov Alexandre Rafalovitch added a comment -

        I absolutely think we need to have the Javadoc. I am biased of course, as that's what's used for generating http://www.solr-start.com/info/update-request-processors/ . But also, it is a way to keep the documentation with code, when the changes are made.

        And as to whether the documentation is required at all, I had to look at the patch to see what it actually does, the description of JIRA was not clear enough. Specifically *operate on request parameters" I read that perhaps something gets put back into parameter map.

        As an aside, once this is finished, we can probably resolve SOLR-9650 too.

        Show
        arafalov Alexandre Rafalovitch added a comment - I absolutely think we need to have the Javadoc. I am biased of course, as that's what's used for generating http://www.solr-start.com/info/update-request-processors/ . But also, it is a way to keep the documentation with code, when the changes are made. And as to whether the documentation is required at all, I had to look at the patch to see what it actually does, the description of JIRA was not clear enough. Specifically *operate on request parameters" I read that perhaps something gets put back into parameter map. As an aside, once this is finished, we can probably resolve SOLR-9650 too.
        Hide
        noble.paul Noble Paul added a comment -

        Relying on javadocs for ref guide is not a common practice. The purpose of javadocs is different . It makes little sense when a feature involves multiple source files. Javadocs must be used to communicate to fellow developers

        Show
        noble.paul Noble Paul added a comment - Relying on javadocs for ref guide is not a common practice. The purpose of javadocs is different . It makes little sense when a feature involves multiple source files. Javadocs must be used to communicate to fellow developers
        Hide
        arafalov Alexandre Rafalovitch added a comment -

        Relying on Javadocs and other code-comments for multiple various things is a common practice out there. Including - as a feature request - for our own code (see SOLR-555 by Hoss Man).

        Show
        arafalov Alexandre Rafalovitch added a comment - Relying on Javadocs and other code-comments for multiple various things is a common practice out there. Including - as a feature request - for our own code (see SOLR-555 by Hoss Man ).
        Hide
        noble.paul Noble Paul added a comment -

        That's going nowhere . In v2 api , we have documentation added to specification files. That's going to be the way forward

        Show
        noble.paul Noble Paul added a comment - That's going nowhere . In v2 api , we have documentation added to specification files. That's going to be the way forward
        Hide
        arafalov Alexandre Rafalovitch added a comment -

        The JIRA has not gone very far because it was hard to do (I just updated my own findings there). solr-start, based on similar principles is up and running, serving user needs and getting good views. And it is finding components that were missed both by Reference Guide AND Release Notes, exactly because it relies on shipped code alone. And, things like cross-reference to usage example, is not something that Reference Guide can maintain by hand with reasonable effort.

        The V2 API might be the magical unicorn when it actually lands, but - as my own comments there indicate - I could not make heads or tails out of it in a meanwhile. So, let's focus on reality, especially the reality for 6.x.

        As open-source Solr developers, we are combining 3 roles that are often separated in commercial organizations: Developers of new - blue-sky - features (master), maintainers and developers of the current features (6.x currently), and tech-support for whatever was already released (6.x.y, 5.x.y). Your suggested approach is about blue-sky which is great, but not sufficient. Let's just take care of these other aspects too, so other people don't have to.

        Show
        arafalov Alexandre Rafalovitch added a comment - The JIRA has not gone very far because it was hard to do (I just updated my own findings there). solr-start, based on similar principles is up and running, serving user needs and getting good views. And it is finding components that were missed both by Reference Guide AND Release Notes, exactly because it relies on shipped code alone. And, things like cross-reference to usage example, is not something that Reference Guide can maintain by hand with reasonable effort. The V2 API might be the magical unicorn when it actually lands, but - as my own comments there indicate - I could not make heads or tails out of it in a meanwhile. So, let's focus on reality, especially the reality for 6.x. As open-source Solr developers, we are combining 3 roles that are often separated in commercial organizations: Developers of new - blue-sky - features (master), maintainers and developers of the current features (6.x currently), and tech-support for whatever was already released (6.x.y, 5.x.y). Your suggested approach is about blue-sky which is great, but not sufficient. Let's just take care of these other aspects too, so other people don't have to.
        Hide
        romseygeek Alan Woodward added a comment -

        I think this is causing test failures? https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1987/

        Show
        romseygeek Alan Woodward added a comment - I think this is causing test failures? https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1987/
        Hide
        noble.paul Noble Paul added a comment -

        Possible. I'll debug it

        Show
        noble.paul Noble Paul added a comment - Possible. I'll debug it
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 6893595f340b748a9c5e5c3884704e71275deab4 in lucene-solr's branch refs/heads/master from Noble Paul
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6893595 ]

        SOLR-9657: Addressing test failures

        Show
        jira-bot ASF subversion and git services added a comment - Commit 6893595f340b748a9c5e5c3884704e71275deab4 in lucene-solr's branch refs/heads/master from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6893595 ] SOLR-9657 : Addressing test failures
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 0ab908694ef503ea45b3a59ccd3707b54978fd5f in lucene-solr's branch refs/heads/branch_6x from Noble Paul
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0ab9086 ]

        SOLR-9657: Addressing test failures

        Show
        jira-bot ASF subversion and git services added a comment - Commit 0ab908694ef503ea45b3a59ccd3707b54978fd5f in lucene-solr's branch refs/heads/branch_6x from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0ab9086 ] SOLR-9657 : Addressing test failures
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 127bf9f772468cbc94478ad67d54652001b175e0 in lucene-solr's branch refs/heads/master from Noble Paul
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=127bf9f ]

        SOLR-9657: Use cache for templates

        Show
        jira-bot ASF subversion and git services added a comment - Commit 127bf9f772468cbc94478ad67d54652001b175e0 in lucene-solr's branch refs/heads/master from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=127bf9f ] SOLR-9657 : Use cache for templates
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit b1e2d02ec3e2d87bd406190f1600dccd679901f9 in lucene-solr's branch refs/heads/branch_6x from Noble Paul
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b1e2d02 ]

        SOLR-9657: Use cache for templates

        Show
        jira-bot ASF subversion and git services added a comment - Commit b1e2d02ec3e2d87bd406190f1600dccd679901f9 in lucene-solr's branch refs/heads/branch_6x from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b1e2d02 ] SOLR-9657 : Use cache for templates
        Hide
        arafalov Alexandre Rafalovitch added a comment -

        The Javadoc comment format is not correct for the class and does not build the description into the resulting Javadoc.

        Basically, you have to use the correct multi-line comment delimiters, as per the rules. Or see the parent or any other class that have them.

        Show
        arafalov Alexandre Rafalovitch added a comment - The Javadoc comment format is not correct for the class and does not build the description into the resulting Javadoc. Basically, you have to use the correct multi-line comment delimiters, as per the rules . Or see the parent or any other class that have them.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit fdb4dd3b322a517ff7f9df2ef64001120e89854c in lucene-solr's branch refs/heads/branch_6x from Alexandre Rafalovitch
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=fdb4dd3 ]

        SOLR-9657: Fixed Javadocs and added example

        Show
        jira-bot ASF subversion and git services added a comment - Commit fdb4dd3b322a517ff7f9df2ef64001120e89854c in lucene-solr's branch refs/heads/branch_6x from Alexandre Rafalovitch [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=fdb4dd3 ] SOLR-9657 : Fixed Javadocs and added example
        Hide
        arafalov Alexandre Rafalovitch added a comment -

        This matches commit 9d692cde53c25230d6db2663816f313cf356535b on the master that - for some reasons - did not link up with Jira.

        Show
        arafalov Alexandre Rafalovitch added a comment - This matches commit 9d692cde53c25230d6db2663816f313cf356535b on the master that - for some reasons - did not link up with Jira.
        Hide
        shalinmangar Shalin Shekhar Mangar added a comment -

        Closing after 6.3.0 release.

        Show
        shalinmangar Shalin Shekhar Mangar added a comment - Closing after 6.3.0 release.

          People

          • Assignee:
            noble.paul Noble Paul
            Reporter:
            noble.paul Noble Paul
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development