Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-18576

kamelets - JSon Property value can't finish with }}

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 3.20.0
    • camel-core
    • None
    • Unknown

    Description

      Hi, I'm using Quarkus Camel 2.12.3.Final with YAML DSL and Kamelets and I have a problem passing a JSON value through URI as a query parameter when trying to use the elasticsearch-search-source with a basic query:

      {"query":{"match_all":{}}} 

      Example on the YAML DSL:

      • Scenario 1
        query: '{"query":{"match_all":{}}}'

        Gives

        java.lang.IllegalArgumentException: Missing {{ from the text: {"query":{"match_all":{}}}
        
      • Scenario 2
        query: '{{myQuery}}'

        Gives

        java.lang.IllegalArgumentException: Missing {{ from the text: {"query":{"match_all":{}}}
        
      • Scenario 3
        query: '#property:myQuery'

        Gives

        java.lang.IllegalArgumentException: Property with key myQuery not found by properties component
        

        But using debug I've checked that the above exception occurs due to the same

        java.lang.IllegalArgumentException: Missing {{ from the text: {"query":{"match_all":{}}}
        
      • Scenario 4
        I've also tried to change the Kamelet YAML definition to use the RAW value directly
        - set-body: 
          constant: "RAW({{myQuery}})"

        With the same result

        java.lang.IllegalArgumentException: Missing {{ from the text: {"query":{"match_all":{}}}
        

      I have a workaround if I add spaces in JSON like below it works in scenarios 1 and 2.

      { "query": { "match_all": { } } } 

      My point is, why do I have to change the JSON value to adapt to the Camel parser? Also, there also might exist some other use case where the value can't be changed.

      Why don't you simply have a function that does not try to resolve properties as placeholders recursively? For example:

      query: '{{raw:myQuery}}'

      Or with a default value

      query: '{{raw:myQuery:{"query":{"match_all":{}}}}}'

      Or at least have a way to escape curly brackets.

      Attachments

        Issue Links

          Activity

            davsclaus Claus Ibsen added a comment -

            Can you paste your route where you use this

            davsclaus Claus Ibsen added a comment - Can you paste your route where you use this
            davsclaus Claus Ibsen added a comment -

            Ah okay, so its the property placeholder that assumes it was a {{ }} pair, where you have no end pair.

            davsclaus Claus Ibsen added a comment - Ah okay, so its the property placeholder that assumes it was a {{ }} pair, where you have no end pair.
            davsclaus Claus Ibsen added a comment -

            I have reproduced this. So its because of using kamelets where the query is passed in as {{ }} placeholder, and that it has json that have }} as well, and the property placeholder performs nested placeholder (eg a placeholder has also a placeholder, recursive).

            So we need to "think" to find a better way.

            davsclaus Claus Ibsen added a comment - I have reproduced this. So its because of using kamelets where the query is passed in as {{ }} placeholder, and that it has json that have }} as well, and the property placeholder performs nested placeholder (eg a placeholder has also a placeholder, recursive). So we need to "think" to find a better way.
            MarcoMartins Marco Martins added a comment -

            Yeah, it is because of the placeholder recursivity feature together with the JSON having }} in its body, which, gets the Camel parser confused trying to find the beginning of a placeholder that does not exist.

            Thank you for your availability.

            MarcoMartins Marco Martins added a comment - Yeah, it is because of the placeholder recursivity feature together with the JSON having }} in its body, which, gets the Camel parser confused trying to find the beginning of a placeholder that does not exist. Thank you for your availability.
            davsclaus Claus Ibsen added a comment -

            You can turn off nested on the properties component, but we need to find a better out of the box way for kamelets

            davsclaus Claus Ibsen added a comment - You can turn off nested on the properties component, but we need to find a better out of the box way for kamelets
            MarcoMartins Marco Martins added a comment -

            Can you give me an example of how I could turn off the nested on configuration option? I can't find any reference to this in ELASTICSEARCH REST, PropertiesComponent documentation nor do I see any obvious property analyzing the ElasticsearchComponent.java itself.

            MarcoMartins Marco Martins added a comment - Can you give me an example of how I could turn off the nested on configuration option? I can't find any reference to this in ELASTICSEARCH REST , PropertiesComponent documentation nor do I see any obvious property analyzing the ElasticsearchComponent.java itself.
            davsclaus Claus Ibsen added a comment -

            See the linked JIRA, and its a new feature in 3.20

            davsclaus Claus Ibsen added a comment - See the linked JIRA, and its a new feature in 3.20
            MarcoMartins Marco Martins added a comment -

            Thank you, I did look into the changes, and I think I've found a discrepancy in one of the commits. On the one that added the new configuration to the documentation camel.component.properties.nested-placeholder mention that the default value is false, although, in  PropertiesComponent.java the default value is true 

            private boolean nestedPlaceholder = true; 

            which makes sense for using old library version behavior as default. 
             

            MarcoMartins Marco Martins added a comment - Thank you, I did look into the changes, and I think I've found a discrepancy in one of the commits. On the one that added the new configuration to the documentation  camel.component.properties.nested-placeholder  mention that the default value is  false , although, in   PropertiesComponent.java the default value is  true   private boolean nestedPlaceholder = true ; which makes sense for using old library version behavior as default.   
            nfilotto Nicolas Filotto added a comment - - edited

            davsclaus I proposed in this PR https://github.com/apache/camel/pull/8521 a fix based on an escape character (backslash) WDYT? 

             

            PS: Sorry to work on your ticket, initially, I just wanted to test an idea. 

            nfilotto Nicolas Filotto added a comment - - edited davsclaus I proposed in this PR https://github.com/apache/camel/pull/8521 a fix based on an escape character (backslash) WDYT?    PS: Sorry to work on your ticket, initially, I just wanted to test an idea. 
            davsclaus Claus Ibsen added a comment - - edited

            Nicolas, that look good for users that can edit their input and escape the JSon input.

            But for out of the box, we should ideally have a way.
            I have run a full test with nested placeholder disabled and we do not use it very much - only in some special tests for camel-jasypt.

            I have also not seen real world use-cases with nested placeholders. Its only when you may go very fine grained and have a placeholder for hostname, and another for port, and then have a 3rd where you concat them together.

            But lets keep this open for a while and think about if we can have a nicer solution out of the box - but even then this problem is only when you have double {{ and }} which ought to be more rare for even JSon payloads.

            MarcoMartins do you see the double `{{` used more often in your ES queries ?

            davsclaus Claus Ibsen added a comment - - edited Nicolas, that look good for users that can edit their input and escape the JSon input. But for out of the box, we should ideally have a way. I have run a full test with nested placeholder disabled and we do not use it very much - only in some special tests for camel-jasypt. I have also not seen real world use-cases with nested placeholders. Its only when you may go very fine grained and have a placeholder for hostname, and another for port, and then have a 3rd where you concat them together. But lets keep this open for a while and think about if we can have a nicer solution out of the box - but even then this problem is only when you have double {{ and }} which ought to be more rare for even JSon payloads. MarcoMartins do you see the double `{{` used more often in your ES queries ?
            MarcoMartins Marco Martins added a comment - - edited

            davsclaus I don't see the `{{` pattern appear often on ES queries since they're JSON payloads and JSON can't have two `{{`, although, they could appear as a field value to be searched. 

            This issue occurred without the {{ since Camel was looking for }} pattern which occurs very often in JSON, and if found it would consider that it was a property placeholder and when looking for the beginning of the "false property" it would not find it since there was no `{{`.

            MarcoMartins Marco Martins added a comment - - edited davsclaus I don't see the `{{` pattern appear often on ES queries since they're JSON payloads and JSON can't have two `{{`, although, they could appear as a field value to be searched.  This issue occurred without the {{ since Camel was looking for }} pattern which occurs very often in JSON, and if found it would consider that it was a property placeholder and when looking for the beginning of the "false property" it would not find it since there was no `{{`.

            davsclaus Should I revert the fix that I proposed?

            nfilotto Nicolas Filotto added a comment - davsclaus Should I revert the fix that I proposed?
            davsclaus Claus Ibsen added a comment -

            Nicolas, no its actually a good new functionality that end users can escape property placeholder values to make them literal values and not the begin or end markers

            davsclaus Claus Ibsen added a comment - Nicolas, no its actually a good new functionality that end users can escape property placeholder values to make them literal values and not the begin or end markers
            davsclaus Claus Ibsen added a comment -

            1
            We could introduce myKey?nested=false as a way to turn off nested for a specific placeholder.

            2
            We can also relax the {{ }} check on level 2+ since its the value of the placeholder that cannot always be assumed to be another placeholder

            davsclaus Claus Ibsen added a comment - 1 We could introduce myKey?nested=false as a way to turn off nested for a specific placeholder. 2 We can also relax the {{ }} check on level 2+ since its the value of the placeholder that cannot always be assumed to be another placeholder
            davsclaus Claus Ibsen added a comment -

            Okay so the best we can do for now is

            a) allow to turn of nested placeholder globally
            b) allow escaping property values
            c) turn of nested on specific placeholder

            davsclaus Claus Ibsen added a comment - Okay so the best we can do for now is a) allow to turn of nested placeholder globally b) allow escaping property values c) turn of nested on specific placeholder

            People

              davsclaus Claus Ibsen
              MarcoMartins Marco Martins
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: