Velocity
  1. Velocity
  2. VELOCITY-587

Backslashes throw lexical errors in double-quoted strings

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.6
    • Fix Version/s: 1.6
    • Component/s: None
    • Labels:
      None

      Description

      The VTL doesn't currently provide a way to express strings containing a single backslash ()

      #set($backslash="\")

      results in a lexical error during template parsing.

      #set($backslash="
      ")

      results in $backslash being assigned a string containing two backslashes (
      )

      It appears the only workaround available now is to construct a variable containing just a single backslash with this idiom:

      #set($twobackslashes="
      ")
      #set($backslash=$twobackslashes.substring(0,1))

      Even if this can't be fixed in the parser, the documentation should be updated to define a preferred idiom for obtaining a string containing a single backslash.

      jon.

      note: this issue partially duplicates a comment by Guido Deinhammer on issue #454 regarding other escaping issues. I have raised a separate issue, so that this one can be addressed separately, if desired.

        Issue Links

          Activity

          Hide
          Jon Seymour added a comment -

          Fixed syntax errors in the examples. e.g. #(set => #set(

          Show
          Jon Seymour added a comment - Fixed syntax errors in the examples. e.g. #(set => #set(
          Hide
          Etienne Massip added a comment -

          maybe you should try single quotes (means string is not parsed) :

          #set($backslash='\')

          see http://velocity.apache.org/engine/devel/vtl-reference-guide.html#aset_-_establishes_the_value_of_a_reference

          Show
          Etienne Massip added a comment - maybe you should try single quotes (means string is not parsed) : #set($backslash='\') see http://velocity.apache.org/engine/devel/vtl-reference-guide.html#aset_-_establishes_the_value_of_a_reference
          Hide
          Nathan Bubna added a comment -

          Etienne is right. You can do this with single quotes.

          Show
          Nathan Bubna added a comment - Etienne is right. You can do this with single quotes.
          Hide
          Christopher Schultz added a comment -

          Even though it's possible to use the single-quotes to get a single backslash, escaping within double-quotes does seem a bit inconsistent.

          Given that the backslash-escaping idiom has been adopted from programming languages, it would be nice to be consistent with their behavior. That is,
          ought to yield \ in all cases, possibly including single-quoted strings.

          Show
          Christopher Schultz added a comment - Even though it's possible to use the single-quotes to get a single backslash, escaping within double-quotes does seem a bit inconsistent. Given that the backslash-escaping idiom has been adopted from programming languages, it would be nice to be consistent with their behavior. That is, ought to yield \ in all cases, possibly including single-quoted strings.
          Hide
          Nathan Bubna added a comment -

          There's plenty of inconsistencies and shortcomings in Velocity's escaping support. Personally i never use or recommend the use of \ for escaping in either general template content nor interpolated strings (where it is most broken!) for that reason. Of course, this years-long practice has significantly reduced my motivation to help remedy this, as it is reflexive to avoid the matter entirely.

          On second thought, there is a bug here, in that neither #set( $foo = "\" ) nor #set( $foo = "\." ) should cause the lexical errors that they do, though that is somewhat different than title. I suppose i should read more thoroughly. Of course, this is Monday and i've loads of email to rush through... :-/ Sorry, Jon. You've got a bug here. I'll just change the title to fit better...

          Show
          Nathan Bubna added a comment - There's plenty of inconsistencies and shortcomings in Velocity's escaping support. Personally i never use or recommend the use of \ for escaping in either general template content nor interpolated strings (where it is most broken!) for that reason. Of course, this years-long practice has significantly reduced my motivation to help remedy this, as it is reflexive to avoid the matter entirely. On second thought, there is a bug here, in that neither #set( $foo = "\" ) nor #set( $foo = "\." ) should cause the lexical errors that they do, though that is somewhat different than title. I suppose i should read more thoroughly. Of course, this is Monday and i've loads of email to rush through... :-/ Sorry, Jon. You've got a bug here. I'll just change the title to fit better...
          Hide
          Jon Seymour added a comment -

          Etienne, thanks for pointing out the use of single quotes - I should have read the doco better. If I had had my shell-script hat on this would have occurred to me but since I didn't have the VTL reference handy when I was struggling with this issue, I had incorrectly assumed that '\' would be treated as a Java character reference.

          Nathan: I guess it doesn't really surprise me that #set($foo="\") is a parsing error, if \'s are meant to be treated as escape characters when parsing strings. More surprising was the failure of #set($foo="\.") since based on other experiments I expected the \ to be preserved.

          It would be good to have some clarity on what \ is meant to do in double quoted strings since I don't think (but I may be wrong) this is specified clearly anywhere.

          Perhaps it would be useful to include a section about the behaviour of backslashes in double quoted strings in the user's guide?

          jon.

          Show
          Jon Seymour added a comment - Etienne, thanks for pointing out the use of single quotes - I should have read the doco better. If I had had my shell-script hat on this would have occurred to me but since I didn't have the VTL reference handy when I was struggling with this issue, I had incorrectly assumed that '\' would be treated as a Java character reference. Nathan: I guess it doesn't really surprise me that #set($foo="\") is a parsing error, if \'s are meant to be treated as escape characters when parsing strings. More surprising was the failure of #set($foo="\.") since based on other experiments I expected the \ to be preserved. It would be good to have some clarity on what \ is meant to do in double quoted strings since I don't think (but I may be wrong) this is specified clearly anywhere. Perhaps it would be useful to include a section about the behaviour of backslashes in double quoted strings in the user's guide? jon.
          Hide
          Nathan Bubna added a comment -

          If you ask me, the only thing in double-quoted strings that should behave differently than it does in a template in general is the double quote (which ends that string and therefor cannot currently be directly put in double-quoted string). A backslash in a double string should follow the same rules as a backslash in a template in general (i.e. only meaningful for escaping valid references/directives/macros). It definitely shouldn't be a parsing error and ideally should not warrant a section in the user's guide.

          Show
          Nathan Bubna added a comment - If you ask me, the only thing in double-quoted strings that should behave differently than it does in a template in general is the double quote (which ends that string and therefor cannot currently be directly put in double-quoted string). A backslash in a double string should follow the same rules as a backslash in a template in general (i.e. only meaningful for escaping valid references/directives/macros). It definitely shouldn't be a parsing error and ideally should not warrant a section in the user's guide.
          Hide
          Christopher Schultz added a comment -

          IMO, you should be able to be a double-quote into a doubly-quoted string. It sounds like Nathan is saying that the backslash should only be used to escape references and directives/macros. I think the backslash should be extended to act like a more general escape character. For instance, I would expect this to work:

          #set($doubleQuote = "\"")

          Instead of $doubleQuote containing a single double-quote, it contains a backslash plus a single double-quote (at least in Velocity 1.4).

          I would think that it would actually be easier to make the parser treat backslash as a general escape character than to have different rules in different places (the different rules being that backslashes are consumed in some circumstances but not in others).

          Show
          Christopher Schultz added a comment - IMO, you should be able to be a double-quote into a doubly-quoted string. It sounds like Nathan is saying that the backslash should only be used to escape references and directives/macros. I think the backslash should be extended to act like a more general escape character. For instance, I would expect this to work: #set($doubleQuote = "\"") Instead of $doubleQuote containing a single double-quote, it contains a backslash plus a single double-quote (at least in Velocity 1.4). I would think that it would actually be easier to make the parser treat backslash as a general escape character than to have different rules in different places (the different rules being that backslashes are consumed in some circumstances but not in others).
          Hide
          Nathan Bubna added a comment -

          We've had a lot of debates on escaping issues over the years. Feel free to search the archives if you'd like to be caught up on the various arguments. But perhaps it will suffice to point out that the only escaping solutions for strings (double or single quoted) that we have ever achieved a consensus on are VELOCITY-520 and VELOCITY-555. 520 was quickly patched, but sadly, Geir never shared the patch for 555 and no one has stepped up to implement it themselves yet.

          Things like VELOCITY-519 may be considered if/when someone starts work on Velocity Engine 2.0, but not before. And when that time comes, you can be sure that i will always be loudly advocating for the maximum possible consistency between parsing done in interpolated strings and parsing done elsewhere. If we do make the backslash a general escape character, you can bet i'll argue to have it work the same everywhere.

          Show
          Nathan Bubna added a comment - We've had a lot of debates on escaping issues over the years. Feel free to search the archives if you'd like to be caught up on the various arguments. But perhaps it will suffice to point out that the only escaping solutions for strings (double or single quoted) that we have ever achieved a consensus on are VELOCITY-520 and VELOCITY-555 . 520 was quickly patched, but sadly, Geir never shared the patch for 555 and no one has stepped up to implement it themselves yet. Things like VELOCITY-519 may be considered if/when someone starts work on Velocity Engine 2.0, but not before. And when that time comes, you can be sure that i will always be loudly advocating for the maximum possible consistency between parsing done in interpolated strings and parsing done elsewhere. If we do make the backslash a general escape character, you can bet i'll argue to have it work the same everywhere.

            People

            • Assignee:
              Unassigned
              Reporter:
              Jon Seymour
            • Votes:
              1 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development