OK. Here's my attempt to fix this issue.
This patch introduces a new token type which I call "Textblock". Textblocks can be marked like this:
#% here you can have any text, characters etc. %#
Velocity will not attempt to parse anything between "#% " and " %#" and will output content between these tokens as such. Note that one whitespace is part of each token.
With this feature it is easy to deal with issues described e.g. in
Reasons for selecting this token format:
1. It's short and thus easy to use.
2. It is quite unlikely that you need to have string " %#" inside the area you want to "escape" with a Textblock. Thus there is no need for a custom end token (which would have been tricky to implement). Note that you can have %# inside the area as long as it doesn't have whitespace in front of it.
This patch also includes some performance tweaks made to the parser:
1. Transformation StringBuffer -> StrBuilder is done with Ant script after the generation.
2. Parser state is stored using a custom inner class instead of Hashtable which wastes space.
Note that I'm a total newbie with JavaCC (I took a closer look at it today for the first time) so the parser might not be as efficient as possible. I had to use the JavaCC MORE feature when in IN_TEXTBLOCK mode. I'm not sure if this is the only way.
NOTICE! You must use "ant parser" command to process the Parser.jjt which then generates Parser.java etc. I didn't include the generated files in this patch since TortoiseSVN complained something about invalid linefeeds. I noticed that JavaCC 4.2 is incompatible with existing classes. JavaCC 4.0 worked OK.
Tweaks to the build.xml:
1. Now it's possible to execute just one Test Case by defining the test case name:
ant -Dtestcase=org.apache.velocity.test.TextblockTestCase test
This makes debugging a lot faster.
2. Generated ParserTokenManager is tweaked to use StrBuilder by simple search & replace.