MyFaces Tomahawk
  1. MyFaces Tomahawk
  2. TOMAHAWK-45

<t:buffer> does not support East Asia characters using UTF-8 encoding

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 1.1.3
    • Fix Version/s: None
    • Component/s: Buffer
    • Labels:
      None
    • Environment:
      Windows XP, JBoss 4.0.3SP1

      Description

      East Asia characters support using UTF-8 encoding (request and response). When rendering into a <t:buffer>, some data conversion corrupts the encoding.

      -------

      It was <t:buffer> that caused the encoding problem. I used <t:buffer> in my code.

      I tried the following simple example without <t:buffer>, it worked fine without problem in typing/displaying back East asia characters. If I wrap the code with <t:buffer>, and output the buffer, it will not work, displaying East Asia characters back as question marks.

      I thought <t:buffer> is just a piece of memory without any character conversion. How does <t:buffer> work exactly? what OutputStream is being used? is it possibly involving data conversion? If yes, is there any workaround?

      Thanks!

      -----------without <t:buffer>, works!---

      <%@ page contentType="text/html; charset=UTF-8"%>
      <%@ taglib uri="http://java.sun.com/jsf/html" prefix="h"%>
      <%@ taglib uri="http://java.sun.com/jsf/core" prefix="f"%>
      <%@ taglib uri="http://myfaces.apache.org/tomahawk" prefix="t"%>
      <HTML>
      <head>
      </head>
      <f:view>
      <h:outputText value="#

      {testBean.name}" />
      <h:form>
      <h:inputText value="#{testBean.name}

      "/>
      <h:commandButton value="Submit" />
      </h:form>
      </f:view>
      </HTML>

      ------------------------------------use <t:buffer>, does not work ----

      <%@ page contentType="text/html; charset=UTF-8"%>
      <%@ taglib uri="http://java.sun.com/jsf/html" prefix="h"%>
      <%@ taglib uri="http://java.sun.com/jsf/core" prefix="f"%>
      <%@ taglib uri="http://myfaces.apache.org/tomahawk" prefix="t"%>
      <HTML>
      <head>
      </head>
      <f:view>
      <t:buffer into="#

      {buf}">
      <h:outputText value="#{testBean.name}" />
      <h:form>
      <h:inputText value="#{testBean.name}"/>
      <h:commandButton value="Submit" />
      </h:form>
      </t:buffer>
      <h:outputText value="#{buf}

      " escape="false" />
      </f:view>
      </HTML>

      1. utf8.jsp
        0.8 kB
        Mathias Werlitz

        Activity

        Hide
        Dave added a comment -

        <t:buffer> encoding should be the same as response encoding.

        Show
        Dave added a comment - <t:buffer> encoding should be the same as response encoding.
        Hide
        Martin Marinschek added a comment -

        This is the code being executed:

        static public HtmlBufferResponseWriterWrapper getInstance(ResponseWriter initialWriter)

        { ByteArrayOutputStream instanceStream = new ByteArrayOutputStream(); PrintWriter instanceWriter = new PrintWriter(instanceStream, true); return new HtmlBufferResponseWriterWrapper(initialWriter, instanceStream, instanceWriter); }

        private HtmlBufferResponseWriterWrapper(ResponseWriter initialWriter, ByteArrayOutputStream stream, PrintWriter writer)

        { super(writer, initialWriter==null?null:initialWriter.getContentType(), initialWriter==null?null:initialWriter.getCharacterEncoding()); this.stream = stream; this.writer = writer; }

        where the initialWriter is the normal facesContext responseWriter.

        I don't see why this shouldn't work - can you debug through and see if the correct encoding is passed in?

        regards,

        Martin

        Show
        Martin Marinschek added a comment - This is the code being executed: static public HtmlBufferResponseWriterWrapper getInstance(ResponseWriter initialWriter) { ByteArrayOutputStream instanceStream = new ByteArrayOutputStream(); PrintWriter instanceWriter = new PrintWriter(instanceStream, true); return new HtmlBufferResponseWriterWrapper(initialWriter, instanceStream, instanceWriter); } private HtmlBufferResponseWriterWrapper(ResponseWriter initialWriter, ByteArrayOutputStream stream, PrintWriter writer) { super(writer, initialWriter==null?null:initialWriter.getContentType(), initialWriter==null?null:initialWriter.getCharacterEncoding()); this.stream = stream; this.writer = writer; } where the initialWriter is the normal facesContext responseWriter. I don't see why this shouldn't work - can you debug through and see if the correct encoding is passed in? regards, Martin
        Hide
        Zhi added a comment -

        I came across the same problem when I updated from MyFaces-1.1.2 to Tomahawk-1.1.3-SNAPSHOT and MyFaces-1.1.4-SNAPSHOT.

        My code works fine with MyFaces-1.1.2, while only letters and digits display normal with Tomahawk-1.1.3-SNAPSHOT and MyFaces-1.1.4-SNAPSHOT. I hope it is helpful.

        Show
        Zhi added a comment - I came across the same problem when I updated from MyFaces-1.1.2 to Tomahawk-1.1.3-SNAPSHOT and MyFaces-1.1.4-SNAPSHOT. My code works fine with MyFaces-1.1.2, while only letters and digits display normal with Tomahawk-1.1.3-SNAPSHOT and MyFaces-1.1.4-SNAPSHOT. I hope it is helpful.
        Hide
        Martin Marinschek added a comment -

        Can you guys debug this and tell me a little more?

        regards,

        Martin

        Show
        Martin Marinschek added a comment - Can you guys debug this and tell me a little more? regards, Martin
        Hide
        Poul Bildsøe added a comment -

        I just experienced the same error running the JBoss Portal 2.4 beta release. This version had en error ( http://jira.jboss.com/jira/browse/JBPORTAL-930?page=all ) related to handling of utf8. I then tried the newest JBoss Portal 2.4 (release candidate 1) in which the error had been resolved and didn't get the Tomahawk buffer encoding error any more. So my bet is that the t:buffer implementation is all right and that the error should be found in the app server being used.

        Show
        Poul Bildsøe added a comment - I just experienced the same error running the JBoss Portal 2.4 beta release. This version had en error ( http://jira.jboss.com/jira/browse/JBPORTAL-930?page=all ) related to handling of utf8. I then tried the newest JBoss Portal 2.4 (release candidate 1) in which the error had been resolved and didn't get the Tomahawk buffer encoding error any more. So my bet is that the t:buffer implementation is all right and that the error should be found in the app server being used.
        Hide
        Mathias Werlitz added a comment -

        The bug does apply to any UTF-8 content rendered to the buffer ... e.g. german umlauts.
        The Writer that buffers the output seems to use the system default encoding (property "file.encoding") not the page encoding defined in the jsp page.

        I have supplied a simple test jsp page.
        On my system (WinXP ger) it does output the following: Cp1252 äöü ??

        A simple workaround for the moment is to set the default encoding to utf-8 with "-Dfile.encoding=utf-8" as a VM parameter.

        Show
        Mathias Werlitz added a comment - The bug does apply to any UTF-8 content rendered to the buffer ... e.g. german umlauts. The Writer that buffers the output seems to use the system default encoding (property "file.encoding") not the page encoding defined in the jsp page. I have supplied a simple test jsp page. On my system (WinXP ger) it does output the following: Cp1252 äöü ?? A simple workaround for the moment is to set the default encoding to utf-8 with "-Dfile.encoding=utf-8" as a VM parameter.
        Hide
        Martin Marinschek added a comment -

        Downgrading this from Blocker, as a workaround is available.

        regards,

        Martin

        Show
        Martin Marinschek added a comment - Downgrading this from Blocker, as a workaround is available. regards, Martin
        Hide
        Martin Marinschek added a comment -

        Related issue: TOMAHAWK-648

        Show
        Martin Marinschek added a comment - Related issue: TOMAHAWK-648
        Hide
        Martin Marinschek added a comment -

        Does my commit for TOMAHAWK-648 help?

        regards,

        Martin

        Show
        Martin Marinschek added a comment - Does my commit for TOMAHAWK-648 help? regards, Martin
        Hide
        Mathias Werlitz added a comment -

        The patch works for me

        Show
        Mathias Werlitz added a comment - The patch works for me

          People

          • Assignee:
            Unassigned
            Reporter:
            Dave
          • Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development