Fop
  1. Fop
  2. FOP-1349

[PATCH] ZWSP works as backspace when font is embedded

    Details

    • Type: Bug Bug
    • Status: Closed
    • Resolution: Fixed
    • Affects Version/s: trunk
    • Fix Version/s: None
    • Component/s: renderer/pdf
    • Labels:
      None
    • Environment:
      Operating System: All
      Platform: All
    • External issue ID:
      42109

      Description

      When fonts are embedded in the PDF, zero-width spaces have a backspace effect:
      the text following the ZWSP is left-shifted by about one character's width, so
      it partly overprints the preceding text. At the end of a text area containing
      ZWSPs there's extra whitespace, up to the point where the text would have ended
      had it been rendered correctly.

      Observed with Times, Verdana and MSMincho, but not with Courier, and only if the
      font is embedded in the PDF file. It makes no difference whether the metrics
      file has an entry for ZWSP (8203d) or not.

      Attached are a FO and a PDF file demonstrating the effect. I also attach a patch
      that intercepts non-stretchable ZWSPs before they are written to the PDF stream.
      This doesn't address the cause of the problem so it's probably not the best
      solution, but at least it cures the disease. And there's no point in writing
      zero-dimension areas to the PDF anyway.

      1. PDFRenderer.java
        63 kB
        Paul Vinkenoog
      2. test-zwsp.fo
        1 kB
        Paul Vinkenoog
      3. test-zwsp.pdf
        29 kB
        Paul Vinkenoog
      4. zwsp.patch
        1 kB
        Paul Vinkenoog
      5. zwsp-layout.patch
        5 kB
        Paul Vinkenoog

        Activity

        Hide
        Paul Vinkenoog added a comment -

        Attachment test-zwsp.fo has been added with description: Example FO

        Show
        Paul Vinkenoog added a comment - Attachment test-zwsp.fo has been added with description: Example FO
        Hide
        Paul Vinkenoog added a comment -

        Attachment test-zwsp.pdf has been added with description: PDF built from example FO

        Show
        Paul Vinkenoog added a comment - Attachment test-zwsp.pdf has been added with description: PDF built from example FO
        Hide
        Paul Vinkenoog added a comment -

        Attachment PDFRenderer.java has been added with description: Patch for org\apache\fop\render\pdf\PDFRenderer.java

        Show
        Paul Vinkenoog added a comment - Attachment PDFRenderer.java has been added with description: Patch for org\apache\fop\render\pdf\PDFRenderer.java
        Hide
        Paul Vinkenoog added a comment -

        Sorry, seem to have added the entire Java file instead of just the patch! Hope
        I get it right this time.

        Show
        Paul Vinkenoog added a comment - Sorry, seem to have added the entire Java file instead of just the patch! Hope I get it right this time.
        Hide
        Paul Vinkenoog added a comment -

        Attachment zwsp.patch has been added with description: Patch for org\apache\fop\render\pdf\PDFRenderer.java

        Show
        Paul Vinkenoog added a comment - Attachment zwsp.patch has been added with description: Patch for org\apache\fop\render\pdf\PDFRenderer.java
        Hide
        Chris Bowditch added a comment -

        Thanks for spotting this and submitting a patch. Good work.

        Bug Confirmed. I tested your FO with Arial in PDF and the same affect occured.

        I was a little concerned that your fix was PDF specific, so I tried to
        recreate the issue in Postscript but the bug doesn't seem to affect the
        Postscript Renderer.

        Show
        Chris Bowditch added a comment - Thanks for spotting this and submitting a patch. Good work. Bug Confirmed. I tested your FO with Arial in PDF and the same affect occured. I was a little concerned that your fix was PDF specific, so I tried to recreate the issue in Postscript but the bug doesn't seem to affect the Postscript Renderer.
        Hide
        Paul Vinkenoog added a comment -

        I dug a little deeper (though not to the bottom) and found out that the problem
        arises when PDFRenderer.escapeText() generates the string to be inserted in the
        content stream. Zero-width spaces get converted to normal spaces here, and an
        adjustment is added which is equal to charwidth(space) - charwidth(zwsp). With
        Times for instance, this is inserted: "( ) 250 " (without the quotes). With
        Helvetica it's "( ) 278 ". In either case, the appearance of the PDF is correct.

        The character widths are queried from the parent area's Font object. If they are
        correct (as is the case with non-embedded Base-14 fonts), there's no visible
        (back)space in the PDF. (However, if you copy-and-paste the text, it contains
        spaces where the ZWSPs were.)

        But if the font is embedded in the PDF, the width of the regular space character
        seems to be reported wrongly by the font and - for Times - this is inserted:
        "<0003> 777 " (<0003> being the correct code for space). The adjustment is way
        too large, hence the backspace effect.

        Anyway, even if rendered correctly, ZWSPs are useless once they've done their
        job as break-opportunity indicators. They add dead weight to the PDF and mess up
        copy-and-paste. As I assume that other renderers have no need for them either, I
        think it would be best if they weren't added to the area tree at all. So I'll
        upload another patch that can be used instead of the previous one (or alongside
        it; they're not in each other's way).

        Side note: while working on this, I also discovered that if ZWSPs are inserted
        through <fo:character>s, they are not used for line breaking.

        Show
        Paul Vinkenoog added a comment - I dug a little deeper (though not to the bottom) and found out that the problem arises when PDFRenderer.escapeText() generates the string to be inserted in the content stream. Zero-width spaces get converted to normal spaces here, and an adjustment is added which is equal to charwidth(space) - charwidth(zwsp). With Times for instance, this is inserted: "( ) 250 " (without the quotes). With Helvetica it's "( ) 278 ". In either case, the appearance of the PDF is correct. The character widths are queried from the parent area's Font object. If they are correct (as is the case with non-embedded Base-14 fonts), there's no visible (back)space in the PDF. (However, if you copy-and-paste the text, it contains spaces where the ZWSPs were.) But if the font is embedded in the PDF, the width of the regular space character seems to be reported wrongly by the font and - for Times - this is inserted: "<0003> 777 " (<0003> being the correct code for space). The adjustment is way too large, hence the backspace effect. Anyway, even if rendered correctly, ZWSPs are useless once they've done their job as break-opportunity indicators. They add dead weight to the PDF and mess up copy-and-paste. As I assume that other renderers have no need for them either, I think it would be best if they weren't added to the area tree at all. So I'll upload another patch that can be used instead of the previous one (or alongside it; they're not in each other's way). Side note: while working on this, I also discovered that if ZWSPs are inserted through <fo:character>s, they are not used for line breaking.
        Hide
        Paul Vinkenoog added a comment -

        Attachment zwsp-layout.patch has been added with description: Patch to keep zero-width spaces out of the area tree

        Show
        Paul Vinkenoog added a comment - Attachment zwsp-layout.patch has been added with description: Patch to keep zero-width spaces out of the area tree
        Hide
        Jeremias Maerki added a comment -

        Thanks a lot for that patch, Paul. I've applied it:
        http://svn.apache.org/viewvc?view=rev&rev=540049

        Paul, please make it a habit of running the test suite (done automatically when
        building from the command-line using Ant) when you want to submit something. One
        of the test cases needed to be adjusted to account for the missing ZWSPs in the
        area tree. Thanks.

        BTW, the only case where the ZWSP probably shouldn't be removed is when at some
        point we will support tagged PDF. But at the moment, this is probably the most
        elegant solution.

        Show
        Jeremias Maerki added a comment - Thanks a lot for that patch, Paul. I've applied it: http://svn.apache.org/viewvc?view=rev&rev=540049 Paul, please make it a habit of running the test suite (done automatically when building from the command-line using Ant) when you want to submit something. One of the test cases needed to be adjusted to account for the missing ZWSPs in the area tree. Thanks. BTW, the only case where the ZWSP probably shouldn't be removed is when at some point we will support tagged PDF. But at the moment, this is probably the most elegant solution.
        Hide
        Paul Vinkenoog added a comment -

        > Paul, please make it a habit of running the test suite (done automatically
        > when building from the command-line using Ant) when you want to submit
        > something.

        OK. I set it up a while ago but ran into a lot of errors and didn't have time to
        fully investigate what went wrong. So I went back to "build package". But I'll
        try again next time, and be a little more perseverant.

        Show
        Paul Vinkenoog added a comment - > Paul, please make it a habit of running the test suite (done automatically > when building from the command-line using Ant) when you want to submit > something. OK. I set it up a while ago but ran into a lot of errors and didn't have time to fully investigate what went wrong. So I went back to "build package". But I'll try again next time, and be a little more perseverant.
        Hide
        Glenn Adams added a comment -

        batch transition pre-FOP1.0 resolved+fixed bugs to closed+fixed

        Show
        Glenn Adams added a comment - batch transition pre-FOP1.0 resolved+fixed bugs to closed+fixed

          People

          • Assignee:
            fop-dev
            Reporter:
            Paul Vinkenoog
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development