Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0-incubator
    • Component/s: Text extraction
    • Labels:
      None

      Description

      [imported from SourceForge]
      http://sourceforge.net/tracker/index.php?group_id=78314&atid=552832&aid=1635950
      Originally submitted by tweakerbee on 2007-01-15 07:09.

      During extraction in certain PDF documents spaces will be lost. I have attached a file in which this problem occurs.

      Here PDFTextStripper.getText() returns:
      gaandeofincidenteleaardis
      whereas it should be
      gaande of incidentele aard is

      I have used the nightly build from today (15-01-07) but the problem still remains.

      [attachment on SourceForge]
      http://sourceforge.net/tracker/download.php?group_id=78314&atid=552832&aid=1635950&file_id=211376
      STB336.pdf (application/pdf), 51425 bytes
      document with erronous text extraction

      [comment on SourceForge]
      Originally sent by tweakerbee.
      Logged In: YES
      user_id=1625706
      Originator: YES

      The problem turned out to be in the splitting algorithm. The values here turned out slightly too conservative.
      Using 0.33f (33%) turned out to yield proper results. This might split words that are not meant to be split, however.

      Maybe you could set this through a field in the TextStripper? So you can adjust your application slightly easier to your specific needs.

      This issue can be considered solved.

      startOfNextWordX = endOfLastTextX + (wordSpacing* 0.33f);
      startOfNextWordX = endOfLastTextX + (((wordSpacing+lastWordSpacing)/2f)* 0.33f);

      [comment on SourceForge]
      Originally sent by tweakerbee.
      Logged In: YES
      user_id=1625706
      Originator: YES

      My previous assumption turned out to be incorrect.
      The context.showString() function is responsible for outputting the string. If anywhere, it should probably output the space here.

      [comment on SourceForge]
      Originally sent by tweakerbee.
      Logged In: YES
      user_id=1625706
      Originator: YES

      I am currently looking into the problem myself as well, but my complete lack of experience with the Portable Document Format as well as being a novice Java programmer are rather limiting.

      What I have found out so far is this:
      The problem is in the TextStream where a TJ operator is being used to show the glyphs. There are no spaces encoded in the file, but instead it uses some character spacing information to space out the words. An example is included below.
      The code I believe is responsible for extracting the text here (org.pdfbox.util.operator.ShowTextGlyph) does not contain any code to determine whether or not a space is needed. Would it be useful to add this here? And will this not breakdown the org.pdfbox.util.PDFHighlighter? (I have noticed some difficulties with certain PDF documents and I wouldn't be surprised if the difference in character count originates from this issue.)

      Any help would be greatly appreciated.

      Example code in STB336.pdf:
      [(7?????)-278(???)-278(? ?"&????)-278(???)-278(???)-278( ??\))-278(???)-278(??????\)????\012)-278( ?????'??&)-278(??)]TJ

        Issue Links

          Activity

          Hide
          Dmitry Gutso added a comment -

          revision 807760. Run org.apache.pdfbox.examples.util.PrintTextLocations. There are no distinctions. The same result. I dont understand, unfortunately.

          Show
          Dmitry Gutso added a comment - revision 807760. Run org.apache.pdfbox.examples.util.PrintTextLocations. There are no distinctions. The same result. I dont understand, unfortunately.
          Hide
          Andreas Lehmkühler added a comment -

          The described bugfix is part of the trunk-version (the upcoming 0.8.0 release). Please try that one first.

          Show
          Andreas Lehmkühler added a comment - The described bugfix is part of the trunk-version (the upcoming 0.8.0 release). Please try that one first.
          Hide
          Dmitry Gutso added a comment - - edited

          I posted my question here because it seems like.
          I have a problem with addition of the space character between words in the
          outer text by PDFTextStripper. I used PDFBox-0.7.3, JAVA 1.6.0

          example (the string is copied by textbuffer of Adobe from the indicated pdf-file):
          source pdf: http://www.cmegroup.com/daily_bulletin/Section02A_Summary_Volume_And_Open_Interest_(Excludes%20TRAKRS)_Comm_Alt_Invest_Futures_And_Options_2009152.pdf

          IOM DIVISION 1523788 456934 1980722 23017621 + 105674 3147531 27408756

          after parsing:

          IOM DIVISION 1523788 4569341980722 23017621 + 105674 3147531 27408756

          cause:

          String[229.68,212.15997 fs=1.0 xscale=6.0 height=3.7500005 width=14.412003]5693
          String[244.08,212.15997 fs=1.0 xscale=6.0 height=3.7500005 width=85.805984]41
          String[290.58,212.15997 fs=1.0 xscale=6.0 height=3.7500005 width=10.8089905]980

          I tried use the classes of PDFBox-0.7.4.jar but it wasn't succes. The code that was checked in for the PDFBOX-349 issue used "0.7.4", but this does not work for me... Sorry.

          Show
          Dmitry Gutso added a comment - - edited I posted my question here because it seems like. I have a problem with addition of the space character between words in the outer text by PDFTextStripper. I used PDFBox-0.7.3, JAVA 1.6.0 example (the string is copied by textbuffer of Adobe from the indicated pdf-file): source pdf: http://www.cmegroup.com/daily_bulletin/Section02A_Summary_Volume_And_Open_Interest_(Excludes%20TRAKRS)_Comm_Alt_Invest_Futures_And_Options_2009152.pdf IOM DIVISION 1523788 456934 1980722 23017621 + 105674 3147531 27408756 after parsing: IOM DIVISION 1523788 4569341980722 23017621 + 105674 3147531 27408756 cause: String [229.68,212.15997 fs=1.0 xscale=6.0 height=3.7500005 width=14.412003] 5693 String [244.08,212.15997 fs=1.0 xscale=6.0 height=3.7500005 width=85.805984] 41 String [290.58,212.15997 fs=1.0 xscale=6.0 height=3.7500005 width=10.8089905] 980 I tried use the classes of PDFBox-0.7.4.jar but it wasn't succes. The code that was checked in for the PDFBOX-349 issue used "0.7.4", but this does not work for me... Sorry.
          Hide
          Justin LeFebvre added a comment -

          This bug is fixed with the code that was checked in for the PDFBOX-349 issue.

          Show
          Justin LeFebvre added a comment - This bug is fixed with the code that was checked in for the PDFBOX-349 issue.

            People

            • Assignee:
              Unassigned
              Reporter:
              Anonymous
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development