Uploaded image for project: 'Tika'
  1. Tika
  2. TIKA-3314

Treat soft hyphens like hyphens

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.25
    • Fix Version/s: 2.0.0, 1.26
    • Component/s: tika-eval
    • Labels:
      None

      Description

      The next PDFBox version identifies soft-hyphens (00AD) and returns them as such. Tika-eval swallows them, thus reporting differences. This can be shown with the file attached to PDFBOX-5115 in "Max-Planck-Institut" and in the attached excel file in line 4.

      Proposed change:
      add

      "\u00AD" => "-"

      to
      lucene-char-mapping.txt

        Attachments

        1. content_diffs_no_exceptions1.xlsx
          44 kB
          Tilman Hausherr

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                tilman Tilman Hausherr
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: