Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-2611

Add line-breaks when parsing HTML block-level elements

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Implemented
    • Affects Version/s: 1.14
    • Fix Version/s: 1.15
    • Component/s: parser
    • Labels:
      None

      Description

      Currently, the HTML and Tika parser only add newlines following text-nodes that contain only whitespaces (e.g </span> <span>), but not based on what the tags are, so for example a </div><div> will not add a new line.

      While some applications do not differentiate between a space and a new line, many others see the semantic difference (two following words in the same sentence are "near", but in separate sentences they are not).

      I believe adding newlines after block-level HTML elements, while not a panacea, will be an improvement on the current behavior.

      NUTCH-2318 is related to this.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                yossi Yossi Tamari
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: