Uploaded image for project: 'Apache Any23 (Retired)'
  1. Apache Any23 (Retired)
  2. ANY23-76

Improve runtime of the Microformat extractor on documents with many relations.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Trivial
    • Resolution: Fixed
    • 0.7.0
    • 0.7.0
    • core
    • None

    Description

      For some large documents with many Microformat tuples the extensive use of XPath in the DomUtils class cause Microformat extraction to be slow. I've market this as trivial as it's a corner case.

      To reproduce the problem the patch addresses, run the Microformat extractor on the folloing url:
      http://en.wikipedia.org/wiki/List_of_Nike_missile_locations

      I include a patch that improves performance at the cost of code simplicity. I hope someone who is more involved in the project can decide if it's a good idea to use the patch or not, or maybe address this issue in another way.. The patch replaces commonly used XPath queries with DOM tree traversals. Eg. getting all nodes with 'class' attributes. On my machine the time to parse the given document is reduced from around 105 seconds to 14 seconds.

      Attachments

        1. MicroformatSpeed.patch
          6 kB
          Timothy Potter

        Activity

          People

            michele.mostarda Michele Mostarda
            timp74 Timothy Potter
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: