Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-12376

New TaggerRequestHandler (aka SolrTextTagger)

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.4
    • Component/s: None
    • Labels:
      None

      Description

      This issue introduces a new RequestHandler: TaggerRequestHandler, AKA the SolrTextTagger from the OpenSextant project https://github.com/OpenSextant/SolrTextTagger. It's used for named entity recognition (NER) of text past to it. It doesn't do any NLP (outside of Lucene text analysis) so it's said to be a "naive tagger", but it's definitely useful as-is and a more complete NER or ERD (entity recognition and disambiguation) system can be built with this as a key component. The SolrTextTagger has been used on queries for query-understanding, and it's been used on full-text, and it's been used on dictionaries that number tens of millions in size. Since it's small and has been used a bunch (including helping win an ERD competition and in Apache Stanbol), several people have asked me when or why isn't this in Solr yet. So here it is.

      To use it, first you need a collection of documents that have a name-like field (short text) indexed with the ConcatenateFilter (LUCENE-8323) at the end. We call this the dictionary. Once that's in place, you simply post text to a TaggerRequestHandler and it returns the offset pairs into that text for matches in the dictionary along with the uniqueKey of the matching documents. It can also return other document data desired. That's the gist; I'll add more details on use to the Solr Reference Guide.

        Attachments

        1. SOLR-12376.patch
          131 kB
          David Smiley
        2. SOLR-12376.patch
          131 kB
          David Smiley
        3. SOLR-12376.patch
          149 kB
          David Smiley

          Issue Links

            Activity

              People

              • Assignee:
                dsmiley David Smiley
                Reporter:
                dsmiley David Smiley
              • Votes:
                2 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: