Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-5426

org.apache.solr.common.SolrException; org.apache.solr.common.SolrException: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 4.4, 4.5.1
    • Fix Version/s: 4.9, 6.0
    • Component/s: highlighter
    • Labels:
      None

      Description

      Highlighter does not work correctly on test-data.
      I added index- and config- files (see attached highlighter.zip) for reproducing this issue.
      Everything works fine if I search without highlighting:

      http://localhost:8983/solr/global/select?q=aa&wt=json&indent=true

      But if search with highlighting:

      http://localhost:8983/solr/global/select?q=aa&wt=json&indent=true&hl=true&hl.fl=*_stx&hl.simple.pre=<em>&hl.simple.post=<%2Fem>

      I'm get the error:

      ERROR - 2013-11-07 10:17:15.797; org.apache.solr.common.SolrException; null:org.apache.solr.common.SolrException: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840
      at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:542)
      at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:414)
      at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:139)
      at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
      at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
      at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
      at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
      at org.eclipse.jetty.server.Server.handle(Server.java:368)
      at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
      at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
      at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
      at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
      at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
      at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
      at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
      at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
      at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
      at java.lang.Thread.run(Unknown Source)
      Caused by: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840
      at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:225)
      at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:527)
      ... 33 more

        Attachments

        1. highlighter.zip
          126 kB
          Nikolay
        2. SOLR-5426.patch
          33 kB
          Uwe Schindler
        3. SOLR-5426.patch
          32 kB
          Hoss Man
        4. SOLR-5426.patch
          33 kB
          Arun Kumar
        5. SOLR-5426.patch
          32 kB
          Ahmet Arslan
        6. SOLR-5426.patch
          48 kB
          Ahmet Arslan
        7. SOLR-5426.patch
          98 kB
          Arun Kumar

          Issue Links

            Activity

              People

              • Assignee:
                hossman Hoss Man
                Reporter:
                nmg Nikolay
              • Votes:
                2 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: