Solr
  1. Solr
  2. SOLR-3852

Admin UI - Cloud Tree ArrayIndexOutOfBoundsException if binary files anywhere in ZK tree

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.0-BETA
    • Fix Version/s: 4.5, 5.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Tomcat 6, external zookeeper-3.3.5

      Description

      Original bug description indicated that when using Solr with embedded ZK everything was fine, but with an external ZK you'd get an ArrayIndexOutOfBoundsException.

      Crux of the problem is some bad assumptions about any ZK node containing data – the ZookeeperInfoServlet powering the tree view of the Cloud Admin UI assumed that any data would be utf8 text.

      If you are using extenral ZK, and other systems are writing data into ZK, then you are more likely to see this problem, because those other systems might be writing binary data in to ZK nodes – if you are using ZK embedded in solr, or using solr with it's own private (external) ZK instance, then you would only see this problem if you explicitly put binary files into solr configs and upconfig them into ZK.


      One workarround for people encountering this problem when using Solr with a ZK instance shared by other tools is to make sure you use a "chroot" patch when pointing Solr at ZK, so that it won't know about any other paths in your ZK tree that might have binary data...

      https://wiki.apache.org/solr/SolrCloud#Zookeeper_chroot

      If you are having this problem because you put binay files into your own config dir (ie: images for velocity or something like that) then there is no straight forward workarround.

      Example stack trace for this bug...

      43242 [qtp965223859-14] WARN  org.eclipse.jetty.servlet.ServletHandler /solr/zookeeper
      java.lang.ArrayIndexOutOfBoundsException: 213
              at org.apache.lucene.util.UnicodeUtil.UTF8toUTF16(UnicodeUtil.java:620)
              at org.apache.lucene.util.BytesRef.utf8ToString(BytesRef.java:168)
              at org.apache.solr.servlet.ZookeeperInfoServlet$ZKPrinter.printTree(ZookeeperInfoServlet.java:303)
              at org.apache.solr.servlet.ZookeeperInfoServlet$ZKPrinter.printTree(ZookeeperInfoServlet.java:339)
              at org.apache.solr.servlet.ZookeeperInfoServlet$ZKPrinter.printTree(ZookeeperInfoServlet.java:339)
      ...
      org.apache.solr.servlet.ZookeeperInfoServlet$ZKPrinter.print(ZookeeperInfoServlet.java:228)
              at org.apache.solr.servlet.ZookeeperInfoServlet.doGet(ZookeeperInfoServlet.java:106)
      

        Activity

        Vadim Kisselmann created issue -
        Hoss Man made changes -
        Field Original Value New Value
        Attachment SOLR-3852.patch [ 12600854 ]
        Hoss Man made changes -
        Summary Admin UI - Cloud Tree with HTTP-Status 500 and an ArrayIndexOutOfBoundsException when using external ZK Admin UI - Cloud Tree ArrayIndexOutOfBoundsException if binary files anywhere in ZK tree
        Description It works with embedded ZK.
        But when we use an external ZK(3.3.5), and this ZK has another nodes like (hbase, broker, etc. and child-nodes with not specified formats) we get this Error in Admin UI in the "Cloud-Tree" View: Loading of undefined failed with HTTP-Status 500 .
        Important(!): The cluster still works. Our external ZK see the Solr Servers (live-nodes) and has the solr config files from initial import. All the nodes like collections, configs, overseer-elect are here.
        Only the Admin UI has a problem to show the "Cloud-Tree". Cloud-Graph works!

        Catalina-LogFiles are free from Error messages, i have only this stack trace from Firebug:

        <html><head><title>Apache Tomcat/6.0.28 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 500 - </h1><HR size="1" noshade="noshade"><p><b>type</b> Exception report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>The server encountered an internal error () that prevented it from fulfilling this request.</u></p><p><b>exception</b> <pre>java.lang.ArrayIndexOutOfBoundsException
        </pre></p><p><b>note</b> <u>The full stack trace of the root cause is available in the Apache Tomcat/6.0.28 logs.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6.0.28</h3></body></html>
        Original bug description indicated that when using Solr with embedded ZK everything was fine, but with an external ZK you'd get an ArrayIndexOutOfBoundsException.

        Crux of the problem is some bad assumptions about any ZK node containing data -- the ZookeeperInfoServlet powering the tree view of the Cloud Admin UI assumed that any data would be utf8 text.

        If you are using extenral ZK, and other systems are writing data into ZK, then you are more likely to see this problem, because those other systems might be writing binary data in to ZK nodes -- if you are using ZK embedded in solr, or using solr with it's own private (external) ZK instance, then you would only see this problem if you explicitly put binary files into solr configs and upconfig them into ZK.

        ----

        One workarround for people encountering this problem when using Solr with a ZK instance shared by other tools is to make sure you use a "chroot" patch when pointing Solr at ZK, so that it won't know about any other paths in your ZK tree that might have binary data...

        https://wiki.apache.org/solr/SolrCloud#Zookeeper_chroot

        If you are having this problem because you put binay files into your own config dir (ie: images for velocity or something like that) then there is no straight forward workarround.

        Example stack trace for this bug...

        {noformat}
        43242 [qtp965223859-14] WARN org.eclipse.jetty.servlet.ServletHandler /solr/zookeeper
        java.lang.ArrayIndexOutOfBoundsException: 213
                at org.apache.lucene.util.UnicodeUtil.UTF8toUTF16(UnicodeUtil.java:620)
                at org.apache.lucene.util.BytesRef.utf8ToString(BytesRef.java:168)
                at org.apache.solr.servlet.ZookeeperInfoServlet$ZKPrinter.printTree(ZookeeperInfoServlet.java:303)
                at org.apache.solr.servlet.ZookeeperInfoServlet$ZKPrinter.printTree(ZookeeperInfoServlet.java:339)
                at org.apache.solr.servlet.ZookeeperInfoServlet$ZKPrinter.printTree(ZookeeperInfoServlet.java:339)
        ...
        org.apache.solr.servlet.ZookeeperInfoServlet$ZKPrinter.print(ZookeeperInfoServlet.java:228)
                at org.apache.solr.servlet.ZookeeperInfoServlet.doGet(ZookeeperInfoServlet.java:106)
        {noformat}
        Hoss Man made changes -
        Resolution Fixed [ 1 ]
        Status Open [ 1 ] Resolved [ 5 ]
        Assignee Hoss Man [ hossman ]
        Fix Version/s 4.5 [ 12324743 ]
        Fix Version/s 5.0 [ 12321664 ]
        Adrien Grand made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Hoss Man
            Reporter:
            Vadim Kisselmann
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development