Uploaded image for project: 'Metron'
  1. Metron
  2. METRON-2088 Support HDP 3.1.0
  3. METRON-2275

Solr Indexing Topology Fails to Start on Secure Cluster with HDP 3.1

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Done
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      Many thanks to Anand Subramanian for finding this bug. 

      The Solr indexing topology will not start on a secured/kerberized cluster running HDP 3.1.

      2019-10-03 10:25:30,948 - Execute['/usr/hcp/1.9.2.0-94/metron/bin/start_solr_topology.sh'] {'logoutput': True, 'tries': 3, 'user': 'metron', 'try_sleep': 5}
      SLF4J: Class path contains multiple SLF4J bindings.
      SLF4J: Found binding in [jar:file:/grid/0/hdp/3.1.4.0-315/storm/lib/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: Found binding in [jar:file:/grid/0/hdp/3.1.4.0-315/storm/contrib/storm-autocreds/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
      SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
      SLF4J: Class path contains multiple SLF4J bindings.
      SLF4J: Found binding in [jar:file:/grid/0/hdp/3.1.4.0-315/storm/lib/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: Found binding in [jar:file:/grid/0/hdp/3.1.4.0-315/storm/contrib/storm-autocreds/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
      SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
      SLF4J: Class path contains multiple SLF4J bindings.
      SLF4J: Found binding in [jar:file:/grid/0/hdp/3.1.4.0-315/storm/lib/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: Found binding in [jar:file:/grid/0/hdp/3.1.4.0-315/storm/contrib/storm-autocreds/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
      SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
      Running: /usr/jdk64/jdk1.8.0_112/bin/java -server -Ddaemon.name= -Dstorm.options= -Dstorm.home=/grid/0/hdp/3.1.4.0-315/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /grid/0/hdp/3.1.4.0-315/storm/*:/grid/0/hdp/3.1.4.0-315/storm/lib/*:/grid/0/hdp/3.1.4.0-315/storm/extlib/*:/usr/hdp/current/storm-supervisor/external/storm-autocreds/* org.apache.storm.daemon.ClientJarTransformerRunner org.apache.storm.hack.StormShadeTransformer /usr/hcp/1.9.2.0-94/metron/lib/metron-solr-storm-0.7.1.1.9.2.0-94-uber.jar /tmp/222f7eaae5c811e9937dfa163ea52b42.jar
      SLF4J: Class path contains multiple SLF4J bindings.
      SLF4J: Found binding in [jar:file:/grid/0/hdp/3.1.4.0-315/storm/lib/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: Found binding in [jar:file:/grid/0/hdp/3.1.4.0-315/storm/contrib/storm-autocreds/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
      SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
      SLF4J: Class path contains multiple SLF4J bindings.
      SLF4J: Found binding in [jar:file:/grid/0/hdp/3.1.4.0-315/storm/lib/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: Found binding in [jar:file:çlog4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
      SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
      SLF4J: Class path contains multiple SLF4J bindings.
      SLF4J: Found binding in [jar:file:/grid/0/hdp/3.1.4.0-315/storm/lib/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: Found binding in [jar:file:/grid/0/hdp/3.1.4.0-315/storm/contrib/storm-autocreds/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
      SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
      Running: /usr/jdk64/jdk1.8.0_112/bin/java -Ddaemon.name= -Dstorm.options= -Dstorm.home=/grid/0/hdp/3.1.4.0-315/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /grid/0/hdp/3.1.4.0-315/storm/*:/grid/0/hdp/3.1.4.0-315/storm/lib/*:/grid/0/hdp/3.1.4.0-315/storm/extlib/*:/usr/hdp/current/storm-supervisor/external/storm-autocreds/*:/tmp/222f7eaae5c811e9937dfa163ea52b42.jar:/home/metron/.storm:/grid/0/hdp/3.1.4.0-315/storm/bin:/usr/hcp/1.9.2.0-94/metron/lib/stellar-common-0.7.1.1.9.2.0-94-uber.jar -Dstorm.jar=/tmp/222f7eaae5c811e9937dfa163ea52b42.jar -Dstorm.dependency.jars=/usr/hcp/1.9.2.0-94/metron/lib/stellar-common-0.7.1.1.9.2.0-94-uber.jar -Dstorm.dependency.artifacts={} org.apache.storm.flux.Flux --remote /usr/hcp/1.9.2.0-94/metron/flux/indexing/random_access/remote.yaml --filter /usr/hcp/1.9.2.0-94/metron/config/solr.properties
      SLF4J: Class path contains multiple SLF4J bindings.
      SLF4J: Found binding in [jar:file:/grid/0/hdp/3.1.4.0-315/storm/lib/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: Found binding in [jar:file:/grid/0/hdp/3.1.4.0-315/storm/contrib/storm-autocreds/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
      SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
      ???????????     ???   ??????  ???
      ???????????     ???   ???????????
      ??????  ???     ???   ??? ??????
      ??????  ???     ???   ??? ??????
      ???     ????????????????????? ???
      ???     ???????? ??????? ???  ???
      +-         Apache Storm        -+
      +-  data FLow User eXperience  -+
      Version: 1.0.1
      Parsing file: /usr/hcp/1.9.2.0-94/metron/flux/indexing/random_access/remote.yaml
      797  [main] INFO  o.a.s.f.p.FluxParser - loading YAML from input stream...
      804  [main] INFO  o.a.s.f.p.FluxParser - Performing property substitution.
      808  [main] INFO  o.a.s.f.p.FluxParser - Not performing environment variable substitution.
      1118 [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
      Exception in thread "main" java.lang.reflect.InvocationTargetException
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      	at org.apache.storm.flux.FluxBuilder.buildObject(FluxBuilder.java:294)
      	at org.apache.storm.flux.FluxBuilder.buildComponents(FluxBuilder.java:342)
      	at org.apache.storm.flux.FluxBuilder.buildTopology(FluxBuilder.java:75)
      	at org.apache.storm.flux.Flux.runCli(Flux.java:153)
      	at org.apache.storm.flux.Flux.main(Flux.java:98)
      Caused by: java.lang.NoSuchMethodError: com.google.common.util.concurrent.MoreExecutors.sameThreadExecutor()Lcom/google/common/util/concurrent/ListeningExecutorService;
      	at org.apache.curator.framework.listen.ListenerContainer.addListener(ListenerContainer.java:40)
      	at org.apache.curator.framework.imps.CuratorFrameworkImpl.start(CuratorFrameworkImpl.java:246)
      	at org.apache.metron.common.utils.KafkaUtils.getBrokersFromZookeeper(KafkaUtils.java:49)
      	at org.apache.metron.storm.kafka.flux.SimpleStormKafkaBuilder.getBootstrapServers(SimpleStormKafkaBuilder.java:246)
      	at org.apache.metron.storm.kafka.flux.SimpleStormKafkaBuilder.<init>(SimpleStormKafkaBuilder.java:203)
      	at org.apache.metron.storm.kafka.flux.SimpleStormKafkaBuilder.<init>(SimpleStormKafkaBuilder.java:187)
      	... 9 more
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                nickwallen Nick Allen
                Reporter:
                nickwallen Nick Allen
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m