Details
-
Bug
-
Status: Done
-
Major
-
Resolution: Done
-
None
-
None
Description
When running any functions that attempt to access HBase from the REPL, an IllegalAccessError exception is thrown. This can be replicated with Stellar functions like ENRICHMENT_GET and PROFILE_GET that attempt to read from HBase.
To replicate, start the Stellar REPL with HBase and Zookeeper running and accessible.
[root@node1 ~]# source /etc/default/metron [root@node1 ~]# cd $METRON_HOME [root@node1 0.7.1]# bin/stellar -z $ZOOKEEPER SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/metron/0.7.1/lib/metron-profiler-repl-0.7.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.6.5.0-292/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Stellar, Go! Functions are loading lazily in the background and will be unavailable until loaded fully. {es.clustername=metron, es.ip=node1:9200, es.date.format=yyyy.MM.dd.HH, parser.error.topic=indexing, update.hbase.table=metron_update, update.hbase.cf=t, es.client.settings={}, profiler.client.period.duration=15, profiler.client.period.duration.units=MINUTES, user.settings.hbase.table=user_settings, user.settings.hbase.cf=cf, bootstrap.servers=node1:6667, source.type.field=source:type, threat.triage.score.field=threat:triage:score, enrichment.writer.batchSize=15, enrichment.writer.batchTimeout=0, profiler.writer.batchSize=15, profiler.writer.batchTimeout=0, geo.hdfs.file=/apps/metron/geo/default/GeoLite2-City.tar.gz, asn.hdfs.file=/apps/metron/asn/default/GeoLite2-ASN.tar.gz}
[Stellar]>>> ENRICHMENT_GET("example","192.168.1.1","example","E") 2019-01-30 08:51:31 ERROR SimpleHBaseEnrichmentFunctions:251 - Unable to call exists: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator at org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:229) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:140) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:879) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:845) at org.apache.metron.enrichment.lookup.EnrichmentLookup$Handler.get(EnrichmentLookup.java:70) at org.apache.metron.enrichment.lookup.EnrichmentLookup$Handler.get(EnrichmentLookup.java:52) at org.apache.metron.enrichment.lookup.Lookup.get(Lookup.java:68) at org.apache.metron.enrichment.stellar.SimpleHBaseEnrichmentFunctions$EnrichmentGet.apply(SimpleHBaseEnrichmentFunctions.java:245) at org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:664) at org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:259) at org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:151) at org.apache.metron.stellar.common.shell.DefaultStellarShellExecutor.executeStellar(DefaultStellarShellExecutor.java:407) at org.apache.metron.stellar.common.shell.DefaultStellarShellExecutor.execute(DefaultStellarShellExecutor.java:257) at org.apache.metron.stellar.common.shell.cli.StellarShell.execute(StellarShell.java:359) at org.jboss.aesh.console.AeshProcess.run(AeshProcess.java:53) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator at org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:596) at org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:580) at org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:559) at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1185) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1152) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1126) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1331) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1155) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1139) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1096) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:931) at org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:83) at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:79) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124) ... 16 more {}
[Stellar]>>> PROFILE_GET("hello-world","192.168.1.1",PROFILE_FIXED(30, "DAYS")) [!] Unable to parse: PROFILE_GET("hello-world","192.168.1.1",PROFILE_FIXED(30, "DAYS")) due to: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator org.apache.metron.stellar.dsl.ParseException: Unable to parse: PROFILE_GET("hello-world","192.168.1.1",PROFILE_FIXED(30, "DAYS")) due to: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator at org.apache.metron.stellar.common.BaseStellarProcessor.createException(BaseStellarProcessor.java:166) at org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:154) at org.apache.metron.stellar.common.shell.DefaultStellarShellExecutor.executeStellar(DefaultStellarShellExecutor.java:407) at org.apache.metron.stellar.common.shell.DefaultStellarShellExecutor.execute(DefaultStellarShellExecutor.java:257) at org.apache.metron.stellar.common.shell.cli.StellarShell.execute(StellarShell.java:359) at org.jboss.aesh.console.AeshProcess.run(AeshProcess.java:53) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator at org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:596) at org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:580) at org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:559) at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1185) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1152) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1126) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1331) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1155) at org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.findAllLocationsOrFail(AsyncProcess.java:940) at org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.groupAndSendMultiAction(AsyncProcess.java:857) at org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.access$100(AsyncProcess.java:575) at org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:557) at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:923) at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:940) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:901) at org.apache.metron.profiler.client.HBaseProfilerClient.doFetch(HBaseProfilerClient.java:138) at org.apache.metron.profiler.client.HBaseProfilerClient.fetch(HBaseProfilerClient.java:120) at org.apache.metron.profiler.client.stellar.GetProfile.apply(GetProfile.java:182) at org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:664) at org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:259) at org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:151) ... 7 more
Attachments
Issue Links
- relates to
-
METRON-2043 Fix profiler-client dependencies
-
- In Progress
-
- links to
