[CARBONDATA-3234] Unable to read data from carbondata table stored in S3 using Presto running on EMR

• Type: Bug
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Component/s: presto-integration
• Environment: Amazon EMR 5.19
• Assignee: Unassigned
• Reporter: charles horrell
• Created: 07/Jan/19 15:08
• Updated: 11/Jan/19 14:00

Description

We are unable to use presto to query a carbondata table stored in S3.

presto:default> select count(*) from test_table;
Query 20190107_135333_00026_8r2c8 failed: tried to access method org.apache.hadoop.metrics2.lib.MutableCounterLong.<init>(Lorg/apache/hadoop/metrics2/MetricsInfo;J)V from class org.apache.hadoop.fs.s3a.S3AInstrumentation

 

presto:default> select * from test_table;
Query 20190107_135610_00028_8r2c8 failed: tried to access method org.apache.hadoop.metrics2.lib.MutableCounterLong.<init>(Lorg/apache/hadoop/metrics2/MetricsInfo;J)V from class org.apache.hadoop.fs.s3a.S3AInstrumentation

The catalog appears to have been picked up okay as show tables works as expected as does describing the table it is just when actually trying to access the data that we see the error.

We configured presto as per the examples here: 

http://carbondata.apache.org/quick-start-guide.html

Querying from Spark works okay however it is vital for our use case that presto also works and with S3.

Amazon EMR version 5.19
Spark 2.3.2
Hadoop 2.8.5
Presto 0.212

 

Stack trace from presto server log

2019-01-07T12:19:57.562Z WARN statement-response-4 com.facebook.presto.server.ThrowableMapper Request failed for /v1/statement/20190107_121957_00004_k6t5p/1
java.lang.IllegalAccessError: tried to access method org.apache.hadoop.metrics2.lib.MutableCounterLong.<init>(Lorg/apache/hadoop/metrics2/MetricsInfo;J)V from class org.apache.hadoop.fs.s3a.S3AInstrumentation
at org.apache.hadoop.fs.s3a.S3AInstrumentation.streamCounter(S3AInstrumentation.java:194)
at org.apache.hadoop.fs.s3a.S3AInstrumentation.streamCounter(S3AInstrumentation.java:216)
at org.apache.hadoop.fs.s3a.S3AInstrumentation.<init>(S3AInstrumentation.java:139)
at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:174)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.<init>(AbstractDFSCarbonFile.java:74)
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.<init>(AbstractDFSCarbonFile.java:66)
at org.apache.carbondata.core.datastore.filesystem.HDFSCarbonFile.<init>(HDFSCarbonFile.java:41)
at org.apache.carbondata.core.datastore.filesystem.S3CarbonFile.<init>(S3CarbonFile.java:41)
at org.apache.carbondata.core.datastore.impl.DefaultFileTypeProvider.getCarbonFile(DefaultFileTypeProvider.java:53)
at org.apache.carbondata.core.datastore.impl.FileFactory.getCarbonFile(FileFactory.java:102)
at org.apache.carbondata.presto.impl.CarbonTableReader.updateCarbonFile(CarbonTableReader.java:202)
at org.apache.carbondata.presto.impl.CarbonTableReader.updateSchemaList(CarbonTableReader.java:216)
at org.apache.carbondata.presto.impl.CarbonTableReader.getSchemaNames(CarbonTableReader.java:189)
at org.apache.carbondata.presto.CarbondataMetadata.listSchemaNamesInternal(CarbondataMetadata.java:86)
at org.apache.carbondata.presto.CarbondataMetadata.getTableMetadata(CarbondataMetadata.java:135)
at org.apache.carbondata.presto.CarbondataMetadata.getTableMetadataInternal(CarbondataMetadata.java:240)
at org.apache.carbondata.presto.CarbondataMetadata.getTableMetadata(CarbondataMetadata.java:232)
at com.facebook.presto.spi.connector.classloader.ClassLoaderSafeConnectorMetadata.getTableMetadata(ClassLoaderSafeConnectorMetadata.java:145)
at com.facebook.presto.metadata.MetadataManager.getTableMetadata(MetadataManager.java:388)
at com.facebook.presto.sql.analyzer.StatementAnalyzer$Visitor.visitTable(StatementAnalyzer.java:850)
at com.facebook.presto.sql.analyzer.StatementAnalyzer$Visitor.visitTable(StatementAnalyzer.java:258)
at com.facebook.presto.sql.tree.Table.accept(Table.java:53)
at com.facebook.presto.sql.tree.AstVisitor.process(AstVisitor.java:27)
at com.facebook.presto.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:270)
at com.facebook.presto.sql.analyzer.StatementAnalyzer$Visitor.analyzeFrom(StatementAnalyzer.java:1772)
at com.facebook.presto.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:954)
at com.facebook.presto.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:258)
at com.facebook.presto.sql.tree.QuerySpecification.accept(QuerySpecification.java:127)
at com.facebook.presto.sql.tree.AstVisitor.process(AstVisitor.java:27)
at com.facebook.presto.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:270)
at com.facebook.presto.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:280)
at com.facebook.presto.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:676)
at com.facebook.presto.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:258)
at com.facebook.presto.sql.tree.Query.accept(Query.java:94)
at com.facebook.presto.sql.tree.AstVisitor.process(AstVisitor.java:27)
at com.facebook.presto.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:270)
at com.facebook.presto.sql.analyzer.StatementAnalyzer.analyze(StatementAnalyzer.java:244)
at com.facebook.presto.sql.analyzer.Analyzer.analyze(Analyzer.java:72)
at com.facebook.presto.sql.analyzer.Analyzer.analyze(Analyzer.java:64)
at com.facebook.presto.execution.SqlQueryExecution.<init>(SqlQueryExecution.java:176)
at com.facebook.presto.execution.SqlQueryExecution$SqlQueryExecutionFactory.createQueryExecution(SqlQueryExecution.java:707)
at com.facebook.presto.execution.SqlQueryManager.createQueryInternal(SqlQueryManager.java:449)
at com.facebook.presto.execution.SqlQueryManager.lambda$createQuery$3(SqlQueryManager.java:382)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)