[DRILL-6337] Kafka plugin, jdbc drill: NoClassDefFoundError: org/apache/hadoop/io/Text

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.12.0
• Component/s: Client - JDBC

Environment

I am using scala version 2.11.8 and running in IntelliJ IDEA.

I have the following dependencies:

"org.apache.drill.exec" % "drill-jdbc-all" % "1.1.0",
"org.slf4j" % "slf4j-log4j12" % "1.7.21",
"org.scalatest" % "scalatest_2.11" % "2.2.6"

• Assignee: Unassigned
• Reporter: Don Tavoletti
• Created: 17/Apr/18 18:07
• Updated: 17/Apr/18 18:07

Description

I am attempting to get information from a Kafka topic.   I am able to get all information except for when I try to get information from a nested field.

I can do a select on any field except for the following field (ingestionStats) which is a nested field.

The following command works perfectly from: "Apache Drill Web Console", however from my scala app, it crashes.

select ingestionStats from kafka.`INGEST_STATUS`

When I call:

// Query kafka data
val resultSet: ResultSet = ourStatement.executeQuery(sqlCommand)

I get the following crash:

 

13:56:48.554 [main] DEBUG o.a.d.j.impl.DrillStatementRegistry - Auto-closing (via open-statements registry): org.apache.drill.jdbc.impl.DrillStatementImpl@1252d480
13:56:48.554 [main] DEBUG o.a.d.j.i.DrillResultSetImpl$ResultsListener - 2 Query listener closing.
13:56:48.554 [main] DEBUG o.a.d.j.impl.DrillStatementRegistry - Removing from open-statements registry: org.apache.drill.jdbc.impl.DrillStatementImpl@1252d480
13:56:48.554 [main] DEBUG o.apache.drill.exec.rpc.BasicClient - Closing client
13:56:48.554 [Client-1] INFO o.a.drill.exec.rpc.user.UserClient - Channel closed /10.25.130.75:52762 <--> cdtskfka32u.rxcorp.com/162.44.24.145:31010.
13:56:48.570 [main] DEBUG o.a.c.f.imps.CuratorFrameworkImpl - Closing
13:56:48.570 [main] DEBUG o.a.curator.CuratorZookeeperClient - Closing
13:56:48.570 [main] DEBUG org.apache.curator.ConnectionState - Closing
13:56:48.570 [main] DEBUG org.apache.zookeeper.ZooKeeper - Closing session: 0x26287e724621a64
13:56:48.570 [main] DEBUG org.apache.zookeeper.ClientCnxn - Closing client for session: 0x26287e724621a64
13:56:48.585 [main-SendThread(10.121.48.202:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x26287e724621a64, packet:: clientPath:null serverPath:null finished:false header:: 13,-11 replyHeader:: 13,365073887558,0 request:: null response:: null
13:56:48.585 [main-SendThread(10.121.48.202:2181)] DEBUG org.apache.zookeeper.ClientCnxn - An exception was thrown while closing send thread for session 0x26287e724621a64 : Unable to read additional data from server sessionid 0x26287e724621a64, likely server has closed socket
13:56:48.585 [main] DEBUG org.apache.zookeeper.ClientCnxn - Disconnecting client for session: 0x26287e724621a64
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/io/Text
at org.apache.drill.exec.util.JsonStringHashMap.<clinit>(JsonStringHashMap.java:39)
13:56:48.601 [main] INFO org.apache.zookeeper.ZooKeeper - Session: 0x26287e724621a64 closed
at org.apache.drill.exec.vector.complex.MapVector$Accessor.getObject(MapVector.java:295)
13:56:48.601 [main-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down
at org.apache.drill.exec.vector.accessor.GenericAccessor.getObject(GenericAccessor.java:38)
at org.apache.drill.exec.vector.accessor.AbstractSqlAccessor.getString(AbstractSqlAccessor.java:101)
at org.apache.drill.exec.vector.accessor.BoundCheckingAccessor.getString(BoundCheckingAccessor.java:119)
at org.apache.drill.jdbc.impl.TypeConvertingSqlAccessor.getString(TypeConvertingSqlAccessor.java:643)
at org.apache.drill.jdbc.impl.AvaticaDrillSqlAccessor.getString(AvaticaDrillSqlAccessor.java:88)
at net.hydromatic.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:205)
at org.apache.drill.jdbc.impl.DrillResultSetImpl.getString(DrillResultSetImpl.java:182)
at DrillDemo.DrillJdbcTest$.method(Main.scala:60)
at DrillDemo.DrillJdbcTest$.main(Main.scala:12)
at DrillDemo.DrillJdbcTest.main(Main.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.io.Text
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 12 more