[ZEPPELIN-2891] Impossible to use jdbc interface with presto-jdbc >= 0.180

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.7.2
• Fix Version/s: 0.9.0
• Component/s: Interpreters
• Labels: regression

Environment

Official apache/zeppelin:0.7.2 docker image environment.
Deployment:
{{docker run \
--name zeppelin072 \
-v /etc/zeppelin/conf/:/zeppelin/conf \
-v /etc/zeppelin/notebook/:/notebook \
-v /etc/zeppelin/log/:/log \
--network=host \
-d \
apache/zeppelin:0.7.2}}

PrestoDB 0.183

Docker 17.03.1-ce running on Ubuntu 16.04.2 LTS (xenial) on amd64

• External issue URL: https://groups.google.com/forum/#!topic/presto-users/koT1Yv3sKG4
• Assignee: Jeff Zhang
• Reporter: vloup
• Created: 31/Aug/17 09:42
• Updated: 09/Jan/22 14:55
• Resolved: 21/Mar/20 05:34

Description

Hello,

I have been trying since yesterday to integrate Presto (0.183) with Apache Zeppelin (0.7.2) (using the presto-jdbc driver interface), and, so far, I am hitting a roadblock.

Since presto version 0.180 and higher, with commit defabcba4e1b89d5df536da51258cfdbfea906cb ("Improve properties/URL parameter handling"), developers did tighten connection properties with com.facebook.presto.jdbc.PrestoDriverUri.validateConnectionProperties.
This function rejects anything that is not specified into com.facebook.presto.jdbc.ConnectionProperties.

Sadly, Zeppelin adds extra property keys such as "url" and "driver" which, naturally, get rejected. (See org.apache.zeppelin.jdbc.JDBCInterpreter.java at function "open")

A typical debug log would look like that:
{{ INFO [2017-08-31 07:14:21,156] (

{pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job remoteInterpretJob_1504163661153 started by scheduler org.apache.zeppelin.jdbc.JDBCInterpreter43219549
DEBUG [2017-08-31 07:14:21,157] ({pool-2-thread-2}

JDBCInterpreter.java[open]:156) - propertyKey: default.password
DEBUG [2017-08-31 07:14:21,157] (

{pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: default, value: password
DEBUG [2017-08-31 07:14:21,157] ({pool-2-thread-2}

JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.auth.type
DEBUG [2017-08-31 07:14:21,159] (

{pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.auth.type
DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}

JDBCInterpreter.java[open]:156) - propertyKey: common.max_count
DEBUG [2017-08-31 07:14:21,160] (

{pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: common, value: max_count
DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}

JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.principal
DEBUG [2017-08-31 07:14:21,160] (

{pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.principal
DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}

JDBCInterpreter.java[open]:156) - propertyKey: default.user
DEBUG [2017-08-31 07:14:21,160] (

{pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: default, value: user
DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}

JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.interpreter.localRepo
DEBUG [2017-08-31 07:14:21,161] (

{pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: interpreter.localRepo
DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}

JDBCInterpreter.java[open]:156) - propertyKey: default.url
DEBUG [2017-08-31 07:14:21,161] (

{pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: default, value: url
DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}

JDBCInterpreter.java[open]:156) - propertyKey: default.driver
DEBUG [2017-08-31 07:14:21,161] (

{pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: default, value: driver
DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}

JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.keytab.location
DEBUG [2017-08-31 07:14:21,161] (

{pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.keytab.location
DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}

JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.concurrent.use
DEBUG [2017-08-31 07:14:21,162] (

{pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.concurrent.use
DEBUG [2017-08-31 07:14:21,162] ({pool-2-thread-2}

JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.concurrent.max_connection
DEBUG [2017-08-31 07:14:21,162] (

{pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.concurrent.max_connection
DEBUG [2017-08-31 07:14:21,162] ({pool-2-thread-2}

JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.interpreter.output.limit
DEBUG [2017-08-31 07:14:21,162] (

{pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: interpreter.output.limit
ERROR [2017-08-31 07:14:21,162] ({pool-2-thread-2}

JDBCInterpreter.java[open]:177) - zeppelin will be ignored. driver.zeppelin and zeppelin.url is mandatory.
DEBUG [2017-08-31 07:14:21,166] (

{pool-2-thread-2} JDBCInterpreter.java[open]:187) - JDBC PropretiesMap: {default={user=presto, url=jdbc:presto://DOMAIN_NAME:4481/hive;LogLevel=6;LogPath=/log/, password=, driver=com.facebook.presto.jdbc.PrestoDriver}, common={max_count=1000}}
DEBUG [2017-08-31 07:14:21,191] ({pool-1-thread-3} Interpreter.java[getProperty]:165) - key: zeppelin.jdbc.concurrent.use, value: true
DEBUG [2017-08-31 07:14:21,192] ({pool-1-thread-3} Interpreter.java[getProperty]:165) - key: zeppelin.jdbc.concurrent.max_connection, value: 10
DEBUG [2017-08-31 07:14:21,208] ({pool-2-thread-2}

JDBCInterpreter.java[interpret]:683) - Run SQL command 'use dataset_demo;
show tables;'
DEBUG [2017-08-31 07:14:21,208] (

{pool-2-thread-2} JDBCInterpreter.java[interpret]:691) - PropertyKey: default, SQL command: 'use dataset_demo;
show tables;'
ERROR [2017-08-31 07:14:21,690] ({pool-2-thread-2}

JDBCInterpreter.java[executeSql]:631) - Cannot run use dataset_demo;
show tables;
java.sql.SQLException: Unrecognized connection property 'url'
at com.facebook.presto.jdbc.PrestoDriverUri.validateConnectionProperties(PrestoDriverUri.java:292)
at com.facebook.presto.jdbc.PrestoDriverUri.<init>(PrestoDriverUri.java:89)
at com.facebook.presto.jdbc.PrestoDriverUri.<init>(PrestoDriverUri.java:79)
at com.facebook.presto.jdbc.PrestoDriver.connect(PrestoDriver.java:86)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:79)
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at org.apache.commons.dbcp2.PoolingDriver.connect(PoolingDriver.java:129)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnectionFromPool(JDBCInterpreter.java:354)
at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnection(JDBCInterpreter.java:424)
at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:565)
at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:692)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)}}

As a workaround, it is possible to locally downgrade the presto-jdbc driver in Zeppelin to version 0.179, but we do lack features included in later versions.

Comes my question: which one shall get patched?

I am doing a similar report on the presto google group.

Issue Links

• links to: GitHub Pull Request #3694