Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-2891

Impossible to use jdbc interface with presto-jdbc >= 0.180

    Details

    • Type: Bug
    • Status: Open
    • Priority: Blocker
    • Resolution: Unresolved
    • Affects Version/s: 0.7.2
    • Fix Version/s: None
    • Component/s: Interpreters
    • Labels:
    • Environment:

      Description

      Hello,

      I have been trying since yesterday to integrate Presto (0.183) with Apache Zeppelin (0.7.2) (using the presto-jdbc driver interface), and, so far, I am hitting a roadblock.

      Since presto version 0.180 and higher, with commit defabcba4e1b89d5df536da51258cfdbfea906cb ("Improve properties/URL parameter handling"), developers did tighten connection properties with com.facebook.presto.jdbc.PrestoDriverUri.validateConnectionProperties.
      This function rejects anything that is not specified into com.facebook.presto.jdbc.ConnectionProperties.

      Sadly, Zeppelin adds extra property keys such as "url" and "driver" which, naturally, get rejected. (See org.apache.zeppelin.jdbc.JDBCInterpreter.java at function "open")

      A typical debug log would look like that:
      {{ INFO [2017-08-31 07:14:21,156] (

      {pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job remoteInterpretJob_1504163661153 started by scheduler org.apache.zeppelin.jdbc.JDBCInterpreter43219549
      DEBUG [2017-08-31 07:14:21,157] ({pool-2-thread-2}

      JDBCInterpreter.java[open]:156) - propertyKey: default.password
      DEBUG [2017-08-31 07:14:21,157] (

      {pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: default, value: password
      DEBUG [2017-08-31 07:14:21,157] ({pool-2-thread-2}

      JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.auth.type
      DEBUG [2017-08-31 07:14:21,159] (

      {pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.auth.type
      DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}

      JDBCInterpreter.java[open]:156) - propertyKey: common.max_count
      DEBUG [2017-08-31 07:14:21,160] (

      {pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: common, value: max_count
      DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}

      JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.principal
      DEBUG [2017-08-31 07:14:21,160] (

      {pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.principal
      DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}

      JDBCInterpreter.java[open]:156) - propertyKey: default.user
      DEBUG [2017-08-31 07:14:21,160] (

      {pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: default, value: user
      DEBUG [2017-08-31 07:14:21,160] ({pool-2-thread-2}

      JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.interpreter.localRepo
      DEBUG [2017-08-31 07:14:21,161] (

      {pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: interpreter.localRepo
      DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}

      JDBCInterpreter.java[open]:156) - propertyKey: default.url
      DEBUG [2017-08-31 07:14:21,161] (

      {pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: default, value: url
      DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}

      JDBCInterpreter.java[open]:156) - propertyKey: default.driver
      DEBUG [2017-08-31 07:14:21,161] (

      {pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: default, value: driver
      DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}

      JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.keytab.location
      DEBUG [2017-08-31 07:14:21,161] (

      {pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.keytab.location
      DEBUG [2017-08-31 07:14:21,161] ({pool-2-thread-2}

      JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.concurrent.use
      DEBUG [2017-08-31 07:14:21,162] (

      {pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.concurrent.use
      DEBUG [2017-08-31 07:14:21,162] ({pool-2-thread-2}

      JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.jdbc.concurrent.max_connection
      DEBUG [2017-08-31 07:14:21,162] (

      {pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: jdbc.concurrent.max_connection
      DEBUG [2017-08-31 07:14:21,162] ({pool-2-thread-2}

      JDBCInterpreter.java[open]:156) - propertyKey: zeppelin.interpreter.output.limit
      DEBUG [2017-08-31 07:14:21,162] (

      {pool-2-thread-2} JDBCInterpreter.java[open]:159) - key: zeppelin, value: interpreter.output.limit
      ERROR [2017-08-31 07:14:21,162] ({pool-2-thread-2}

      JDBCInterpreter.java[open]:177) - zeppelin will be ignored. driver.zeppelin and zeppelin.url is mandatory.
      DEBUG [2017-08-31 07:14:21,166] (

      {pool-2-thread-2} JDBCInterpreter.java[open]:187) - JDBC PropretiesMap: {default={user=presto, url=jdbc:presto://DOMAIN_NAME:4481/hive;LogLevel=6;LogPath=/log/, password=, driver=com.facebook.presto.jdbc.PrestoDriver}, common={max_count=1000}}
      DEBUG [2017-08-31 07:14:21,191] ({pool-1-thread-3} Interpreter.java[getProperty]:165) - key: zeppelin.jdbc.concurrent.use, value: true
      DEBUG [2017-08-31 07:14:21,192] ({pool-1-thread-3} Interpreter.java[getProperty]:165) - key: zeppelin.jdbc.concurrent.max_connection, value: 10
      DEBUG [2017-08-31 07:14:21,208] ({pool-2-thread-2}

      JDBCInterpreter.java[interpret]:683) - Run SQL command 'use dataset_demo;
      show tables;'
      DEBUG [2017-08-31 07:14:21,208] (

      {pool-2-thread-2} JDBCInterpreter.java[interpret]:691) - PropertyKey: default, SQL command: 'use dataset_demo;
      show tables;'
      ERROR [2017-08-31 07:14:21,690] ({pool-2-thread-2}

      JDBCInterpreter.java[executeSql]:631) - Cannot run use dataset_demo;
      show tables;
      java.sql.SQLException: Unrecognized connection property 'url'
      at com.facebook.presto.jdbc.PrestoDriverUri.validateConnectionProperties(PrestoDriverUri.java:292)
      at com.facebook.presto.jdbc.PrestoDriverUri.<init>(PrestoDriverUri.java:89)
      at com.facebook.presto.jdbc.PrestoDriverUri.<init>(PrestoDriverUri.java:79)
      at com.facebook.presto.jdbc.PrestoDriver.connect(PrestoDriver.java:86)
      at java.sql.DriverManager.getConnection(DriverManager.java:664)
      at java.sql.DriverManager.getConnection(DriverManager.java:208)
      at org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:79)
      at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205)
      at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
      at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
      at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
      at org.apache.commons.dbcp2.PoolingDriver.connect(PoolingDriver.java:129)
      at java.sql.DriverManager.getConnection(DriverManager.java:664)
      at java.sql.DriverManager.getConnection(DriverManager.java:270)
      at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnectionFromPool(JDBCInterpreter.java:354)
      at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnection(JDBCInterpreter.java:424)
      at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:565)
      at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:692)
      at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
      at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
      at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
      at org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:748)}}

      As a workaround, it is possible to locally downgrade the presto-jdbc driver in Zeppelin to version 0.179, but we do lack features included in later versions.

      Comes my question: which one shall get patched?

      I am doing a similar report on the presto google group.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              vloup vloup
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated: