Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-7759

Failed to start thrift server when metastore is postgre sql

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Duplicate
    • None
    • None
    • SQL
    • None

    Description

      I am using today's master branch to start thrift server with setting metastore to postgre sql, and it shows error like:
      `15/05/20 20:43:57 DEBUG Schema: DROP TABLE DELETEME1432125837197 CASCADE
      15/05/20 20:43:57 ERROR Datastore: Error thrown executing DROP TABLE DELETEME1432125837197 CASCADE : Syntax error: Encountered "CASCADE" at line 1, column 34.
      java.sql.SQLSyntaxErrorException: Syntax error: Encountered "CASCADE" at line 1, column 34.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
      at org.datanucleus.store.rdbms.datasource.dbcp.DelegatingStatement.execute(DelegatingStatement.java:264)
      at org.datanucleus.store.rdbms.datasource.dbcp.DelegatingStatement.execute(DelegatingStatement.java:264)`

      But it works well with earlier master branch (on 7th, April).
      After printing their debug level log, I found current branch tries to connect with derby but didn't know why, maybe the big reconstructure in sql module cause this issue.

      The Datastore shows in current branch:
      15/05/20 20:43:57 DEBUG Datastore: ======================= Datastore =========================
      15/05/20 20:43:57 DEBUG Datastore: StoreManager : "rdbms" (org.datanucleus.store.rdbms.RDBMSStoreManager)
      15/05/20 20:43:57 DEBUG Datastore: Datastore : read-write
      15/05/20 20:43:57 DEBUG Datastore: Schema Control : AutoCreate(None), Validate(None)
      15/05/20 20:43:57 DEBUG Datastore: Query Languages : [JDOQL, JPQL, SQL, STOREDPROC]
      15/05/20 20:43:57 DEBUG Datastore: Queries : Timeout=0
      15/05/20 20:43:57 DEBUG Datastore: ===========================================================
      15/05/20 20:43:57 DEBUG Datastore: Datastore Adapter : org.datanucleus.store.rdbms.adapter.PostgreSQLAdapter
      15/05/20 20:43:57 DEBUG Datastore: Datastore : name="Apache Derby" version="10.10.1.1 - (1458268)"
      15/05/20 20:43:57 DEBUG Datastore: Datastore Driver : name="Apache Derby Embedded JDBC Driver" version="10.10.1.1 - (1458268)"
      15/05/20 20:43:57 DEBUG Datastore: Primary Connection Factory : URL[jdbc:derby:;databaseName=/tmp/spark-8b38e943-01e5-4341-9c92-7c250f2dec96/metastore;create=true]
      15/05/20 20:43:57 DEBUG Datastore: Secondary Connection Factory : URL[jdbc:derby:;databaseName=/tmp/spark-8b38e943-01e5-4341-9c92-7c250f2dec96/metastore;create=true]
      15/05/20 20:43:57 DEBUG Datastore: Datastore Identifiers : factory="datanucleus1" case=UPPERCASE catalog= schema=SPARK
      15/05/20 20:43:57 DEBUG Datastore: Supported Identifier Cases : "MixedCase" UPPERCASE "MixedCase-Sensitive"
      15/05/20 20:43:57 DEBUG Datastore: Supported Identifier Lengths (max) : Table=128 Column=128 Constraint=128 Index=128 Delimiter="
      15/05/20 20:43:57 DEBUG Datastore: Support for Identifiers in DDL : catalog=false schema=true
      15/05/20 20:43:57 DEBUG Datastore: Datastore : checkTableViewExistence, rdbmsConstraintCreateMode=DataNucleus, initialiseColumnInfo=ALL
      15/05/20 20:43:57 DEBUG Datastore: Support Statement Batching : yes (max-batch-size=50)
      15/05/20 20:43:57 DEBUG Datastore: Queries : Results direction=forward, type=forward-only, concurrency=read-only
      15/05/20 20:43:57 DEBUG Datastore: Java-Types : string-default-length=255
      15/05/20 20:43:57 DEBUG Datastore: JDBC-Types : [id=2009], BLOB, CLOB, TIME, DATE, BOOLEAN, VARCHAR, DECIMAL, NUMERIC, CHAR, BINARY, FLOAT, LONGVARBINARY, VARBINARY, JAVA_OBJECT
      15/05/20 20:43:57 DEBUG Datastore: ===========================================================

      The Datastore in earlier master branch:
      15/05/20 20:18:10 DEBUG Datastore: ======================= Datastore =========================
      15/05/20 20:18:10 DEBUG Datastore: StoreManager : "rdbms" (org.datanucleus.store.rdbms.RDBMSStoreManager)
      15/05/20 20:18:10 DEBUG Datastore: Datastore : read-write
      15/05/20 20:18:10 DEBUG Datastore: Schema Control : AutoCreate(None), Validate(None)
      15/05/20 20:18:10 DEBUG Datastore: Query Languages : [JDOQL, JPQL, SQL, STOREDPROC]
      15/05/20 20:18:10 DEBUG Datastore: Queries : Timeout=0
      15/05/20 20:18:10 DEBUG Datastore: ===========================================================
      15/05/20 20:18:10 DEBUG Datastore: Datastore Adapter : org.datanucleus.store.rdbms.adapter.PostgreSQLAdapter
      15/05/20 20:18:10 DEBUG Datastore: Datastore : name="GaussDB" version="9.2.1"
      15/05/20 20:18:10 DEBUG Datastore: Datastore Driver : name="PostgreSQL Native Driver" version="PostgreSQL 9.1 JDBC4 (build 902)"
      15/05/20 20:18:10 DEBUG Datastore: Primary Connection Factory : URL[jdbc:postgresql://9.91.88.98:20051/sparkhivemeta]
      15/05/20 20:18:10 DEBUG Datastore: Secondary Connection Factory : URL[jdbc:postgresql://9.91.88.98:20051/sparkhivemeta]
      15/05/20 20:18:10 DEBUG Datastore: Datastore Identifiers : factory="datanucleus1" case=UPPERCASE schema=PUBLIC
      15/05/20 20:18:10 DEBUG Datastore: Supported Identifier Cases : "MixedCase" UPPERCASE "MixedCase-Sensitive"
      15/05/20 20:18:10 DEBUG Datastore: Supported Identifier Lengths (max) : Table=63 Column=63 Constraint=63 Index=63 Delimiter="
      15/05/20 20:18:10 DEBUG Datastore: Support for Identifiers in DDL : catalog=false schema=true
      15/05/20 20:18:10 DEBUG Datastore: Datastore : checkTableViewExistence, rdbmsConstraintCreateMode=DataNucleus, initialiseColumnInfo=ALL
      15/05/20 20:18:10 DEBUG Datastore: Support Statement Batching : yes (max-batch-size=50)
      15/05/20 20:18:10 DEBUG Datastore: Queries : Results direction=forward, type=forward-only, concurrency=read-only
      15/05/20 20:18:10 DEBUG Datastore: Java-Types : string-default-length=255
      15/05/20 20:18:10 DEBUG Datastore: JDBC-Types : BLOB, CLOB, TIMESTAMP, TIME, STRUCT, DATE, [id=3838], ARRAY, VARCHAR, NUMERIC, CHAR, REAL, LONGVARCHAR, LONGVARBINARY, SMALLINT, INTEGER, BIGINT, BIT, DOUBLE, DISTINCT
      15/05/20 20:18:10 DEBUG Datastore: ===========================================================

      I pasted the two logs and hive-site.xml here, the launch command is just "./start-thriftserver.sh".

      Hope someone can help solve this. THanks!

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. SPARK-7758 Failed to start thrift server when metastore is postgre sql

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Activity

            People

              Unassigned Unassigned
              WangTaoTheTonic Tao Wang (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: