Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-3230

Upgrade code running concurrently on different JVMs could make clients unusuable

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.8.1
    • Labels:
      None

      Description

      If two separate Phoenix connections try to upgrade Phoenix from v4.7 to 4.8.1 then second connection fails with the following exception. This happens even if second connection is couple of seconds apart but within upgrade window. This is likely to happen in situation where pool of client machines all get upgraded to latest Phoenix version. After this exception, all clients will cease to work with undefined column exception due to restore/aborted upgrade.

      WARN query.ConnectionQueryServicesImpl: Table already modified at this timestamp, so assuming add of these columns already done: IS_NAMESPACE_MAPPED BOOLEAN
      WARN query.ConnectionQueryServicesImpl: Table already modified at this timestamp, so assuming add of these columns already done: AUTO_PARTITION_SEQ VARCHAR
      WARN query.ConnectionQueryServicesImpl: Table already modified at this timestamp, so assuming add of these columns already done: APPEND_ONLY_SCHEMA BOOLEAN
      WARN query.ConnectionQueryServicesImpl: Starting restore of SYSTEM.CATALOG using snapshot SNAPSHOT_SYSTEM.CATALOG_4.7.x_TO_4.8.0_20160831114048-0700 because upgrade failed
      16/08/31 11:41:05 WARN query.ConnectionQueryServicesImpl: Successfully restored SYSTEM.CATALOG using snapshot SNAPSHOT_SYSTEM.CATALOG_4.7.x_TO_4.8.0_20160831114048-0700
      16/08/31 11:41:09 WARN query.ConnectionQueryServicesImpl: Successfully restored and enabled SYSTEM.CATALOG using snapshot SNAPSHOT_SYSTEM.CATALOG_4.7.x_TO_4.8.0_20160831114048-0700
      Error: ERROR 504 (42703): Undefined column. columnName=IS_NAMESPACE_MAPPED (state=42703,code=504)
      org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504 (42703): Undefined column. columnName=IS_NAMESPACE_MAPPED
      	at org.apache.phoenix.schema.PTableImpl.getColumn(PTableImpl.java:693)
      	at org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:449)
      	at org.apache.phoenix.compile.UpsertCompiler.compile(UpsertCompiler.java:418)
      	at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableUpsertStatement.compilePlan(PhoenixStatement.java:590)
      	at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableUpsertStatement.compilePlan(PhoenixStatement.java:578)
      	at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:333)
      	at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:328)
      	at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
      	at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:326)
      	at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:247)
      	at org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:172)
      	at org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:177)
      	at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2275)
      	at org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:920)
      	at org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:193)
      	at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:340)
      	at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:328)
      	at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
      	at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:326)
      	at org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:1369)
      	at org.apache.phoenix.query.ConnectionQueryServicesImpl$13.call(ConnectionQueryServicesImpl.java:2486)
      	at org.apache.phoenix.query.ConnectionQueryServicesImpl$13.call(ConnectionQueryServicesImpl.java:2282)
      	at org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:78)
      	at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2282)
      	at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:231)
      	at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:144)
      	at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:202)
      	at sqlline.DatabaseConnection.connect(DatabaseConnection.java:157)
      	at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:203)
      
      

        Attachments

        1. PHOENIX-3230_v2_nowhitespacediff.patch
          15 kB
          Samarth Jain
        2. PHOENIX-3230_nowhitespacediff.patch
          14 kB
          Samarth Jain
        3. PHOENIX-3230_addendum.patch
          7 kB
          Samarth Jain

          Activity

            People

            • Assignee:
              samarthjain Samarth Jain
              Reporter:
              mujtabachohan Mujtaba Chohan
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: