Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-856

Allow phoenix driver reuse after being closed

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.0.0, 4.0.0
    • 3.0.0, 4.0.0
    • None
    • None

    Description

      Attached patch will allow to close the phoenix driver and re-use it later on.

      Motivation

      I ran into this because I have a testcase where I start an HBaseTestingUtility, do some tests, stop it, and start it again to run the same tests with some different configs. This happens in one testcase because I want to compare the results of the two runs.

      Since the 'cluster' is stopped between the two runs, I need to stop the Phoenix driver as well using PhoenixDriver.INSTANCE.close() (If I don't do this, it works as well but takes tons of time, I think because HBase needs to figure out it needs to create a new ZK connection). Previously this worked fine, but was recently broke likely because "services.getExecutor().shutdownNow()" was added.

      Proposed changes

      The proposed patch sets services to null after the services are closed, so that they will be reopened the next time the driver is used.

      Now, this change might lead to unexpected behavior: you call close(), and if after that some other thread still does something with Phoenix, the effect of the close call would be cancelled.

      Besides this, if you currently try to use the driver after it is closed, it gives a rather confusing message:

      java.sql.SQLException: ERROR 2006 (INT08): Incompatible jars detected between client and server. Ensure that phoenix.jar is put on the classpath of HBase in every region server: Task org.apache.phoenix.job.JobManager$JobFutureTask@2c46358e rejected from org.apache.phoenix.job.JobManager$1@35661457[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 43]
      	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
      	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
      	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
      	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)
      	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processExecs(HConnectionManager.java:1462)
      	at org.apache.phoenix.query.ConnectionQueryServicesImpl.checkClientServerCompatibility(ConnectionQueryServicesImpl.java:828)
      	at org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:735)
      	at org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:983)
      	at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:1133)
      

      Therefore:

      • I introduced a closed flag so that we can check if the driver is closed and throw an exception if it is used after closing
      • I introduced a softClose() method for the case where you want to close current resources but want to allow continued use of the driver.

      Attachments

        1. check-driver-closed.patch.txt
          3 kB
          Bruno Dumon
        2. allow-driver-restart.patch.txt
          3 kB
          Bruno Dumon

        Activity

          People

            Unassigned Unassigned
            bruno Bruno Dumon
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: