Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14152

datanucleus.autoStartMechanismMode should set to 'Ignored' to allow rolling downgrade

    XMLWordPrintableJSON

    Details

      Description

      We see the following issue when downgrading metastore:
      1. Run some query using new tables
      2. Downgrade metastore
      3. Restart metastore will complain the new table does not exist

      In particular, constaints tables does not exist in branch-1. If we run Hive 2 and create a constraint, then downgrade metastore to Hive 1, datanucleus will complain:

      javax.jdo.JDOFatalUserException: Error starting up DataNucleus : a class "org.apache.hadoop.hive.metastore.model.MConstraint" was listed as being persisted previously in this datastore, yet the class wasnt found. Perhaps it is used by a different DataNucleus-enabled application in this datastore, or you have changed your class names.
      	at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:528)
      	at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788)
      	at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
      	at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
      	at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
      	at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
      	at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
      	at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:377)
      	at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:406)
      	at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:299)
      	at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:266)
      	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
      	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
      	at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:60)
      	at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:69)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:650)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:628)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:677)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:484)
      	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:77)
      	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:83)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5905)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5900)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:6159)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:6084)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
      	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
      

      Apparently datanucleus cache some trace about the new table and retry to reinstantiate later. This breaks downgrading we shall disable this behavior.

      We need to set "datanucleus.autoStartMechanismMode" to "Ignored" to disable the check since it becomes a norm in downgrading case.

        Attachments

        1. HIVE-14152.1.patch
          1 kB
          Jianyong Dai
        2. HIVE-14152.2.patch
          9 kB
          Thejas Nair
        3. HIVE-14152.3.patch
          11 kB
          Thejas Nair
        4. HIVE-14152.addendum.patch
          0.6 kB
          Thejas Nair

          Issue Links

            Activity

              People

              • Assignee:
                thejas Thejas Nair
                Reporter:
                daijy Jianyong Dai
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: