Uploaded image for project: 'Sentry'
  1. Sentry
  2. SENTRY-872 Uber jira for HMS HA + Sentry HA redesign
  3. SENTRY-1629

Current MAuthzPathsMapping table definition may cause error 'Duplicate entry XX for key PRIMARY'

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.0.0
    • Component/s: Hdfs Plugin
    • Labels:
      None

      Description

      In MAuthzPathsMapping table defines 1-N mapping of authzObjName to HMSPaths. However, current JDO metadata/SQL definition may cause duplicate entry error.

       2017-02-13 07:03:06,909 ERROR org.apache.sentry.provider.db.service.persistent.TransactionManager: The transaction has reached max retry numbe, rAdd request failed : INSERT INTO `MAUTHZPATHSMAPPING_PATHS` (`AUTHZ_OBJ_ID_OID`,`PATHS`) VALUES (?,?) 
      javax.jdo.JDODataStoreException: Add request failed : INSERT INTO `MAUTHZPATHSMAPPING_PATHS` (`AUTHZ_OBJ_ID_OID`,`PATHS`) VALUES (?,?) 
      FailedObject:MSentryPathsUpdate [authzObj=default.web_logs], paths=[[user/admin/2015_11_21, user/admin/2015_11_20, user/hive/warehouse/web_logs, user/admin/2015_11_18, user/admin/2015_11_19]], createTimeMs=[1486969386906]
      	at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:449)
      	at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
      	at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
      	at org.apache.sentry.provider.db.service.persistent.SentryStore.createAuthzPathsMappingCore(SentryStore.java:2200)
      	at org.apache.sentry.provider.db.service.persistent.SentryStore.access$2500(SentryStore.java:88)
      	at org.apache.sentry.provider.db.service.persistent.SentryStore$31.execute(SentryStore.java:2174)
      	at org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransaction(TransactionManager.java:114)
      	at org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransactionWithRetry(TransactionManager.java:183)
      	at org.apache.sentry.provider.db.service.persistent.SentryStore.persistFullPathsImage(SentryStore.java:2170)
      	at org.apache.sentry.service.thrift.HMSFollower.run(HMSFollower.java:263)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      NestedThrowablesStackTrace:
      org.datanucleus.store.rdbms.exceptions.MappedDatastoreException: INSERT INTO `MAUTHZPATHSMAPPING_PATHS` (`AUTHZ_OBJ_ID_OID`,`PATHS`) VALUES (?,?) 
      	at org.datanucleus.store.rdbms.scostore.JoinSetStore.doInternalAdd(JoinSetStore.java:754)
      	at org.datanucleus.store.rdbms.scostore.JoinSetStore.internalAdd(JoinSetStore.java:502)
      	at org.datanucleus.store.rdbms.scostore.JoinSetStore.addAll(JoinSetStore.java:433)
      	at org.datanucleus.store.rdbms.mapping.java.CollectionMapping.postInsert(CollectionMapping.java:136)
      	at org.datanucleus.store.rdbms.request.InsertRequest.execute(InsertRequest.java:519)
      	at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertTable(RDBMSPersistenceHandler.java:167)
      	at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:143)
      	at org.datanucleus.state.JDOStateManager.internalMakePersistent(JDOStateManager.java:3784)
      	at org.datanucleus.state.JDOStateManager.makePersistent(JDOStateManager.java:3760)
      	at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2219)
      	at org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:2065)
      	at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1913)
      	at org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)
      	at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:727)
      	at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
      	at org.apache.sentry.provider.db.service.persistent.SentryStore.createAuthzPathsMappingCore(SentryStore.java:2200)
      	at org.apache.sentry.provider.db.service.persistent.SentryStore.access$2500(SentryStore.java:88)
      	at org.apache.sentry.provider.db.service.persistent.SentryStore$31.execute(SentryStore.java:2174)
      	at org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransaction(TransactionManager.java:114)
      	at org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransactionWithRetry(TransactionManager.java:183)
      	at org.apache.sentry.provider.db.service.persistent.SentryStore.persistFullPathsImage(SentryStore.java:2170)
      	at org.apache.sentry.service.thrift.HMSFollower.run(HMSFollower.java:263)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.sql.BatchUpdateException: Duplicate entry '102' for key 'PRIMARY'
      	at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1815)
      	at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1277)
      	at com.jolbox.bonecp.StatementHandle.executeBatch(StatementHandle.java:469)
      	at org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeBatch(ParamLoggingPreparedStatement.java:372)
      	at org.datanucleus.store.rdbms.SQLController.processConnectionStatement(SQLController.java:683)
      	at org.datanucleus.store.rdbms.SQLController.executeStatementUpdate(SQLController.java:409)
      	at org.datanucleus.store.rdbms.scostore.JoinSetStore.doInternalAdd(JoinSetStore.java:733)
      	... 28 more
      Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '102' for key 'PRIMARY'
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
      	at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
      	at com.mysql.jdbc.Util.getInstance(Util.java:360)
      	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:971)
      	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
      	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
      	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
      	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
      	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2530)
      	at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1907)
      	at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2141)
      	at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1773)
      	... 34 more
      

      The possible fix could be either changing JDO metadata definition of 1-N mapping with adding a new HMSPaths table. Or change the SQL scripts.

        Attachments

        1. SENTRY-1629.001-sentry-ha-redesign.patch
          33 kB
          Krishna Kalyan
        2. SENTRY-1629.001-sentry-ha-redesign.patch
          33 kB
          Krishna Kalyan
        3. SENTRY-1629.002-sentry-ha-redesign.patch
          33 kB
          Krishna Kalyan

          Issue Links

            Activity

              People

              • Assignee:
                kkalyan Krishna Kalyan
                Reporter:
                hahao Hao Hao
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: