[SENTRY-1792] Ensure DB to sort delta changes by CHANGE_ID

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Component/s: Sentry
• Labels: sentry-ha
• Assignee: Vamsee K. Yarlagadda
• Reporter: Na Li
• Created: 05/Jun/17 22:12
• Updated: 24/Jul/17 14:51
• Resolved: 08/Jun/17 01:49

Description

Sentry server returns full permission snapshot almost every 10 seconds. The desired behavior is to reduce the chance of getting full snapshot significantly.

From debug message, we can see the returned change list is empty.
That causes Sentry to send back full snapshot instead of delta changes. It has performance penalty because it takes much longer to get a full snapshot than getting the delta changes.

2017-06-05 14:00:21,030 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1076942, curSeqNum:1076948 ; type: PermImageRetriever
2017-06-05 14:00:31,905 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1076949, curSeqNum:1077064 ; type: PermImageRetriever
2017-06-05 14:01:00,041 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1077354, curSeqNum:1077361 ; type: PermImageRetriever
2017-06-05 14:01:25,735 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1077630, curSeqNum:1077636 ; type: PermImageRetriever
2017-06-05 14:01:42,197 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1077820, curSeqNum:1077826 ; type: PermImageRetriever
2017-06-05 14:01:53,491 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1077827, curSeqNum:1077962 ; type: PermImageRetriever
2017-06-05 14:02:09,472 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078147, curSeqNum:1078155 ; type: PermImageRetriever
2017-06-05 14:02:21,527 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078300, curSeqNum:1078305 ; type: PermImageRetriever
2017-06-05 14:02:35,761 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078483, curSeqNum:1078489 ; type: PermImageRetriever
2017-06-05 14:02:50,079 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078683, curSeqNum:1078686 ; type: PermImageRetriever
2017-06-05 14:03:09,889 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078958, curSeqNum:1078961 ; type: PermImageRetriever
2017-06-05 14:03:21,641 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1079144, curSeqNum:1079150 ; type: PermImageRetriever
2017-06-05 14:03:32,600 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1079324, curSeqNum:1079331 ; type: PermImageRetriever
2017-06-05 14:03:46,223 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1079537, curSeqNum:1079546 ; type: PermImageRetriever
2017-06-05 14:03:59,870 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1079762, curSeqNum:1079769 ; type: PermImageRetriever

Issue Links

• is cloned by: SENTRY-1796 Add better debug logging for retrieving the delta changes
• is related to: SENTRY-1795 Delta tables should not have holes
• links to: Review board