XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0
    • Sentry

    Description

      Sentry server returns full permission snapshot almost every 10 seconds. The desired behavior is to reduce the chance of getting full snapshot significantly.

      From debug message, we can see the returned change list is empty.
      That causes Sentry to send back full snapshot instead of delta changes. It has performance penalty because it takes much longer to get a full snapshot than getting the delta changes.

      2017-06-05 14:00:21,030 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1076942, curSeqNum:1076948 ; type: PermImageRetriever
      2017-06-05 14:00:31,905 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1076949, curSeqNum:1077064 ; type: PermImageRetriever
      2017-06-05 14:01:00,041 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1077354, curSeqNum:1077361 ; type: PermImageRetriever
      2017-06-05 14:01:25,735 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1077630, curSeqNum:1077636 ; type: PermImageRetriever
      2017-06-05 14:01:42,197 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1077820, curSeqNum:1077826 ; type: PermImageRetriever
      2017-06-05 14:01:53,491 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1077827, curSeqNum:1077962 ; type: PermImageRetriever
      2017-06-05 14:02:09,472 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078147, curSeqNum:1078155 ; type: PermImageRetriever
      2017-06-05 14:02:21,527 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078300, curSeqNum:1078305 ; type: PermImageRetriever
      2017-06-05 14:02:35,761 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078483, curSeqNum:1078489 ; type: PermImageRetriever
      2017-06-05 14:02:50,079 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078683, curSeqNum:1078686 ; type: PermImageRetriever
      2017-06-05 14:03:09,889 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078958, curSeqNum:1078961 ; type: PermImageRetriever
      2017-06-05 14:03:21,641 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1079144, curSeqNum:1079150 ; type: PermImageRetriever
      2017-06-05 14:03:32,600 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1079324, curSeqNum:1079331 ; type: PermImageRetriever
      2017-06-05 14:03:46,223 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1079537, curSeqNum:1079546 ; type: PermImageRetriever
      2017-06-05 14:03:59,870 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1079762, curSeqNum:1079769 ; type: PermImageRetriever
      

      Attachments

        1. SENTRY-1792.001-sentry-ha-redesign.patch
          5 kB
          Na Li
        2. SENTRY-1792.002-sentry-ha-redesign.patch
          11 kB
          Vamsee K. Yarlagadda
        3. SENTRY-1792.003-sentry-ha-redesign.patch
          12 kB
          Vamsee K. Yarlagadda
        4. SENTRY-1792.004-sentry-ha-redesign.patch
          0.8 kB
          Vamsee K. Yarlagadda

        Issue Links

          Activity

            People

              vamsee Vamsee K. Yarlagadda
              linaataustin Na Li
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: