Uploaded image for project: 'Sentry (Retired)'
  1. Sentry (Retired)
  2. SENTRY-2302 Optimize HDFS ACL Synchronization.
  3. SENTRY-2305

Optimize time taken for persistence HMS snapshot by persisting in parallel



    • Sub-task
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 2.1.0
    • None
    • Sentry
    • None


      There are couple of options

      1. Break the total snapshot into to batches and persist all of them in parallel in different transactions. As sentry uses repeatable_read isolation level we should be able to have parallel writes on the same table. This bring an issue if there is a failure in persisting any of the batches. This approach needs additional logic of cleaning the partially persisted snapshot. I’m evaluating this option.
        • Result: Initial results are promising. Time to persist the snapshot came down by 60%.
      2. Try disabling L1 Cache for persisting the snapshot.
      3. Try persisting the snapshot entries sequentially in separate transactions. As transactions which commit huge data might take longer as they take a lot of CPU cycles to keep the rollback log up to date.


        1. SENTRY-2305.007.patch
          25 kB
          Krishna Kalyan
        2. SENTRY-2305.005.patch
          21 kB
          Krishna Kalyan
        3. SENTRY-2305.004.patch
          18 kB
          Krishna Kalyan
        4. SENTRY-2305.003.patch
          17 kB
          Krishna Kalyan
        5. SENTRY-2305.002.patch
          14 kB
          Krishna Kalyan
        6. SENTRY-2305.001.patch
          11 kB
          Krishna Kalyan

        Issue Links



              kkalyan Krishna Kalyan
              kkalyan Krishna Kalyan
              0 Vote for this issue
              3 Start watching this issue