[SENTRY-872] Uber jira for HMS HA + Sentry HA redesign

• Type: New Feature
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.5.0, 2.0.0
• Fix Version/s: 2.0.0
• Component/s: Hdfs Plugin
• Assignee: Alex Kolbasov
• Reporter: Sravya Tirukkovalur
• Created: 02/Sep/15 22:57
• Updated: 27/Nov/17 13:32
• Resolved: 21/Nov/17 14:38

Issue Links

• contains: SENTRY-871 Refactor HA components based on Sentry-852
• contains: SENTRY-873 [HMS HA] Have a HMS leader which would be responsible for sending path updates to Sentry
• is blocked by: SENTRY-1596 Hive tests failing for sentry-ha-redesign branch
• is blocked by: SENTRY-1759 UpdatableCache leaks connections
• is blocked by: SENTRY-1323 Bump the hive version to 1.2.0
• is blocked by: SENTRY-1474 createSentryRole() isn't thread-safe
• is blocked by: SENTRY-1476 SentryStore is subject to JDQL injection
• is blocked by: SENTRY-1486 Sentry should use repeatable-read consistency level
• is blocked by: SENTRY-1491 Sentry transactions are not rolled back immediately when commit fails
• is blocked by: SENTRY-1644 Partition ACLs disappear after renaming Hive table with partitions
• is blocked by: SENTRY-1681 SentryHdfsServiceException is an unchecked exception
• is blocked by: SENTRY-1120 Show role / privileges info in Sentry Service Webpage
• is blocked by: SENTRY-1507 Sentry should use Datanucleus version of javax.jdo
• is related to: SENTRY-852 Create PathUpdateForwarder and PermUpdateForwarder
• is related to: SENTRY-1314 HDFS Sync improvements
• is related to: SENTRY-1589 Flaky test TestDbPrivilegeCleanupOnDrop.testReCreateObjects
• is related to: SENTRY-1508 MetastorePlugin.java does not handle properly initialization failure
• is related to: SENTRY-1500 Sentry should use Datanucleus version of javax.jdo
• is related to: SENTRY-1506 Importing JDO or datanucleus properties from config is broken
• is related to: SENTRY-1523 sentry-dist is missing dependency on mysql connector
• is related to: SENTRY-1526 Sentry processed stays alive after being killed
• is related to: SENTRY-1644 Partition ACLs disappear after renaming Hive table with partitions
• is related to: SENTRY-1699 MetastoreCacheInitializer shouldn't use Preconditions for HMS data
• is related to: SENTRY-1788 Sentry Store may use JDO object after the associated data is removed in DB
• is related to: SENTRY-1797 SentryKerberosContext should use periodic executor instead of managing periodic execution via run() method.
• is related to: SENTRY-1801 Sentry Namenode Plugin should handle unknown permissions
• is related to: SENTRY-1915 Sentry is doing a lot of work to convert list of paths to HMSPaths structure
• is related to: SENTRY-1916 Sentry should not store paths outside of the prefix
• is related to: SENTRY-1574 SentryMetastorePostEventListener class is not used by anything
• is related to: SENTRY-1524 sentry-dist is missing dependency on sentry-hdfs-dist
• is related to: SENTRY-1597 Document Sentry protocols
• is related to: SENTRY-1653 Dump Sentry metrics to a file periodically
• is related to: SENTRY-1494 Evaluate using a JSON notification log message as a Map rather than using a Deserializer with Java message object.
• is related to: SENTRY-1537 Investigate changing ROLE:PRIV model from M:N to 1:N
• is related to: SENTRY-1579 Sentry should avoid using Proxy
• is related to: SENTRY-627 Extend Sentry HA to work with Hive Metastore HA and HDFS
• is related to: SENTRY-1501 SentryStore shouldn't synchronize openTransaction() and commitUpdateTransaction()
• is related to: SENTRY-1518 Add metrics for SentryStore transactions
• is related to: SENTRY-1525 Provide script to run Sentry directly from the repo
• is related to: SENTRY-1573 SentryMetastoreListenerPlugin should be an interface
• is related to: SENTRY-1820 Add JSON file reporter for Sentry metrics
• is related to: SENTRY-1867 DataNucleus.Query INFO level logging spams Sentry log files
• is related to: SENTRY-1531 Add transaction metrics for SentryStore
• is related to: SENTRY-1594 TransactionBlock should become generic
• is related to: SENTRY-1540 SentryStore.isMultiActionsSupported() is always true
• is related to: SENTRY-1503 Remove all sequence ID checks from TestSentryStore
• is related to: SENTRY-1502 Multiple tests in TestSentryStore are not testing useful things
• is related to: SENTRY-1482 Evaluate 4000 char limit for URI and semantics around overflow
• Parent Feature: SENTRY-1721 Unit test failures in TestSentryStore due to changeId miscount
• Parent Feature: SENTRY-1771 HDFS client concurrently requests full permission update multiple times
• relates to: SENTRY-1440 Fencing implementation in sentry-ha can create two fencing tables
• relates to: SENTRY-880 Delete the path cache children as soon as it is processed.
• relates to: SENTRY-883 [HA] Pre existing children in the path cache trigger child added events
• relates to: SENTRY-876 Handle path updates when Sentry HA + HDFS sync - Send updates to all sentry instances
• relates to: SENTRY-1481 Evaluate use of paths in MAuthzPathsMapping
• relates to: SENTRY-662 SentryServiceIntegrationBase should use UGI based login
• relates to: SENTRY-870 Create UpdateForwarders for paths and permissions
• relates to: SENTRY-882 HMS leader should handle out of order updates when HA
• relates to: SENTRY-1315 Add an interface in WebUI to request for a Sentry full update
• relates to: SENTRY-1322 Persist HMSPaths full update into Sentry DB
• relates to: SENTRY-877 Recreate the metastore cache if ZK connection is lost.
• links to: Review link