[SUBMARINE-409] Submarine spark security

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.4.0
• Component/s: Security
• Target Version: 0.4.0
• Assignee: Kent Yao 2
• Reporter: Liu Xun
• Created: 07/Mar/20 12:52
• Updated: 11/Nov/21 08:02
• Resolved: 19/Mar/20 14:27

Description

By developing the spark ranger plugin according to the interface of ranger and spark, Spark can be connected with the ranger system, and the database and tables in sparksql can be authorized to air.

Issue Links

• is related to: SPARK-25003 Pyspark Does not use Spark Sql Extensions
• is related to: SUBMARINE-408 [Umbrella] Submarine security
• is related to: SUBMARINE-1078 Retire Spark security module from submarine project
• relates to: SUBMARINE-834 Ranger support for Spark Datasources

Sub-Tasks

• Basic SQL Standard Authorization ACL Management for Spark: Resolved
• Custom show databases command to allow exposing filtered databases: Resolved
• Custom show tables command to allow exposing filtered tables: Resolved
• Suppport Row Filter: Resolved
• Support Data Masking: Resolved
• Full TPCDS Test coverage: Resolved
• Support Configuration Restriction: Resolved
• Expose Security API with Authz w/ w/o DataMasking and Row Filtering: Resolved
• Refine data masking plan resolution for spark security: Resolved
• Add documentation for spark security: Resolved
• Predicates push down support through submarine temp markers: Resolved
• Make RangerSparkPlugin static: Resolved
• Fix escaped correlated subquery cases: Resolved
• ResetCommand should not wipe out all configs: Resolved