[ATLAS-3314] Update relationshipCategory between spark_table and spark_storagedesc type

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Component/s: atlas-intg
• Assignee: Unassigned
• Reporter: Jungtaek Lim
• Created: 02/Jul/19 23:39
• Updated: 27/Mar/20 04:58
• Resolved: 03/Jul/19 04:56

Description

Spark models define the relationship of "spark_table" and "spark_storagedesc" as "COMPOSITION", which requires "spark_storagedesc" to refer "spark_table". (Note that Atlas 1.x didn't require in same relationship.)

This enforces the sequence of creating entities (as Atlas doesn't support referring other entity in same creation request) to "spark_table" -> "spark_storagedesc", which doesn't seem to work in every usages, at least this requires major changes on Spark Atlas Connector.

As a workaround, we can loosen the relationship via defining the relationship as "ASSOCIATION", so that they can refer each other at any side.

Issue Links

• links to: GitHub Pull Request #64