[ATLAS-3702] Entity Creation: Improve Edge Creation

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: trunk
• Fix Version/s: trunk
• Component/s: atlas-core
• Assignee: Ashutosh Mestry
• Reporter: Ashutosh Mestry
• Created: 29/Mar/20 16:49
• Updated: 18/May/20 03:24
• Resolved: 02/Apr/20 18:41

Description

Background

When creating entities with Atlas at scale, few inefficiencies were observed.

They were:

• GraphHelper.getAdjacentEdgesByLabel: The existing approach results in high memory usage. The method could use graph api that already exists.
• AtlasRelationshipStoreV2.getRelationshipEdge: This does full vertex comparison. This is expensive.
• Edge label is not index. Indexing this will improve speed of edge by label lookup.

Solution

Profile Atlas with (using metrics) to establish baseline.

Use the fixes mentioned to the places above and profile.

 

Issue Links

• contains: ATLAS-669 GraphHelper.getEdgeForLabel() can use edge index