[ATLAS-3715] Atlas: Improve Speed of Primitive Attribute Assignment

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: trunk
• Fix Version/s: trunk
• Component/s: atlas-core
• Assignee: Ashutosh Mestry
• Reporter: Ashutosh Mestry
• Created: 02/Apr/20 06:01
• Updated: 01/May/20 18:47
• Resolved: 01/May/20 18:47

Description

Background

While profiling entity creation as part of zip file-based migration import, it was observed that over 50% of time is spent in assigning values to attributes that are of primitive type viz. string, date, etc.

Methodology Used

The entity creation flow already has MetricRecorder added to key steps within the entity creation flow. Additional methods (like mapPrimitiveValue) were profiled to get complete breakdown.

Data set contained large number of entities of various types. Ones used were hdfs_path, hive_db, hive_table, hive_column, hive_process, hive_process_execution, hive_column_lineage.

Solution

Within the migration request context, use the setEncodedProperty instead of using the current route of creating AtttributeMutationContext.