[HIVE-25414] Optimise Hive::addWriteNotificationLog: Reduce FS call per notification

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Labels: pull-request-available
• Assignee: mahesh kumar behera
• Reporter: Rajesh Balamohan
• Created: 02/Aug/21 22:52
• Updated: 15/Sep/21 08:54
• Resolved: 15/Sep/21 08:54

Description

AddWriteNotification is slow due to FS interactions (i.e to get the set of insert file information). This can be avoided as FileStatus can be passed instead of Path from parent methods.

https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L3572

https://github.com/apache/hive/blob/96b39cd5190f0cfadb677e3f3b7ead1d663921b2/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L3620

 

 

Issue Links

• links to: GitHub Pull Request #2576