[HIVE-16591] DR for function Binaries on HDFS

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: HiveServer2
• Assignee: Anishek Agarwal
• Reporter: Anishek Agarwal
• Created: 05/May/17 05:42
• Updated: 22/May/18 23:57
• Resolved: 31/May/17 16:21

Description

1. We have to make sure that during incremental dump we dont allow functions to be copied if they have local filesystem "file://" resources. – depends how much system side work we want to do, We are going to explicitly provide a caveat for replicating functions where in, only functions created "using" clause will be replicated and the "using" clause prohibits creating functions with the local "file://" resources and hence doing additional checks when doing repl dump might not be required.
2. We have to make sure that during the bootstrap / incremental dump we append the namenode host + port if functions are created without the fully qualified location of uri on hdfs, not sure how this would play for S3 or WASB filesystem.
3. We have to copy the binaries of a function resource list on CREATE / DROP FUNCTION . The change management file system has to keep a copy of the binary when a DROP function is called, to provide capability of updating binary definition for existing functions along with DR. An example of list of steps is given in doc (ReplicateFunctions.pdf ) attached in parent Issue.

Issue Links

• relates to: HIVE-16843 PrimaryToReplicaResourceFunctionTest.createDestinationPath fails with AssertionError
• links to: GitHub Pull Request #189