[FALCON-389] Submit Hcat export workflow to oozie on source cluster rather than to oozie on destination cluster

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.4
• Assignee: Unassigned
• Reporter: Arpit Gupta
• Created: 07/Apr/14 18:10
• Updated: 30/Apr/14 17:24

Description

Noticed this on hadoop-2 with oozie 4.x that when you run an hcat replication job where source and destination cluster's are different all jobs are submitted to oozie on the destination cluster. Then oozie runs an table export job that it submits to RM on cluster 1.

Now if the oozie server on the target cluster is not running with all hadoop configs it will not know all the appropriate hadoop configs and yarn job will fail. We saw jobs fail with errors like

org.apache.hadoop.security.token.SecretManager$InvalidToken: Password not found for ApplicationAttempt appattempt_1395965672651_0010_000002

on unsecure cluster as well.

Issue Links

• is related to: FALCON-379 Document limitations of Hive support in Falcon
• is related to: FALCON-401 replicate hcat table to two targets can fail with export dir exists on source