[BIGTOP-1763] Handle broken symlinks when copying jars

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.9.0, 1.0.0
• Fix Version/s: 1.0.0
• Component/s: deployment
• Assignee: Peter Slawski
• Reporter: Peter Slawski
• Created: 18/Mar/15 20:25
• Updated: 27/Mar/15 00:44
• Resolved: 27/Mar/15 00:44

Description

When Hive is installed on a cluster and HBase isn't, init-hcfs.groovy will fail copying jars to HDFS for Oozie during puppet apply. This is because of a broken simlink in /usr/lib/hive/lib, namely hbase.jar which links to /usr/lib/hbase/hbase.jar.

This is introduced by BIGTOP-1235.

The exception thrown in init-hcfs.groovy:

java.io.FileNotFoundException: File /usr/lib/hive/lib/hbase.jar does not exist
  at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:516)
  at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:729)
  at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:506)
  at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:407)
  at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
  at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1951)
  at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1919)
  at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1884)
  at org.apache.hadoop.fs.FileSystem$copyFromLocalFile$1.call(Unknown Source)
  at init-hcfs$_run_closure7_closure8.doCall(init-hcfs.groovy:258)
  at init-hcfs$_run_closure7.doCall(init-hcfs.groovy:252)
  at init-hcfs.run(init-hcfs.groovy:281)

Issue Links

• relates to: BIGTOP-1235 Speed-up init-hdfs.sh for complete HCFS compliant provisioning