[HADOOP-9782] Datanode daemon cannot be started on OS X

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha1

Environment

OS X

• Assignee: Unassigned
• Reporter: Arpit Agarwal
• Created: 27/Jul/13 20:57
• Updated: 12/May/16 18:25
• Resolved: 30/Jul/13 01:57

Description

Datanode fails to start with the following exception on OS X.

java.lang.UnsupportedOperationException: stat is not supported on this platform
        at org.apache.hadoop.fs.Stat.getExecString(Stat.java:91)
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:405)
        at org.apache.hadoop.util.Shell.run(Shell.java:400)
        at org.apache.hadoop.fs.Stat.getFileStatus(Stat.java:65)
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:792)
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:523)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:398)
        at org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:129)
        at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:146)
        at org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker.checkDir(DataNode.java:1782)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.getDataDirsFromURIs(DataNode.java:1829)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1807)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1726)
        at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1190)
        at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:665)
        at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:334)
        at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:316)
        at org.apache.hadoop.mapreduce.v2.TestMRJobs.setup(TestMRJobs.java:112)

It appears to be caused by Stat#getExecString not supporting OS X.

Issue Links

• is broken by: HADOOP-9652 Allow RawLocalFs#getFileLinkStatus to fill in the link owner and mode if requested
• is duplicated by: HADOOP-9783 Fix OS detection for RawLocalFileSystem#getFileLinkStatus fallback path
• relates to: HADOOP-9783 Fix OS detection for RawLocalFileSystem#getFileLinkStatus fallback path