Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-6048

DFSClient fails if native library doesn't exist

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Duplicate
    • Affects Version/s: 3.0.0, 2.4.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      When I executed FSShell commands (such as hdfs dfs -ls, -mkdir, -cat) in trunk, UnsupportedOperationException occurred in o.a.h.net.unix.DomainSocketWatcher and the commands failed.

        Issue Links

          Activity

          Hide
          Akira AJISAKA added a comment -

          Thank you for notifying me, Aaron T. Myers.

          Show
          Akira AJISAKA added a comment - Thank you for notifying me, Aaron T. Myers .
          Hide
          Aaron T. Myers added a comment -

          Hi Akira, I think this will be addressed by HDFS-6040, which should be committed shortly.

          Show
          Aaron T. Myers added a comment - Hi Akira, I think this will be addressed by HDFS-6040 , which should be committed shortly.
          Hide
          Akira AJISAKA added a comment -

          The stacktrace is as follows:

          [root@trunk ~]# hdfs dfs -ls
          14/03/04 10:28:59 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
          -ls: Fatal internal error
          java.lang.UnsupportedOperationException: libhadoop cannot be loaded.
          	at org.apache.hadoop.net.unix.DomainSocketWatcher.<init>(DomainSocketWatcher.java:229)
          	at org.apache.hadoop.hdfs.client.DfsClientShmManager.<init>(DfsClientShmManager.java:404)
          	at org.apache.hadoop.hdfs.client.ShortCircuitCache.<init>(ShortCircuitCache.java:380)
          	at org.apache.hadoop.hdfs.ClientContext.<init>(ClientContext.java:96)
          	at org.apache.hadoop.hdfs.ClientContext.get(ClientContext.java:145)
          	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:587)
          	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:507)
          	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:144)
          	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2396)
          	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
          	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2430)
          	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2412)
          	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
          	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:167)
          	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:352)
          	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
          	at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:325)
          	at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:228)
          	at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:211)
          	at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:194)
          	at org.apache.hadoop.fs.shell.Command.run(Command.java:155)
          	at org.apache.hadoop.fs.FsShell.run(FsShell.java:255)
          	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
          	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
          	at org.apache.hadoop.fs.FsShell.main(FsShell.java:308)
          
          Show
          Akira AJISAKA added a comment - The stacktrace is as follows: [root@trunk ~]# hdfs dfs -ls 14/03/04 10:28:59 WARN util.NativeCodeLoader: Unable to load native -hadoop library for your platform... using builtin-java classes where applicable -ls: Fatal internal error java.lang.UnsupportedOperationException: libhadoop cannot be loaded. at org.apache.hadoop.net.unix.DomainSocketWatcher.<init>(DomainSocketWatcher.java:229) at org.apache.hadoop.hdfs.client.DfsClientShmManager.<init>(DfsClientShmManager.java:404) at org.apache.hadoop.hdfs.client.ShortCircuitCache.<init>(ShortCircuitCache.java:380) at org.apache.hadoop.hdfs.ClientContext.<init>(ClientContext.java:96) at org.apache.hadoop.hdfs.ClientContext.get(ClientContext.java:145) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:587) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:507) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:144) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2396) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2430) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2412) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:167) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:352) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:325) at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:228) at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:211) at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:194) at org.apache.hadoop.fs.shell.Command.run(Command.java:155) at org.apache.hadoop.fs.FsShell.run(FsShell.java:255) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.hadoop.fs.FsShell.main(FsShell.java:308)

            People

            • Assignee:
              Unassigned
              Reporter:
              Akira AJISAKA
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development