Hadoop Common
  1. Hadoop Common
  2. HADOOP-9909

org.apache.hadoop.fs.Stat should permit other LANG

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 2.3.0
    • Component/s: fs
    • Labels:
      None
    • Environment:

      CentOS 6.4 / LANG=ja_JP.UTF-8

    • Target Version/s:

      Description

      I executed "hdfs dfs -put" command and displayed following warning message. And "hdfs dfs -put" command was success.
      This is because Stat.parseExecResult() check a message of stat command from only English.

      [hadoop@trunk ~]$ hdfs dfs -put fugafuga.txt .
      13/08/27 16:24:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      13/08/27 16:24:37 WARN fs.FSInputChecker: Problem opening checksum file: file:/home/hadoop/fugafuga.txt.  Ignoring exception:
      java.io.IOException: Unexpected stat output: stat: cannot stat `/home/hadoop/.fugafuga.txt.crc': そのようなファイルやディレクトリはありません
              at org.apache.hadoop.fs.Stat.parseExecResult(Stat.java:163)
              at org.apache.hadoop.util.Shell.runCommand(Shell.java:489)
              at org.apache.hadoop.util.Shell.run(Shell.java:417)
              at org.apache.hadoop.fs.Stat.getFileStatus(Stat.java:68)
              at org.apache.hadoop.fs.RawLocalFileSystem.getNativeFileLinkStatus(RawLocalFileSystem.java:806)
              at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:738)
              at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:523)
              at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1397)
              at org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:210)
              at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:143)
              at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:339)
              at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:763)
              at org.apache.hadoop.fs.shell.CommandWithDestination.copyFileToTarget(CommandWithDestination.java:239)
              at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:183)
              at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:168)
              at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:310)
              at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:282)
              at org.apache.hadoop.fs.shell.CommandWithDestination.processPathArgument(CommandWithDestination.java:163)
              at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:264)
              at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:248)
              at org.apache.hadoop.fs.shell.CommandWithDestination.processArguments(CommandWithDestination.java:140)
              at org.apache.hadoop.fs.shell.CopyCommands$Put.processArguments(CopyCommands.java:224)
              at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:194)
              at org.apache.hadoop.fs.shell.Command.run(Command.java:155)
              at org.apache.hadoop.fs.FsShell.run(FsShell.java:255)
              at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
              at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
              at org.apache.hadoop.fs.FsShell.main(FsShell.java:308)
      Caused by: java.lang.NumberFormatException: For input string: "stat: cannot stat `/home/hadoop/.fugafuga.txt.crc': そのようなファイルやディレクトリはありません"
              at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
              at java.lang.Long.parseLong(Long.java:441)
              at java.lang.Long.parseLong(Long.java:483)
              at org.apache.hadoop.fs.Stat.parseExecResult(Stat.java:128)
              ... 27 more
      
      1. HADOOP-9909.patch
        1 kB
        Shinichi Yamashita
      2. HADOOP-9909.patch
        1 kB
        Shinichi Yamashita
      3. HADOOP-9909.patch
        3 kB
        Shinichi Yamashita
      4. HADOOP-9909.patch
        3 kB
        Shinichi Yamashita
      5. HADOOP-9909.patch
        3 kB
        Shinichi Yamashita
      6. console.log
        9 kB
        Akira AJISAKA

        Activity

        Shinichi Yamashita created issue -
        Shinichi Yamashita made changes -
        Field Original Value New Value
        Description I executed "hdfs dfs -put" command and displayed following warning message. And "hdfs dfs -put" command was success.
        This is because Stat.parseExecResult() check a message of stat command from only English.

        {quote}
        [hadoop@trunk ~]$ hdfs dfs -put fugafuga.txt .
        13/08/27 16:24:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
        13/08/27 16:24:37 WARN fs.FSInputChecker: Problem opening checksum file: file:/home/hadoop/fugafuga.txt. Ignoring exception:
        java.io.IOException: Unexpected stat output: stat: cannot stat `/home/hadoop/.fugafuga.txt.crc': そのようなファイルやディレクトリはありません
                at org.apache.hadoop.fs.Stat.parseExecResult(Stat.java:163)
                at org.apache.hadoop.util.Shell.runCommand(Shell.java:489)
                at org.apache.hadoop.util.Shell.run(Shell.java:417)
                at org.apache.hadoop.fs.Stat.getFileStatus(Stat.java:68)
                at org.apache.hadoop.fs.RawLocalFileSystem.getNativeFileLinkStatus(RawLocalFileSystem.java:806)
                at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:738)
                at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:523)
                at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1397)
                at org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:210)
                at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:143)
                at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:339)
                at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:763)
                at org.apache.hadoop.fs.shell.CommandWithDestination.copyFileToTarget(CommandWithDestination.java:239)
                at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:183)
                at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:168)
                at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:310)
                at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:282)
                at org.apache.hadoop.fs.shell.CommandWithDestination.processPathArgument(CommandWithDestination.java:163)
                at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:264)
                at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:248)
                at org.apache.hadoop.fs.shell.CommandWithDestination.processArguments(CommandWithDestination.java:140)
                at org.apache.hadoop.fs.shell.CopyCommands$Put.processArguments(CopyCommands.java:224)
                at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:194)
                at org.apache.hadoop.fs.shell.Command.run(Command.java:155)
                at org.apache.hadoop.fs.FsShell.run(FsShell.java:255)
                at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
                at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
                at org.apache.hadoop.fs.FsShell.main(FsShell.java:308)
        Caused by: java.lang.NumberFormatException: For input string: "stat: cannot stat `/home/hadoop/.fugafuga.txt.crc': そのようなファイルやディレクトリはありません"
                at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
                at java.lang.Long.parseLong(Long.java:441)
                at java.lang.Long.parseLong(Long.java:483)
                at org.apache.hadoop.fs.Stat.parseExecResult(Stat.java:128)
                ... 27 more
        {quote}
        I executed "hdfs dfs -put" command and displayed following warning message. And "hdfs dfs -put" command was success.
        This is because Stat.parseExecResult() check a message of stat command from only English.

        {code}
        [hadoop@trunk ~]$ hdfs dfs -put fugafuga.txt .
        13/08/27 16:24:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
        13/08/27 16:24:37 WARN fs.FSInputChecker: Problem opening checksum file: file:/home/hadoop/fugafuga.txt. Ignoring exception:
        java.io.IOException: Unexpected stat output: stat: cannot stat `/home/hadoop/.fugafuga.txt.crc': そのようなファイルやディレクトリはありません
                at org.apache.hadoop.fs.Stat.parseExecResult(Stat.java:163)
                at org.apache.hadoop.util.Shell.runCommand(Shell.java:489)
                at org.apache.hadoop.util.Shell.run(Shell.java:417)
                at org.apache.hadoop.fs.Stat.getFileStatus(Stat.java:68)
                at org.apache.hadoop.fs.RawLocalFileSystem.getNativeFileLinkStatus(RawLocalFileSystem.java:806)
                at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:738)
                at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:523)
                at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1397)
                at org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:210)
                at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:143)
                at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:339)
                at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:763)
                at org.apache.hadoop.fs.shell.CommandWithDestination.copyFileToTarget(CommandWithDestination.java:239)
                at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:183)
                at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:168)
                at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:310)
                at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:282)
                at org.apache.hadoop.fs.shell.CommandWithDestination.processPathArgument(CommandWithDestination.java:163)
                at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:264)
                at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:248)
                at org.apache.hadoop.fs.shell.CommandWithDestination.processArguments(CommandWithDestination.java:140)
                at org.apache.hadoop.fs.shell.CopyCommands$Put.processArguments(CopyCommands.java:224)
                at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:194)
                at org.apache.hadoop.fs.shell.Command.run(Command.java:155)
                at org.apache.hadoop.fs.FsShell.run(FsShell.java:255)
                at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
                at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
                at org.apache.hadoop.fs.FsShell.main(FsShell.java:308)
        Caused by: java.lang.NumberFormatException: For input string: "stat: cannot stat `/home/hadoop/.fugafuga.txt.crc': そのようなファイルやディレクトリはありません"
                at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
                at java.lang.Long.parseLong(Long.java:441)
                at java.lang.Long.parseLong(Long.java:483)
                at org.apache.hadoop.fs.Stat.parseExecResult(Stat.java:128)
                ... 27 more
        {code}
        Shinichi Yamashita made changes -
        Attachment HADOOP-9909.patch [ 12600494 ]
        Shinichi Yamashita made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Target Version/s 3.0.0 [ 12320357 ]
        Shinichi Yamashita made changes -
        Attachment HADOOP-9909.patch [ 12600519 ]
        Shinichi Yamashita made changes -
        Attachment HADOOP-9909.patch [ 12600637 ]
        Akira AJISAKA made changes -
        Attachment console.log [ 12600661 ]
        Akira AJISAKA made changes -
        Priority Minor [ 4 ] Major [ 3 ]
        Shinichi Yamashita made changes -
        Attachment HADOOP-9909.patch [ 12600665 ]
        Shinichi Yamashita made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Shinichi Yamashita made changes -
        Attachment HADOOP-9909.patch [ 12600685 ]
        Shinichi Yamashita made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Andrew Wang made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Target Version/s 3.0.0 [ 12320357 ] 3.0.0, 2.3.0 [ 12320357, 12324587 ]
        Fix Version/s 3.0.0 [ 12320357 ]
        Fix Version/s 2.3.0 [ 12324587 ]
        Resolution Fixed [ 1 ]
        Arun C Murthy made changes -
        Fix Version/s 2.3.0 [ 12325254 ]
        Fix Version/s 2.4.0 [ 12324587 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Allen Wittenauer made changes -
        Fix Version/s 3.0.0 [ 12320357 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Shinichi Yamashita
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development