[HDFS-4678] libhdfs casts Japanese character incorrectly to Java API

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.1.2
• Component/s: libhdfs

Environment

Platform:    Linux64
Locale:    Japanese (ja_JP.UTF-8)

• Target Version/s: 1.3.0
• Assignee: Unassigned
• Reporter: Jiqiu
• Created: 09/Apr/13 08:27
• Updated: 20/Feb/14 03:25

Description

put a local file with Japanese characters to hdfs,
while browsing it in hdfs, it cannot be recognized.

here is the test.c

#include "hdfs.h"
#include <stdio.h>
#include <locale.h>

int main(int argc, char **argv) {
if(!setlocale(LC_CTYPE, "ja_JP"))

{ printf("Can not set locale type\n"); }

printf("0\n");
hdfsFS fs = hdfsConnect("localhost", 9000);
printf("1\n");
const char* writePath = "/tmp/\xF0\xA0\x80\x8B.txt";
printf("2\n");
hdfsFile writeFile = hdfsOpenFile(fs, writePath, O_WRONLY|O_CREAT, 0, 0, 0);
if(!writeFile)

{ fprintf(stderr, "Failed to open %s for writing!\n", writePath); exit(-1); }

char* buffer = "Hello, World! \xF0\xA0\x80\x8B";
tSize num_written_bytes = hdfsWrite(fs, writeFile, (void*)buffer, strlen(buffer)+1);
if (hdfsFlush(fs, writeFile))

{ fprintf(stderr, "Failed to 'flush' %s\n", writePath); exit(-1); }

printf("3\n");
hdfsCloseFile(fs, writeFile);
}