[PIG-771] PigDump does not properly output Chinese UTF8 characters - they are displayed as question marks ??

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.7.0
• Assignee: Unassigned
• Reporter: David Ciemiewicz
• Created: 20/Apr/09 16:28
• Updated: 27/Aug/10 23:32
• Resolved: 27/Aug/10 23:32

Description

PigDump does not properly output Chinese UTF8 characters.

The reason for this is that the function Tuple.toString() is called.

DefaultTuple implements Tuple.toString() and it calls Object.toString() on the opaque object d.

Instead, I think that the code should be changed instead to call the new DataType.toString() function.

    @Override
    public String toString() {
        StringBuilder sb = new StringBuilder();
        sb.append('(');
        for (Iterator<Object> it = mFields.iterator(); it.hasNext();) {
            Object d = it.next();
            if(d != null) {
                if(d instanceof Map) {
                    sb.append(DataType.mapToString((Map<Object, Object>)d));
                } else {
                    sb.append(DataType.toString(d));  // <<< Change this one line
                    if(d instanceof Long) {
                        sb.append("L");
                    } else if(d instanceof Float) {
                        sb.append("F");
                    }
                }
            } else {
                sb.append("");
            }
            if (it.hasNext())
                sb.append(",");
        }
        sb.append(')');
        return sb.toString();
    }