[PIG-1745] Disable converting bytes loading from BinStorage

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Fix Version/s: 0.9.0
• Component/s: impl
• Hadoop Flags: Reviewed

Release Note

1. Converting bytes loading from BinStorage() will now result an error.
2. If user clearly knows the caster for the BinStorage, he/she can pass the caster class name to the construct of BinStorage. For example, if data is load from PigStorage (or other LoadFunc using Utf8StorageConverter) and store using BinStorage, user can pass "org.apache.pig.builtin.Utf8StorageConverter" to the construct of BinStorage (or pass "Utf8StorageConverter", since we will search for "org.apache.pig.builtin" package by default). By doing this, converting bytes to other type will still work.

• Assignee: Daniel Dai
• Reporter: Daniel Dai
• Created: 23/Nov/10 19:58
• Updated: 04/Aug/11 00:34
• Resolved: 06/Dec/10 06:56

Description

If we load bytes from BinStorage, we don't actually know how we get these bytes originally, and we will not have a way to cast those bytes. Ideally we shall encode caster into BinStorage data file, but we are not there yet. Currrently bytesToXXX methods for BinStorage is wrong and it results unexpected errors. Eg.

a = load '1.txt' as (a0, a1, a2);
store a into '1.bin' as BinStorage();

a = load '1.bin' using BinStorage as (a0, a1, a2);
b = foreach a generate (long)a0;
dump b;

The code will run but produce wrong data. It's less confusing if we throw an exception in this case.

Release Notes:
Pig will throw exception in the case we want to convert bytes loading from BinStorage

Issue Links

• incorporates: PIG-1341 BinStorage cannot convert DataByteArray to Chararray and results in FIELD_DISCARDED_TYPE_CONVERSION_FAILED
• incorporates: PIG-1222 cast ends up with NULL value