Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.13.0
-
None
Description
see also in following parquet issue:
https://github.com/Parquet/parquet-mr/issues/290
The schema written for maps isn't compatible between hive and pig. This means any files written in one cannot be properly read in the other.
More specifically, for the same map column c1, parquet-pig generates schema:
message pig_schema {
optional group c1 (MAP) {
repeated group map (MAP_KEY_VALUE)
}
}
while parquet-hive generates schema:
message hive_schema {
optional group c1 (MAP_KEY_VALUE) {
repeated group map
}
}