Details
-
New Feature
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
This issue is about adding a new datatype to Pig that abstracts a spatial attribute. Following OGC http://www.opengeospatial.org/, we will add a new datatype called 'Geometry' that abstracts all standard shapes (e.g., Point, Polygon and Linestring). This datatype is automatically parsed from either a Well-Known Text (WKT) or Well-Known Binary (WKB) represented as a Hex string. These two types are the standard export formats for OGC shapes and they are supported by many existing tools including PostGIS http://postgis.net/. Exporting through PigStorage should default to a WKB represented as Hex string and there will be additional functions to convert to WKT.
This new datatype maps internally to the class OGCGeometry https://github.com/Esri/geometry-api-java/blob/master/src/com/esri/core/geometry/ogc/OGCGeometry.java licensed under Apache license. This class contains functionality to import/export to the WKT and WKB formats.
Data manipulation functions to the new datatype will be all done through UDFs. Currently, there is a spatial extension to Pig (called Pigeon) https://github.com/aseldawy/pigeon that provides basic spatial functionality via UDFs powered by the aforementioned library. Currently, it automatically converts WKB and WKT fields to OGCGeometry class, performs the spatial operation, and produces the result back as WKB. Once the Geometry datatype is added, it will natively use it to avoid the conversion.