Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
The postProcessing method in the pig server checks whether a logical graph contains stores to and loads from the same location. If so, it will either connect the store and load, or optimize by throwing out the load and connecting the store predecessor with the successor of the load.
Ideally the introduction of the store and load connection should happen in the query compiler, while the optimization should then happen in an separate optimizer step as part of the optimizer framework.