[SYSTEMDS-1662] Extended HOP DAG validator

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Done
• Fix Version/s: SystemML 0.15
• Component/s: Compiler
• Labels: beginner
• Assignee: Shana Hutchison
• Reporter: Matthias Boehm
• Created: 03/Jun/17 22:07
• Updated: 30/Oct/17 03:59
• Resolved: 10/Jun/17 21:07

Description

This task aims to extend the existing HOP DAG validator (see org.apache.sysml.hops.rewrite.HopDagValidator, which can be enabled via org.apache.sysml.hops.rewriteProgramRewriter.CHECK) in various ways in order to provide better developer tooling for checking the correctness of new and existing rewrites.

So far, this validator, checks only for:

• Correct parent node linking
• Correct child node linking
• Non-empty children (for all hops other than DataOp and LiteralOp)

Possible extensions include (but are not limited to):

• Correct HOP output data types
• Correct HOP output value types
• Correct number of expected child nodes
• Correct output size information wrt input sizes
• Correct visit status

These extensions would be very useful for multiple reasons. First, they would detect rewrite issues early on in the development process. This is important because rewrite issues usually lead to strange and non-obvious behavior of real application scripts. Second, the HOP DAG validator provides a systematic way of debugging optimizer issues. The intended future workflow is as follows:

• 1. Disable rewrites via optimization level 1 to determine if rewrites are the issue.
• 2. Use the extended HopDagValidator validator to find the source of corruption.
• 3. If (2) did not find the issue, resort to low-level debugging, and extend the HopDagValidator to capture the root cause of the issue.

Issue Links

• is related to: SYSTEMDS-1980 Integer matrices