[HIVE-869] Allow ScriptOperator to consume not all input data

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.5.0
• Fix Version/s: 0.5.0
• Hadoop Flags: Reviewed
• Release Note: HIVE-869. Allow ScriptOperator to consume not all input data. (Paul Yang via zshao)
• Assignee: Paul Yang
• Reporter: Zheng Shao
• Created: 05/Oct/09 21:49
• Updated: 17/Dec/11 00:05
• Resolved: 12/Oct/09 19:06

Description

The ScriptOperator (SELECT TRANSFORM(a, b, c) USING 'myscript' AS (d, e, f) ...) has a problem:
If the user script exits without consuming all data from standard input, then we will report an error even if the exit code from the user script is 0.

We want to have an option, when enabled, ScriptOperator will return successfully in that case.

If the option is not enabled, then we should stick to the current behavior.

The option can be called: "hive.exec.script.allow.partial.consumption ".

Issue Links

• is duplicated by: HIVE-386 Streaming processes should be able to return successfully without consuming all their input data.