[HADOOP-5404] GenericOptionsParser should parse generic options even if they appear after Tool-specific options

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.19.1
• Component/s: util
• Labels: newbie

Environment

All

• Assignee: Unassigned
• Reporter: Milind Barve
• Created: 05/Mar/09 01:01
• Updated: 17/Feb/12 22:06
• Resolved: 17/Feb/12 22:06

Description

Currently, when GenericOptionsParser encounters an unrecognized option, it stops processing command-line arguments, and returns the rest to the specific Tool. This forces users to remember the order of arguments, and leads to errors such as following:

org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option:
-Dmapred.reduce.tasks=4
at org.apache.commons.cli.Parser.processOption(Parser.java:368)
at org.apache.commons.cli.Parser.parse(Parser.java:185)
at org.apache.commons.cli.Parser.parse(Parser.java:70)
at
MyTool.run(MyTool.java.java:290)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at
MyTool.main(MyTool.java:19)

In Hadoop-streaming as well, -D parameters should appear before streaming-specific arguments, such as -mapper, -reducer etc.

If GenericOptionsParser were to scan the entire command-line, ignoring unrecognized (tool-specific) options, and returning all unrecognized options back to the tool, this problem would be solved.

Issue Links

• duplicates: HADOOP-7995 GenericOptionsParser ought to have better options parsing, and not pick only the options in the front