Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4847

Command Parsing in Hadoop Streaming

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: contrib/streaming
    • Labels:

      Description

      Hadoop streaming parse the mapper and reducer commands by itself, this is not a good choice, when I write a complex mapper/reducer script inline, such as 'perl -ne ...', it don't work.
      An alternative way is to send the command to the shell, simply create new process(sh -c "command_and_args"), this not also simplize the streaming code, but also improve its capability!

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              penglei Peng Lei
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 4h
                4h
                Remaining:
                Remaining Estimate - 4h
                4h
                Logged:
                Time Spent - Not Specified
                Not Specified