Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1773

streaming doesn't support jobclient.output.filter

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: contrib/streaming
    • Labels:
      None
    • Environment:

      Linux

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Improved console messaging for streaming jobs by using the generic JobClient API itself instead of the existing streaming-specific code.

      Description

      the streaming Jobclient implementation i.e contrib/streaming/src/java/org/apache/hadoop/streaming/StreamJob.java is significantly different than the core hadoop mapred/org/apache/hadoop/mapred/JobClient.java.

      for example unlike StreamJob.java, JobClient.java it gets tasks log when jobclient.output.filter=ALL is specified .

      With hod-logs going away in hadoop 0.20 (due to new scheduler) user has no good way of programmitically getting logs

      We should have intermediate adaptor class to implement Tools for the purpose of submitting jobs via m/r, streaming, pipes so that we don't miss some core functionality.

      GenericJobClient implements Tools and then StreamJob extends GenericJobClient, JobClient extends GenericJobClient

      Alok

      1. patch-1773-1.txt
        2 kB
        Amareshwari Sriramadasu
      2. patch-1773.txt
        2 kB
        Amareshwari Sriramadasu

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Amareshwari Sriramadasu
            Reporter:
            Alok Singh
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development