Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2410

document multiple keys per reducer oddity in hadoop streaming FAQ

    Details

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Add an FAQ entry regarding the differences between Java API and Streaming development of MR programs.
    • Tags:
      streaming

      Description

      Hi,
      for a newcomer to hadoop streaming, it comes as a surprise that the reducer receives arbitrary keys, unlike the "real" hadoop where a reducer works on a single key.
      An explanation for this is @ http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201103.mbox/browser

      I suggest to add this to the FAQ of hadoop streaming

      1. MAPREDUCE-2410.r1.diff
        1.0 kB
        Harsh J
      2. MAPREDUCE-2410.r2.diff
        1 kB
        Harsh J
      3. MAPREDUCE-2410.r3.diff
        1 kB
        Harsh J

        Activity

        Dieter Plaetinck created issue -
        Amareshwari Sriramadasu made changes -
        Field Original Value New Value
        Project Hadoop Common [ 12310240 ] Hadoop Map/Reduce [ 12310941 ]
        Key HADOOP-7213 MAPREDUCE-2410
        Component/s contrib/streaming [ 12312905 ]
        Component/s documentation [ 12312910 ]
        Component/s documentation [ 12311160 ]
        Todd Lipcon made changes -
        Labels documentation newbie
        Harsh J made changes -
        Attachment MAPREDUCE-2410.r1.diff [ 12478785 ]
        Harsh J made changes -
        Assignee Harsh J Chouraria [ qwertymaniac ]
        Harsh J made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.20.2 [ 12314205 ]
        Fix Version/s 0.23.0 [ 12315570 ]
        Harsh J made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Harsh J made changes -
        Attachment MAPREDUCE-2410.r2.diff [ 12478828 ]
        Harsh J made changes -
        Attachment MAPREDUCE-2410.r3.diff [ 12478845 ]
        Harsh J made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Release Note Add an FAQ entry regarding the differences between Java API and Streaming development of MR programs.
        Todd Lipcon made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Fix Version/s 0.22.0 [ 12314184 ]
        Fix Version/s 0.23.0 [ 12315570 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Harsh J
            Reporter:
            Dieter Plaetinck
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 40m
              40m
              Remaining:
              Remaining Estimate - 40m
              40m
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development