Details

    • Type: Task
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: documentation
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This JIRA addresses issues in the Streaming doc that also require a cross-link to and update in the M/R Tutorial doc. All changes approved by the reviewing engineer.

      Streaming issues:

      1. During the execution of a streaming job, the names of the "mapred" parameters are transformed. The dots ( . ) become underscores ( _ ).

      Docs affected: streaming and m/r tutorial (new sub-sections added under Task Execution & Environment section)

      2. For -files and -archives options, Hadoop now creates symlink with same name as file (user-defined symlinks, #mysymlink, currently not supported)

      Docs affected:streaming

      3. Streaming supports streaming command options and generic command options. Generic options must be placed before streaming options, otherwise command fails.

      Docs affected: streaming (reorganized the streaming doc to make distinctions between 2 sets of command options more clear)

      1. ASF.LICENSE.NOT.GRANTED--patch-813-0.20.txt
        46 kB
        Amareshwari Sriramadasu
      2. mapred_tutorial.pdf
        159 kB
        Corinne Chandel
      3. MAPREDUCE-813.patch
        44 kB
        Corinne Chandel
      4. MAPREDUCE-813-2.patch
        46 kB
        Corinne Chandel
      5. MAPREDUCE-813-3.patch
        67 kB
        Devaraj Das
      6. patch-813-ydist.txt
        47 kB
        Amareshwari Sriramadasu
      7. streaming.pdf
        62 kB
        Corinne Chandel

        Issue Links

          Activity

          Hide
          chandec Corinne Chandel added a comment -

          1. mapred_tutorial.pdf - shows doc changes

          2. streaming.pdf - shows doc changes

          3. MAPREDUCE-813.patch - patch file

          Show
          chandec Corinne Chandel added a comment - 1. mapred_tutorial.pdf - shows doc changes 2. streaming.pdf - shows doc changes 3. MAPREDUCE-813 .patch - patch file
          Hide
          chandec Corinne Chandel added a comment -

          Apply this patch to the hadoop-m/r branch: https://svn.apache.org/repos/asf/hadoop/mapreduce/trunk

          Note: No new test code required; changes to documentation only.

          Show
          chandec Corinne Chandel added a comment - Apply this patch to the hadoop-m/r branch: https://svn.apache.org/repos/asf/hadoop/mapreduce/trunk Note: No new test code required; changes to documentation only.
          Hide
          amareshwari Amareshwari Sriramadasu added a comment -

          Some minor comments:
          1. mapred_tutorial.xml has some tabs instead of spaces.
          2.

           <p>
          
          -Hadoop streaming is a utility that comes with the Hadoop distribution. The utility allows you to create and run Map/Reduce jobs with any executable or script as the mapper and/or the reducer. For example:
          
          +Hadoop streaming is a utility that comes with the Hadoop distribution. The utility allows you to create and run map/reduce jobs with any executable or 
          
          +script as the mapper and/or the reducer. For example:
          
           </p>
          
          

          The documentation change here is the change from Map/Reduce to map/reduce. But HADOOP-3691 made sure in all the places, it should be Map/Reduce. I see the same change in many places. Was this intentional?

          Show
          amareshwari Amareshwari Sriramadasu added a comment - Some minor comments: 1. mapred_tutorial.xml has some tabs instead of spaces. 2. <p> -Hadoop streaming is a utility that comes with the Hadoop distribution. The utility allows you to create and run Map/Reduce jobs with any executable or script as the mapper and/or the reducer. For example: +Hadoop streaming is a utility that comes with the Hadoop distribution. The utility allows you to create and run map/reduce jobs with any executable or +script as the mapper and/or the reducer. For example: </p> The documentation change here is the change from Map/Reduce to map/reduce . But HADOOP-3691 made sure in all the places, it should be Map/Reduce. I see the same change in many places. Was this intentional?
          Hide
          chandec Corinne Chandel added a comment -

          Patch file #2.

          (1) replaced map/reduce with MAP/REDUCE (m/r tutorial doc, streaming doc)

          (2) removed tabs (m/r tutorial doc)

          Show
          chandec Corinne Chandel added a comment - Patch file #2. (1) replaced map/reduce with MAP/REDUCE (m/r tutorial doc, streaming doc) (2) removed tabs (m/r tutorial doc)
          Hide
          amareshwari Amareshwari Sriramadasu added a comment -

          Document changes look good. But some minor comments on indentation :
          1. mapred_tutorial.xml has changes like

          -  
          +
          
          and 
          -              <code>package org.myorg;</code>
          +                <code>package org.myorg;</code>
          
          

          etc.

          This are not necessary.

          {code + <section> + <title>Configured Parameters</title> {code}

          More spaces for title are not necessary. (Not there for other sections).

          2. In steaming.xml,

          -Hadoop streaming is a utility that comes with the Hadoop distribution. The utility allows you to create and run Map/Reduce jobs with any executable or script as the mapper and/or the reducer. For example:
          
          +Hadoop streaming is a utility that comes with the Hadoop distribution. The utility allows you to create and run Map/Reduce jobs with any executable or 
          
          +script as the mapper and/or the reducer. For example:
          

          Change here is unnecessary. If you want, you can make the lines fit into 80 character lines.
          The same applies to similar changes in the doc.

          Show
          amareshwari Amareshwari Sriramadasu added a comment - Document changes look good. But some minor comments on indentation : 1. mapred_tutorial.xml has changes like - + and - <code> package org.myorg;</code> + <code> package org.myorg;</code> etc. This are not necessary. {code + <section> + <title>Configured Parameters</title> {code} More spaces for title are not necessary. (Not there for other sections). 2. In steaming.xml, -Hadoop streaming is a utility that comes with the Hadoop distribution. The utility allows you to create and run Map/Reduce jobs with any executable or script as the mapper and/or the reducer. For example: +Hadoop streaming is a utility that comes with the Hadoop distribution. The utility allows you to create and run Map/Reduce jobs with any executable or +script as the mapper and/or the reducer. For example: Change here is unnecessary. If you want, you can make the lines fit into 80 character lines. The same applies to similar changes in the doc.
          Hide
          chandec Corinne Chandel added a comment -

          I'm not cleaning up the indentation on the XML files.
          If there are not other content changes, then we can go with MAPREDUCE-813-2.patch.

          Thanks/C

          Show
          chandec Corinne Chandel added a comment - I'm not cleaning up the indentation on the XML files. If there are not other content changes, then we can go with MAPREDUCE-813 -2.patch. Thanks/C
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12415063/MAPREDUCE-813-2.patch
          against trunk revision 800329.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 6 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/438/console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415063/MAPREDUCE-813-2.patch against trunk revision 800329. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/438/console This message is automatically generated.
          Hide
          devaraj Devaraj Das added a comment -

          For some reason, the 813-2.patch doesn't apply cleanly on trunk. Attaching a patch that applies cleanly on trunk

          Show
          devaraj Devaraj Das added a comment - For some reason, the 813-2.patch doesn't apply cleanly on trunk. Attaching a patch that applies cleanly on trunk
          Hide
          amareshwari Amareshwari Sriramadasu added a comment -

          I ran forrest on my machine with patch. It is Successful.

          Show
          amareshwari Amareshwari Sriramadasu added a comment - I ran forrest on my machine with patch. It is Successful.
          Hide
          devaraj Devaraj Das added a comment -

          I just committed this. Thanks, Corinne!

          Show
          devaraj Devaraj Das added a comment - I just committed this. Thanks, Corinne!
          Hide
          hudson Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #46 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/46/)

          Show
          hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #46 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/46/ )
          Hide
          amareshwari Amareshwari Sriramadasu added a comment -

          Patch for Yahoo! distribution.
          Ran ant docs and it was successful.

          Show
          amareshwari Amareshwari Sriramadasu added a comment - Patch for Yahoo! distribution. Ran ant docs and it was successful.
          Hide
          amareshwari Amareshwari Sriramadasu added a comment -

          Patch for branch 0.20.

          Show
          amareshwari Amareshwari Sriramadasu added a comment - Patch for branch 0.20.
          Hide
          amareshwari Amareshwari Sriramadasu added a comment -

          I ran ant docs with the patch for 0.20 : https://issues.apache.org/jira/secure/attachment/12441581/patch-813-0.20.txt. It ran successfully. Can somebody please commit the patch to branch 0.20?

          Show
          amareshwari Amareshwari Sriramadasu added a comment - I ran ant docs with the patch for 0.20 : https://issues.apache.org/jira/secure/attachment/12441581/patch-813-0.20.txt . It ran successfully. Can somebody please commit the patch to branch 0.20?

            People

            • Assignee:
              Unassigned
              Reporter:
              chandec Corinne Chandel
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development