Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3388

Streaming task with special char gets wrong output

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Invalid
    • 0.23.0, 2.0.0-alpha
    • None
    • contrib/streaming, mrv2
    • None

    Description

      In 0.20.204:

      hadoop jar $HADOOP_HOME/hadoop-streaming.jar '-Dmapreduce.job.acl-view-job=*' -files 'file:///tmp/InputDir#testlink!@$&*()-_+=' -input input.txt -mapper 'xargs cat' -reducer cat -output output -jobconf mapred.job.name=streamingTest-1000 -jobconf 'mapreduce.job.acl-view-job=*'
      

      Output:

      hadoop fs -cat output/*
      and-so-is-the-c#
      c++-also-supports-object-oriented-concepts
      hadoop-apache-org-core  
      hadoop-streaming
      hod-is-the-part-of-hadoop
      it-is-residing-on-apache-under-repos-asf
      java-an object-oriented-language
      smalltalk-is-also-object-oriented-language
      streaming-is-also-part-of-it
      

      For 0.23:

      hadoop jar $HADOOP_MAPRED_HOME/hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* -files 'file:///tmp/InputDir#testlink!@$&*()-_+=' -input input.txt  -mapper 'xargs cat' -reducer cat -output output -jobconf mapred.job.name=streamingTest-1000 -jobconf 'mapreduce.job.acl-view-job=*'
      

      Output:

      testlink!@$&*()-_+=/input1.txt testlink!@$&*()-_+=/input2.txt
      

      The contents of input.txt are as follows:

      hadoop fs -cat Streaming/streaming-1000/input.txt
      testlink!@$&*()-_+=/input1.txt
      testlink!@$&*()-_+=/input2.txt
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            revans2 Robert Joseph Evans
            revans2 Robert Joseph Evans
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment