[MAPREDUCE-7400] New Map Reduce Example - Simple Sentiment Analysis

• Type: Improvement
• Status: Patch Available
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.4.0
• Component/s: examples
• Labels: pull-request-available
• Target Version/s: 3.5.0
• Tags: mapreduce example
• Flags: Patch
• Assignee: Unassigned
• Reporter: Meetu Patel
• Created: 28/Jul/22 07:33
• Updated: 16/Jan/24 08:39

Description

I am looking to add a new map reduce example, i.e, sentiment analysis. Sentiment analysis map reduce job helps in determining the sentiment score for a user. It takes each tweet made by an user and assigns a sentiment score for that tweet/sentence for a particular user and then aggregates the sentiment scores for all tweets made by all users.

This example takes the twitter dataset which contains users and the tweets made by users and gives the output as <username, sentiment score>. For each user, the sentiment score is calculated for all the tweets made by that particular user.

This mapreduce examples takes in two input files - input twitter dataset and a file containing list of words.
The word list file contains positive, negative and negation words which are used to give a sentiment score to the words in tweets.

You can use command:
bin/hadoop jar /HADOOP_PATH/share/hadoop/mapreduce/mapreduce-examples.jar sentimentanalysis <input file/dir path> <output dir path> <word list file path/dir path>

For example, you can use the sample files and run the above command as:
bin/hadoop jar /HADOOP_PATH/share/hadoop/mapreduce/mapreduce-examples.jar sentimentanalysis sample_data.txt <output dir path> sample_words.txt

Issue Links

• is duplicated by: YARN-11242 New Map Reduce Example - Simple Sentiment Analysis
• links to: GitHub Pull Request #4808