Tuesday, February 17, 2015

Twitter Sentiment Analysis

Sentiment Analysis in simple words is just reading between the lines of text, a very common technique you use when you read reviews about movies, restaurants etc. to make a choice. This technique is now being highly used by the organizations for pervasive analysis, customer profiling and accurate market campaigning.

While everyone was curiously waiting for the Delhi Elections 2015 results,  during our casual discussion, me and my friend  Harshit Pandey  decided to find out the Tweet trends for the competing parties. We chose Twitter Streaming API as the source for our analysis, Mongo DB for archiving the Tweets, Python for performing sentiment analysis and Tableau Public for visualization.

Here's how it looks like, this dashboard shows the popular hashtags used by the users as well as the trends of the positive negative tweets (specific to each party) before and after the election results were announced.


Algorithm Used: 


To perform the contextual analysis for each party within a tweet we wrote a custom method using decision trees for generating score for the two competing parties. We created a dictionary of biased hashtags and searched for their occurrence in the tweets.  In addition to it we tokenized the tweet text  based on the party name and searched for the occurrence of nearest positive or negative words and assigned the score accordingly. The code can be accessed from the Github repository.

4 comments :

  1. Hi, Nice article, I wanted to learn more about sentiment analysis, can you suggest where and how to start?

    ReplyDelete
  2. Hi Archer, Thanks for visiting! I would recommend watching the Coursera course https://class.coursera.org/nlp/lecture as it covers all the aspects of language processing to start with.

    ReplyDelete
  3. Hi Uday,

    Thanks for the reply, So do you work in this domain, or this is you hobby project. If you regularly blog about these topics then I am interested in reading about your thoughts.

    ReplyDelete
  4. Thanks for following! Yes, I do work in this domain. However, I did this project apart from my routine work and I will be publishing more articles shortly on the similar lines.

    ReplyDelete