2016 - Detecting sarcasm in customer tweets: an NLP based approach…
2016 - Detecting sarcasm in customer
tweets: an NLP based approach
Pradip Kumar Bala
This paper uses novel features to identify sarcasm in online text which is one of the most challenging problems in natural language processing
In spite of its success, under certain circumstances sentiment analysis can be gravely inadequate. One such situation is when the sentences are laden with sarcasm, for example, sarcastic user tweets.
It is quite possible that a sarcastic tweet, which mockingly praises a product while actually deriding it, be classified as positive customer emotion.
Sarcasm, being a special type of communication, where the explicit meaning differs from the implicit one, cannot be effectively identified with conventional data mining techniques such as sentiment analysis (Yee Liau and Pei Tan, 2014).
The ambiguous nature of sarcasm makes it difficult even for humans to detect it in sentences. Despite the difficulties, the huge benefit of detecting sarcasm has been recognized in many computer interaction-based applications, such as review summarization, dialogue systems and review ranking systems (Davidov et al., 2010).
In the case of microblogs, such as Twitter, messages can be annotated with hashtags that are an indication of the sentiment being expressed in tweets.
These hashtags are reliable indicators of the emotion being expressed by the tweets, as the author explicitly conveys the emotion of the tweet through them (e.g. #happy, #joy, #sad).
We utilized this behavior to formulate hashtags (#sarcasm, #sarcastic) for our data set. We considered the sentences
that end in #sarcasm or #sarcastic to be the gold standard for sarcastic sentences
We trained our classifiers on multiple different feature types
The words that have little or no significant meaning outside the premise of the sentence.
part of speech n-grams
Content words + function words
We have also extracted both these feature types. together and used as a single feature type to capture both style- and topic-based features.
Function words +
part of speech
We have combined function words and part of speech n-grams and used them as a single feature for classification. These features exclusively capture style-based features
Content words + function words +
part of speech n-grams
Using this feature type we have tried to capture both style-based as well as topic-based features.
Part of speech tags
We hypothesize that sarcasm in a sentence is dependent not only on the content words of the sentence but also on the authorial or writing style of the author, which are best depicted by function words and parts of speech of the sentences (Argamon et al., 2003; Koppel, 2002).
We have used a data set of 5,000 tweets consisting of sarcastic and non-sarcastic tweets
In this paper, we have proposed that for effective identification of sarcasm, both the content as well as writing style of the author plays a crucial role. Through our classification algorithms, we have identified a set of features that capture authorial style.
We downloaded around 15,000 tweets using hashtags such as #sarcasm, #sarcastic along with sincere tweets using R software
To train and test the classifiers, the data were split into two sets randomly. The data set was divided into a ratio of 3:1. The mentioned ratio has been extensively applied in classification literature (Schürer and Muskal, 2013).
A tenfold cross-validation was performed on the training set. In choosing the training testing ratio, the stress is on generalizability of the results which is achieved by the K-fold cross-validation as explained later in this section (Domingos, 2012)
One needs to ensure that the training data does not overfit the training set as it could drastically distort the result for the test set. This is usually addressed by the K-fold cross-validation.
Type of classficiation in NLP
Learn the joint probability of the inputs and the labels (classes like in our case sarcasm/non-sarcasm), and make the prediction by using the Bayes rule to select the most likely label
Model the posterior probability directly or learn a direct map of inputs to the class label (Ng and Jordan, 2002)
Naïve Bayesian Classifier
We considered a document vector model (Manning and Schutze, 1999) for representing a document with the help of terms which can be used as inputs.
Maximum entropy classifier
Unlike the Naïve Bayes classifier, the maximum entropy classifier does not assume that the features are conditionally independent of each other. Maximum entropy is therefore a less restrictive model than Naïve Bayesian model
It is based on the principle of maximum entropy and from all the models which fit the training data, it selects the one which has the highest entropy.
The maximum entropy classifier requires more time to train compared to Naïve Bayes due to the optimization problem that needs to be solved in order to estimate the parameters of the model.
We have formulated both the types of classification models, the Naïve Bayes model (generative classifier) and the maximum entropy model (discriminative classifier).
Authorial style-based features
Best captured by the function words and the part of speech used in the sentence (Argamon et al., 2003).
Naïve Bayes and maximum entropy show that function words and content words together are most effective in identifying sarcasm in tweets.
The most differentiating features between a sarcastic and a non-sarcastic tweet were identified
Features that are independent of the text lead to an increase in the sarcasm detection accuracy.
We also found that Naïve Bayes classifier performed relatively better than the maximum entropy classifier in differentiating sarcastic tweets from the non-sarcastic ones.
Among the authorial style-based features, we observed that function words performed better than part of speech tags and part of speech n-grams.
Study sarcasm in online text – specifically on twitter – to better
understand customer opinions about social issues, products, services, etc
The first one is to identify characteristics in sentences which can be used for detecting sarcasm in tweets
Difficult to detect sarcasm
This makes sarcasm detection from unstructured text data a relevant and challenging problem. This is also because it is unaided by any visual or vocal cues that assist humans in understanding sarcasm.
One of the major issues in sarcasm detection is the absence of naturally occurring expressions that can be used for training purposes (Davidov et al., 2010).