TEXT SENTIMENT ANALYSIS BASED ON CNNS AND SVM

The traditional text sentiment analysis method is mainly based on machine learning. However, its dependence on emotion dictionary construction and artificial design and extraction features makes the generalization ability limited. In contrast, depth models have more powerful expressive power, and can learn complex mapping functions from data to affective semantics better. In this paper, a Convolution Neural Networks (CNNs) model combined with SVM text sentiment analysis is proposed. The experimental results show that the proposed method improves the accuracy of text sentiment classification effectively compared with traditional CNN, and confirms the effectiveness of sentiment analysis based on CNNs and SVM


Introduction
In

Research on Emotion Classification
The development of the Internet makes the social network and e-commerce platform produce a large amount of text data, which contains valuable information which needs effective processing methods to mine. Therefore, Natural Language Processing as an important direction of artificial intelligence field, acquire more and more attention by the academia and industry. Different from image and speech, text has its own special features in many aspects. For example, the text data contains human higher-level emotional characteristics, resulting in ambiguity and polysemy, makes Natural Language Processing encountered more difficulties in the process of research. Sentiment analysis, also known as comment mining, uses Natural Language Processing, text analysis, machine learning, computer linguistics and other methods to analyze, process, reason and induce texts with emotional colors. Emotion classification is the core problem of sentiment analysis technology, whose goal is to judge the sentiment tendency in the review. According to the method of learning, emotion classification can be divided into supervised, unsupervised and semi supervised methods. Supervised learning is mainly realized by machine learning. Pang [2] who used simple Bias, maximum entropy and support vector machine (SVM) methods to classify the movie reviews data, and achieved good results. Ye [3] who applied supervised emotion classification methods to online travel notes, which improved the accuracy and recall rate of the returned travel notes of the search engines, and the unsupervised method had no annotation information, and classify the emotion mainly through the prior knowledge. Turney [4] proposed a sentiment classification method combining part-of-speech and grammatical structure, which is mainly used in the comment dataset. Zagibalov [5] who implemented an unsupervised sentiment classification method by automatically selecting seed words, and applied it to Chinese product reviews data sets. The semi-supervised method is suitable for data sets with only a small amount of annotations. Dasgupta [6] and other first used spectral analysis techniques to mine clear and unambiguous comments, and then classified the ambiguous comments with active learning and transudative learning. The semi-supervised sentiment classification model proposed by Li [7] gives full consideration to the unbalanced number of topics in the dataset.

Research on Deep Learning
Deep learning is a new field of machine learning research in recent years. It uses multi-layer neural network structure to simulate the human brain to analyze a large number of data and extract effective features. The features extracted from deep learning can be regarded as an abstract representation of the original data on a higher level, and it is very suitable for solving some abstract recognition tasks. Since its birth, deep learning has made many outstanding achievements in many fields such as computer vision, speech recognition and Natural Language Processing. Hinton [8] who first proposed the concept of deep learning, and then used a deep learning network to construct a high quality language model to deal with natural language problems. LeCun [9] who used CNNs for the first time to solve the problem of handwriting recognition, and the concept of CNNs was proposed. In 2010, Mikolov [10,11]  emotion analysis. The RecNN model learns the eigenvectors of variable length by recursive computation; Liang Jun [15] who used deep learning to do Chinese text micro-blog sentiment analysis. They tried to use LSTM recurrent neural network to discover features, introduce emotional polarity transfer model according to the relevance of the words before and after to do emotional analysis, and achieved good results. Compared with the traditional machine learning method, the expression ability of deep neural network has a qualitative leap, and get rid of the shackles of feature engineering. Deep neural network uses semantic synthesis of high-level text sentiment semantic feature vector, so as to get high-level emotional semantic expression of text, effectively improve the generalization ability of the model.

Sentiment Classification Model Based on CNN's and SVM's
In recent years, with the further development of natural language research, the traditional feature extraction method based on machine learning has been unable to meet the current demand, some scholars tried to use the method of deep learning to solve some problems in Natural Language Processing and achieved good results. For this reason, we also use deep learning model to deal with text sentiment classification problem.

Brief Introduction of Word2vec
The first step of translating natural language understanding problem into machine learning problem is to find a way to make these symbols mathematically. In the past, word expression is used by one-hot representation, which expresses a word with a very long vector, the dimensionality of the vector is the size of the D dictionary, a vector only has one dimension values 1, the remaining dimensions are all indexed in D by the position of 0,1. But the expression of the word vector is ea similarity between words and words. Word2vec is a tool implementation of a training word vector model introduced by Google, which is made up of a simple three layers neural network. It uses the distributed representation proposed by Hinton [16], which can overcome the shortcomings of onehot representation. The basic idea is: By the language model training, each word in a language is mapped into a fixed-length short vector, all of these vectors constitute a word vector space, each vector is regarded as a point in space, grammar, words is similar in grammar and semantic, so are their distance. In Word2vec, CBOW and Skip-gram are proposed [17]. The former trains the current word expression through contextual word expression, while the latter, on the contrary, predicts the probability of the context of the word by the given word. Word expression obtained from the Word2vec model training fully considers the semantic information of words, showing that words similar in semantic of expression distance are smaller, and reflects the situations of polysemy. Word2vec creates features without the need for manual intervention, including the context features of the word. These contexts come from the windows of multiple words. If there is enough data, usage, and context, word2vec can accurately predict the meaning of a word based on the appearance of the word.
Although word2vec itself does not achieve deep learning, word2vec turns text into a vector form that can be understood by deep learning. Firstly, CNNs calculate the convolution of the input layer and output, equivalent to the local connection. Each local input area connects one output neuron. And then CNNs applies different filter to each layer, learns the weight of the filter automatically based on tasks that need to be done, drop the sampling by pooling layer to summarizes the results, and build a multilayer the network layer processing mechanism to imitate human brain perception of visual signals, in order to realize the automatic extraction and recognition of visual signals. With the successful application of convolution neural network in the field of computer vision, researchers have begun to apply it to other fields. Kim [18] applied the idea of CNN to text classification, and implemented CNN-based text classification model. Experimental results show that CNN-based text classification method is more accurate than the best method at that time. Thanks to Johnson [19] and Zhang [20] et al , CNN-based text classification methods have been improved in special scenarios. Kalchbrenner [21] et al introduced a dynamic convolution neural network (dynamic convolution neural network) for English sentence modeling in 2014.

Convolutional Neural Network
As one of the deep learning models, CNNs is the first supervised learning algorithm to successfully train multilayer network structure. It uses spatial relative relation to reduce the number of parameters to improve training performance. Its essence is multi-layer convolution. In the traditional feedforward neural network, each input neuron corresponds to the next layer of output neurons connected to the next layer, which is called the full connection layer or the affine layer. The convol full text feature vector can be acquired through K -max pooling operation, so as to get rid of the dependence of decision tree method, and a good text classification effect is shown in the evaluation of English problem classification. The text classification model based on CNN, as shown in Figure 3 -1, is composed of the input layer, the convolution layer, the pool layer and the output layer. ution neural network uses a series of convolution layer, pool layer and a fully connected output layer to construct the model. The last layer of text sentiment classification model based on CNNs is the full connection layer. The input of this layer is the feature vector outputted from the pool layer, and the output is the probability distribution of the samples on each category label, and the emotion of the input sentence is classified according to the probability size.

Text Emotion Classification Model Based on CNNs and SVM
Convolution neural network can extract meaningful feature representation from input samples effectively, but the classification ability of fully connected classification layer is weak for nonlinear separable data. SVM is a supervised machine learning model, which is a two-[0] classification model. The SVM method is based on the theory of the VC dimension of statistical [1] learning theory and the principle of minimum structural risk. According to the limited data [1]  information, SVM try to find the best compromise between the complexity and the learning ability, in order to get the best generalization ability. Among them, the learning ability refers to [0] the ability to detect any error-free samples, and the complexity of the model refers to the learning that the model can get from a specific training sample. SVM represents the data [0] feature vectors in the feature space, the support vector refers to those samples from the training data that are closest to the classified hyper plane. For linear no separable data, SVM [0] can map the data into a high dimensional space through a kernel function, and then transform the linear no separable problem into a linearly separable problem. That is, CNN's is good at [1] learning the characteristics of the invariance, and SVM can find the optimal classification surface for the characteristics. Combined with the above factors, this paper combines CNNs and SVM to deal with text emotional analysis.
Because the output vectors of pooling layer of convolution neural network are represented by distributed features of input samples, the distributed feature representation can be used as feature input in support vector machine. Therefore, CNNs can be used as an automatic feature learner, and SVM is used as an emotional classifier, and the two can be combined to deal with the problem of text emotional analysis.

Experiment Setting
The experimental data used in this paper are NLPCC2014 emotional analysis evaluation task data set based on deep learning technology, which is designed to evaluate the performance of sentiment analysis technology based on deep learning. There is 10000 of the training data, including 5000 positive and 5000 negative emotional polarity data, 2500 test data, including 1250 positive and 1250 negative emotional polarity data. To verify the effectiveness of the proposed method, we choose the traditional CNN-based text sentiment analysis method and the NLPCC-SCDL-best method (the best system in NLPCC-SCDL evaluation task) to make a comparative experiment. First, the experimental data are processed, the word segmentation is filtered to denoise, and then the filtered data set is trained by Word2vec tool to become the 300 Dimension emotional feature vector.

Experimental Results and Analysis
The evaluation criteria of emotional analysis task based on deep learning technology in NLPCC2014 is regarded as the evaluation index of experimental results, according to the evaluation standard, calculate the accuracy rate, recall rate and F value. The results of the experiment are shown in Table 1. According to table 1, the results show that in the emotion classification task, the accuracy of using the CNN-SVM model is much higher than that of CNN and NLPCC-SCDL-best models, which proves that the model is feasible in dealing with text sentiment classification.

Concluding Remarks
Sentiment analysis is a research hotspot in Natural Language Processing. This paper [0] combines the advantages of CNN's and SVM and constructs a text sentiment analysis model-based on CNN's and SVM. The trained word vector is used as input, and CNN's is used as [0]an automatic feature learner, and SVM is the final text classifier. The text affective analysis [7] model based on CNN's and SVM proposed in this paper can effectively improve the performance of text classification.