Research on text summarization classification based on crowdfunding projects

. In recent years, artificial intelligence technologies represented by deep learning and natural language processing have made huge breakthroughs and have begun to emerge in the field of crowdfunding project analysis. Natural language processing technology enables machines to understand and analyze the text of crowdfunding projects, and classify them based on the summary description of the project, which can help companies and individuals improve the project pass rate, so it has received widespread attention. However, most of the current researches are mostly applied to topic modeling of project texts. Few studies have proposed effective solutions for classification prediction based on abstracts of crowdfunding projects. Therefore, this paper proposes a sequence-enhanced capsule network model for this problem. Specifically, based on the work of the capsule network, we propose to connect BiGRU and CapsNet in order to achieve the effect of considering both the sequence semantic information and spatial location information of the text. We apply the proposed method to the kickstarter-NLP dataset, and the experimental results prove that our model has a good classification effect in this case.


Introduction
Crowdfunding is a new model of Internet finance. Its operating principle is that the promoters openly raise capital from the public through the Internet platform to support the development of the project, and provide investors with equity and in-kind returns after the project is finally successful. Crowdfunding provides new financing channels and marketing methods for individual entrepreneurs and start-ups. The crowdfunding market has huge potential. Kickstarter, the world's largest crowdfunding platform, has raised $1.3 billion for more than 70,000 projects, with more than 7 million investors participating. Regrettably, more than 50% of projects still fail to raise funds [1]. The effect of crowdfunding financing determines the rise and fall of crowdfunding platforms. Crowdfunding behavior is largely determined by investors' subjective factors, and an important factor that affects subjective judgment is the persuasiveness of language. This is a typical user-generated content (UGC), and the project initiator can describe the project in any type of language style. Different language styles will change investors' perception of project prospects, and in turn affect their willingness to invest.
On the Internet, users often face information overload. In the crowdfunding community, there are a large number of projects going online every day, and investors cannot read the detailed text of each project carefully, and will only selectively browse the introduction of some projects. The text introduction of the crowdfunding project provides a function similar to the abstract of an academic paper. When reading academic papers, readers generally choose to read the abstract first, and detailed research papers only if they can arouse interest [2,3]. The project summary has attracted the attention of investors for its short and conciseness, and it has also been valued by the crowdfunding platform. In crowdfunding projects, investors will also read the introduction of the crowdfunding project first, and then continue to research the project and participate in investment if they are interested. In practice, project sponsors use different forms to describe and promote projects. In view of the important role of the project summary, it is worth studying according to its persuasiveness [4,5].
This article uses text mining methods to predict the success of crowdfunding project abstracts, hoping to help entrepreneurs increase their success rate in the near future [6,7]. This paper proposes a sequence-enhanced capsule network model for modeling. The proposed method is evaluated on a large public data set containing 215,513 projects on the Kickstarter platform. Our contributions can be summarized as follows: 1.A method of predicting success based on the summary of crowdfunding projects is proposed to assist entrepreneurs to improve the success rate of projects.
2.A sequence-enhanced capsule network model is proposed. We propose to connect BiGRU and CapsNet in order to achieve the effect of considering the sequence semantic information and spatial location information of texts at the same time, achieving better feature extraction results, and improving prediction performance.

Method
In this section, we first introduce our distributed feature extraction method in Section 2.1. Next, a sequence-enhanced capsule network model is introduced in Section 2.2. The model frame is shown in Fig. 1.

Distributed feature extraction method
Text feature representation is very important for text classification, and it directly affects the classification results. Traditional text representation still stays at the surface processing of vocabulary. For example, the Vector Space Model (VSM) expresses text data through a vector composed of feature words and weights, and the Bag-of-Words model (Bag-of-Words) , BoW) takes text as a collection of several words and the words are independent of each other. Traditional artificial feature extraction methods ignore the contextual information of the text and cannot express its relevance, which has a great impact on the semantic expression of the text. Some researchers map the original high-dimensional feature word space to a low-dimensional latent semantic space or topic space to explore the potential semantic structure of the text. For example, Latent Semantic Analysis (LSA) represents text as a semantic vector in a low-dimensional latent semantic space [8], which improves its sparsity while reducing dimensionality and denoising. However, the process of dimensionality reduction may cause classification problems. Loss problem, and the semantics of each dimension in the semantic space is not clear. The Latent Dirichlet Allocation (LDA) model expresses the text as a probability distribution of implicit topics [9], but its effect on the sparsity of short texts is limited. Literature [10] proposed the Word2Vec model, which can train the word vectors in the corpus quickly and efficiently. Compared with the LSA and LDA models, the Word2Vec model can effectively use contextual semantic information. While solving the problem of dimensionality, it will not ignore The relative position of phrases in the corpus retains the semantic relevance of phrases.
Word2Vec includes Skip-Gram and Continuous Bag-of-Words Model (CBOW) two models, which can quickly train word vectors according to the given corpus. This paper maps the data set text to a 256-dimensional word vector matrix. The word vector matrix is used as the input of the model.

Distributed Feature Extraction Method
In this section, the capsule network (capsule) [11]and the bidirectional gated recurrent unit network (BiGRU) [12] are combined, the capsule network is used to extract the local features of the text, and the BiGRU network is used to extract the global features of the text. The two levels of the extracted Feature fusion obtains multi-level features of the text.
The bidirectional gated recurrent unit network (BiGRU) is a bidirectional gated-based recurrent neural network composed of a combination of forward GRU and backward GRU. The text is traversed through the network in two directions to obtain information containing the text context, which solves the problem that the GRU model can only contain the above information. The GRU model is a variant of the long short-term memory network (LSTM). The gated loop unit GRU is the 3 gates of LSTM [13] reduced to 2, which are respectively the update gates t z reset gate t r .
( ) The update gate is used to control the degree to which the state information at the previous moment is substituted into the current state. The larger the update gate value, the more state information is substituted at the previous moment; the reset gate is used to control the degree of discarding the information of the previous state, and the reset gate the smaller the value, the more information is discarded. The final hidden layer state t h and t h sum be calculated as: Expression t x is the output at time t equals, -1 t h is the output of the hidden layer at the previous time. t W and h U is the weight matrix of the forget gate, t b is offset term the forget gate,σ is Activation function called sigmoid. Compared with LSTM, the GRU model is simple, has fewer parameters, and has high training efficiency. Therefore, this article chooses GRU as a tool for sequence information learning. The Bi-GRU model used in this article consists of a forward-propagating GRU and a backward-propagating GRU. The BiGRU module can fully input the relevant information before and after the sequence, extract the text information from two directions, and obtain the global features of the text vector.
In the capsule network, the convolution operation is first performed, and the capsule convolution operation is performed through the main capsule layer. After the squeeze function operation, it is used as the output of the main capsule layer. After the dynamic routing protocol mechanism is calculated, it is connected to the classification capsule layer to classify the capsule. capsules share their weights with each other. The low-level feature information obtained through the initial capsule layer is aggregated into the high-level capsule layer (TextCaps) with an 8-dimensional vector through dynamic routing. Dynamic routing performs 3 iterations of important spatial hierarchical relationships between PrimaryCaps and TextCaps. The corresponding dynamic routing algorithm is given in Table  1, and the method used to calculate the coupling coefficient is described. The output result of the layer is expanded as a local feature vector of the text. As the input of the fully connected network, the last layer of the fully connected network uses the sigmoid function as the classifier for judgment.

Dataset and Experimental Setup
In order to evaluate the performance of the model on the task of text similarity, this article uses the kickstarter-NLP dataset. Kickstarter was established in New York, USA in April 2009. It is a crowdfunding website platform designed to raise funds for companies with creative solutions. The kickstarter-NLP data set contains 215,513 crowdfunding projects, of which 172411 were successful, with the label set to 1, and 43102 failed, set to 0. In the experiment of this article, the data set is divided, 80% as the training set, 20% as the test set. The experiment carried out in this paper is implemented based on the keras framework.. It uses the Adam optimizer. The number of model iterations is 15. And the Dropout rate is set to 0.5.

Evaluation Index
This article uses accuracy, precision, recall and F1 value (F-measure) as the evaluation indicators of the model. Let TP be the number of correct classes predicted as correct; TN be the number of wrong classes predicted as wrong; FP be the number of correct classes predicted as correct; FN be the number of correct classes predicted Number. The calculation formula of the evaluation index is as follows.  Table 2. It is clear that our BiGRU-Capsule model consistently performs better than all baselines on our kickstarter-NLP datases.The structure can more fully extract the information of the text context,and extract spatial location information.This method uses dynamic routing methods to store and cluster features, and encapsulate the features into a set of sentence-level semantic capsules. The similarity between the input of the lower-level capsule and the output of the higher-level capsule is obtained by the dot product algorithm, so the lower-level capsule will send its output to the similar high-level capsule, and the capsule in the classified capsule layer contains the highest The characteristics of the order, each capsule represents the category that the text may belong to, so as to achieve a higher accuracy. As shown in Fig.2. we also compared the performance under different batches,it is clear that our method performs better when its batch is 128. Simultaneously BiGRU-Capsule performs better than its branch structure.

Conclusion
This paper proposes a sequence-enhanced capsule network model for text classification tasks based on crowdfunding project summaries. Based on the work of the capsule network, considering the influence of the sequence semantic information and spatial location information of the text, it is recommended to connect BiGRU and CapsNet in order. This strategy achieves good results and is superior to other compared models. This work will increase the pass rate of crowdfunding projects for entrepreneurs in the near future.