key: cord-0068779-nez3x1qg authors: Yu, Xiujin; Liu, Shengfu; Zhang, Hui title: Chinese Language Feature Analysis Based on Multilayer Self-Organizing Neural Network and Data Mining Techniques date: 2021-10-14 journal: Comput Intell Neurosci DOI: 10.1155/2021/4105784 sha: de5f5c33dc54a18349202e8a0ddaf601b6fdb719 doc_id: 68779 cord_uid: nez3x1qg As one of the oldest languages in the world, Chinese has a long cultural history and unique language charm. The multilayer self-organizing neural network and data mining techniques have been widely used and can achieve high-precision prediction in different fields. However, they are hardly applied to Chinese language feature analysis. In order to accurately analyze the characteristics of Chinese language, this paper uses the multilayer self-organizing neural network and the corresponding data mining technology for feature recognition and then compared it with other different types of neural network algorithms. The results show that the multilayer self-organizing neural network can make the accuracy, recall, and F1 score of feature recognition reach 68.69%, 80.21%, and 70.19%, respectively, when there are many samples. Under the influence of strong noise, it keeps high efficiency of feature analysis. This shows that the multilayer self-organizing neural network has superior performance and can provide strong support for Chinese language feature analysis. As an independent language, Chinese has remarkable characteristics in pronunciation, grammar, vocabulary, and other aspects. e analysis of the characteristics of Chinese language is helpful to understand its cultural charm and artistic value. To achieve this goal, we need to use the corresponding algorithms to extract Chinese language features accurately. To some extent, Chinese reflects China's structural and cultural characteristics or global model, which is seen through some management departments in the university research model. ese findings show that the strategic development of Chinese characters is different from the global trend, but there is also some selective convergence [1] . At the same time, whether and to what extent the rhetorical structure of English abstracts published in Chinese journals in the fields of accounting and finance is in line with the international common practice of managing the writing of academic abstracts in these two fields. Some studies have identified the typical rhetorical move structures of abstracts in the two corpora and some important differences between the disciplines represented by these corpora [2] . From the perspective of Chinese language teaching, only by combining the Internet plus the background of the times and rationally optimizing the teaching of Chinese, can we better meet the actual needs and promote the progress of students. is paper analyzes the related problems of Chinese teaching optimization under the background of network + era, hoping to be beneficial to reality [3] . Nasir et al. proposed a hybrid model combining language signal and self-organizing mapping. e model can accurately predict the best features in cutting language signal and provides a powerful intelligent model for complex fonts [4] . Self-organizing neural network has been widely used in too many fields. Liu et al. proposed multilayer connected self-organizing feature mapping and related learning methods, which proved that the interlayer connection between winning neurons in each competitive layer has an important influence on the relationship between the internal microstructure and physical parameters of nanowires [5] . Amorim et al. have studied the biological monitoring technology. rough the analysis of artificial neural network, self-organizing and multilayer perceptron, it is proved that this method can take the influence of biosensor as a reference and provide a powerful auxiliary role for water quality monitoring and management [6] . Sultana et al. believed that the computer-aided system can help radiologists diagnose microcalcification patterns. By using the multilayer perceptron neural network, modular neural network, and self-organization feature neural network, breast cancer tumors are classified and compared. e results show that the multilayer self-organizing neural network has better detection effect than other networks [7] . In order to predict the heat transfer parameters of double tube nanofluid heat exchanger, Ghasemi et al. used the improved double tube heat exchanger twisted tape and multilayer perceptron structure backpropagation artificial neural network. It was found that there was a certain relationship between the heat transfer parameters and the input data. e network could better predict the total heat transfer coefficient and pressure drop and other heat transfer parameters [8] . Li et al. proposed a self-organizing radial basis function neural network algorithm for aeroengine thrust estimation, which can effectively determine the connection weights and generate high-precision self-organizing neural network, proving the effectiveness and practicability of the algorithm in aeroengine thrust estimation [9] . Dey et al. has improved the probabilistic neural network classifier based on self-organizing map and the multilayer perceptron classifier based on self-organizing map, compared it with the traditional classifiers, and proved that the algorithm has better classification effect [10] . Sadabad et al. studied the direct biopower generation from waste activated sludge and built the model by using self-organizing feature map and multilayer perceptron. e results showed that the added waste activated sludge content, pH value, and process operation temperature had a significant impact on the power density and removal rate of power generation [11] . Eslamizadeh et al. adopted the improved neighborhood annealing training artificial neural network to effectively complete the detection and recording of real heart sound, which proved the practicability of the algorithm [12] . Nilashi et al. used two types of artificial networks: self-organizing map and multilayer perceptron to investigate the factors that affect the survival rate of entrepreneurs in public support projects. e experimental results have high accuracy, which shows the effectiveness of self-organizing map neural network [13] . To sum up, the multilayer self-organizing neural network and data mining techniques can be widely used and can achieve high-precision prediction in different fields. However, they are hardly applied to Chinese language feature analysis. In view of this, this paper aims at the problem of Chinese language feature analysis, using multilayer self-organizing neural network and data mining to explore, in order to achieve better prediction and analysis effect. In this paper, five sections are included. In Section 1, we introduced the background and importance of Chinese feature analysis and application of multilayer self-organizing neural network and data mining in various fields. In Section 2, we introduced the common multilayer neural network model and its application to Chinese feature analysis. In Section 3, we introduced the multilayer self-organizing neural network and how to apply it to Chinese feature analysis. In Section 4, some experimental examples are tested and experimental results are discussed. In Section 5, we conclude this paper and propose some important findings as well as the future work. Based on Multilayer Neural Network 2.1. Implementation Process of Multilayer Neural Network and Self-Coding. As a widely used method in machine learning, the multilayer neural network can efficiently complete the induction and summary of various data and can effectively solve the core problems in learning [14, 15] . In the network structure of multilayer neural network, the most typical one is a multilayer perceptron with multiple hidden layers, and its model is shown in Figure 1 . As can be seen from Figure 1 , the main function is to effectively decompose a large amount of data, making it a series of maps after nested processing, which are described by different network layers of the network model. e target learning object is initialized through the input layer; then, it is transmitted to the first hidden layer to complete the edge feature processing. After that, it is transmitted to the second hidden layer to learn and train its connection and contour. After the depth analysis of the third hidden layer, that is, the specific object layer, the output layer can output. In this experiment, we propose a trestle self-coding algorithm to analyze the features of Chinese language. e task of feature analysis is regarded as a classification task, and the features of Chinese language are divided into sentence patterns, rhetorical devices, turning words, emotional words, part of speech, negative elements, and so on. e details can be seen in Figure 2 . Figure 2 shows that the Chinese language features are mainly divided into two parts: the raw corpus and the experimental corpus. rough the way of word embedding and feature extraction, the construction of training samples and test samples is completed, and they are taken as the highlevel abstract features of the Chinese language text. en, in the form of fusion, the corresponding word vector and emotional semantic features are processed by the training classifier. e classifier includes the deep trestle self-coding learning model and regression model, which can realize the extraction of the main structural information and semantic features of Chinese language under the action of multimapping unit and finally achieve effective feature classification and extraction. As a powerful means of multilayer neural network, the self-coding learning model can mine the latent meaning of words and effectively reduce the dimension of various parameters. First of all, we need to define a training set with labels (x (i) , y (i) ) m i−1 , as the input data of training network, where x is the sample and x ∈ R n , the y label indicates the corresponding relationship with the sample, m indicates the number of samples, and n represents the sample dimension. e corresponding output result can be expressed as follows: In formula (1), x ∈ x 1 , x 2 , · · · , x n , W T represents the corresponding weight, b represents the offset, and f represents the activation function used in network training, i.e., logic function, as shown in the following equation: According to formula (2), the activation value of the ith neuron in the lth neural network layer can be expressed as a l i , the weight of the layer is W (i) , and the bias is b (i) . In the self-coding model, the weights and offsets can be updated by iterative updating. If the hypothesis function of the network model is h w,b (x) ≈ x, the training process can be represented by the minimum loss function, as shown in the following equation: In equation (3), J(W, b) represents the minimum loss function, λ/2‖W‖ 2 represents weight attenuation term, and λ represents weight attenuation factor. Under the effect of gradient descent algorithm, the weight and offset are updated. Each iteration process can be expressed as equations (4) and (5), respectively: Computational Intelligence and Neuroscience Expressions (4) and (5) represent the learning rate of network training, and their values are usually in the range [0, 1]. Sentence Vector. When analyzing the Chinese language, we need to preprocess the original text and transform it into experimental data. As the original text is usually obtained by web crawler, it often has different types of noise data, such as web page tags and white space characters, so it is very necessary to preprocess them, as shown in Figure 3 . In Figure 3 , the first step of Chinese corpus preprocessing is deduplication and denoising, the main purpose of which is to eliminate duplicate text information or web page tag information; second, sentence recognition is carried out and different sentences are separated by using boundary information such as punctuation marks and saved in training samples. en, part of speech, emotional words, turning words, negative elements, and sentence patterns are annotated in order to provide text data basis for subsequent extraction of Chinese language features. In the analysis of Chinese language features, word vector and sentence vector occupy an important position. Word vector contains rich semantic information and context information, which usually needs to be trained by open-source word vector tool. is tool can simplify the operation process of text processing, transform it into an operation method based on dimensional space vector, and calculate and obtain cosine similarity between different word vectors; it can express the relevance degree of Chinese text semantics. In the opensource word vector tools, there are mainly two models, Continuous Bag-of-Words and Continuous Skip-Gram, and both of them have three network layers: input layer, projection layer, and output layer. If a group of phrase sequence in the corpus is expressed as w 1 , w 2 , w 3 , . . . , w n , the Continuous Skip-Gram model can be expressed as follows: According to equation (6), the main goal of the model is to maximize it, and the maximum function is shown in the following equation: e training of Continuous Skip-Gram model needs to be carried out under the action of hierarchical softmax algorithm; that is to say, the target vocabulary can be expressed in a distributed way, so that the initial value of j in coding is 1, and its root node is expressed as n(w, 1). en, the probability defined by hierarchical softmax algorithm can be calculated by the following equation: In equation (8), if there are two different values, 1 and −1, respectively, then ch(n(w, j)) can represent not only the left child node of n(w, j) but also the right child node. Because the word vectors in the same corpus have similar implicit semantics in each dimension, all the word vectors can be fused to obtain the sentence vectors covering the deep semantic information. If any sentence in the corpus is S and contains words, then there is d words, the vector of words is expressed as S � w 1 , w 2 , . . . , w d and 1 ≤ i ≤ d. e vector of word w is expressed as V(w), and then the vector of dimensional sentence synthesized by different words is shown in the following formula: rough formula (9), we can regard S as a set, which contains a variety of semantic information and context information of the word vector and is consistent with the dimension of the word vector, both of them are N-dimensional vector. erefore, the construction of the sentence vector can be regarded as a process of mapping the sentence to a N-dimensional feature space. Analysis Method Based on Multilayer Self-Organizing Neural Network Neural Network. Artificial Intelligence (AI) is a very hot topic in recent years, and it has been widely applied in a lot of fields [16] [17] [18] [19] [20] [21] [22] . As one of AI techniques, the self-organizing neural network can effectively adjust the weights of the network through self-organizing feature mapping. On this basis, the network can converge to a stable state. e multilayer self-organizing neural network carries out selforganizing learning under unsupervised condition, which makes different neurons sensitive to different input modes. It shows strong objectivity and high accuracy and can stand out in many clustering algorithms with its significant advantages. e SOM neural network does not need to label the training samples in the application process. e accuracy and efficiency of this algorithm are significantly higher than other neural network algorithms. e typical multilayer selforganizing neural network has two network levels of input and output, and its network topology is shown in Figure 4 . In Figure 4 , the input vector in the input layer of multilayer self-organizing neural network has the same dimension as the number of neurons. e neurons in the output layer are arranged in the form of matrix, which can represent the categories of input vectors and have corresponding weight vectors. Multilayer self-organizing neural network clustering mainly includes three different stages, namely, competition stage, cooperation stage, and weight adjustment stage. In the competition stage, the most similar node, the winning neuron, is selected by calculating the similarity between the input vector and the weight vector contained in the output layer. In the cooperation stage, the winning neuron can determine the spatial position of the active neuron in the neural network, that is, the topological domain of the cooperative neuron, and its amplitude decreases with the increase of its own center distance and constantly tends to 0.Weight adjustment is mainly to simulate the excited state of biological cells when they are stimulated, so as to judge their inhibition to the surrounding cells. In view of the phenomenon that most samples are concentrated in a certain region in the sample space, the network training results often show that the node in the local region becomes the winning node, while the node in other remote regions becomes the dead node and the weight around it cannot be adjusted. erefore, the multilayer selforganizing neural network is used to adjust the weight initialization matrix in this experiment. After calculating the average vector in the sample set, a new random number is added to it, and the result is taken as the initial weight vector. e calculation formula is shown as follows: Equation (10) represents the size of the sample space, W ij represents the improved initial weight matrix obtained after calculation. When the weight adjustment is carried out, the learning rate function needs to be adjusted accordingly. Formula (11) is the calculation formula at the initial stage of clustering: In equation (11) , β t1 shows the learning rate obtained at the initial stage of clustering, t represents the number of iterations of weight, β 0 is the initial learning rate, which is usually set to a large number, and can accelerate the convergence speed, and B represents a constant whose function is to promote the continuous decrease of learning rate. e learning rate at the later stage of clustering is calculated by the following equation: Formula (12) can ensure the effective adjustment of weight and the accuracy of clustering results by using linear function. e training process of multilayer self-organizing neural network can use the above content to obtain the winning node, as shown in the following equations: In formula (13) , X i represents the input sample and X I and W J represent the normalized result of the input sample and the weight vector, respectively, and then the winning node is selected through the following formula: e neighborhood range can be determined by formula (14) . Because the winning node can promote and inhibit the neurons which are close to and far away from each other, the neighborhood range can be determined by the following formula: In equation (15), r c and r j , respectively, represent the positions of other nodes and winning nodes in the adjacent region, θ t represents the radius of the initial neighborhood θ 0 after the end of the t th iteration. e gradient descent method combined with equation (12) is used to complete the Computational Intelligence and Neuroscience 5 final update of the learning rate, as shown in the following equation: e neighborhood radius is adjusted by combining with equation (13) , and the neighborhood range is significantly reduced. See equation (17) for details: Self-organizing mapping in the model can form the distribution pattern of discrete approximate input space by applying domain function in the process of continuous iteration [4, [23] [24] [25] . e details are shown in Figure 5 . In Figure 5 , the self-organizing map first randomly selects the corresponding text information for training and usually selects the neurons close to the training data set, that is, the purple part in Figure 5 . In the continuous iterative training, the training data will change accordingly, and its distribution is shown in the green part. Finally, it will be displayed as the current training result, that is, the red area. After all the iterative training, the output layer of the selforganizing will match the distribution of the training data. Self-Organizing Neural Network. According to the above content, we can use the way of combining different word vectors to complete the construction of the feature matrix, namely, the word embedding algorithm. Because the dimensions of different word vectors will increase with the expansion of the scale of the feature matrix, resulting in serious numerical differentiation in the feature matrix, it is usually necessary to reduce the dimension of the feature matrix to ensure that the noise impact in the dataset is eliminated. e clustering process is shown in Figure 6 . According to Figure 6 , the word vector is mapped to lowdimensional space based on the multilayer self-organizing neural network, and this clustering method can effectively achieve dimension reduction. First, the target Chinese language documents are processed by sentence segmentation, word segmentation, and frequency statistics. en, the obtained processing results are transmitted to the next stage, and the features among the word vectors are selected and expressed to obtain the corresponding word vector similarity matrix. Finally, under the function of clustering analysis, the accurate description of clustering results is completed and the effective analysis of Chinese language features is realized. In this experiment, 200 Chinese language and culture works are selected and a complete training sample set is constructed. e multilayer self-organizing neural network is applied to the iterative training process. With the continuous progress of network training, the results of extraction and analysis of Chinese language features by the multilayer selforganizing neural network are more and more clear, as shown in Figure 7 . e results of Chinese language feature analysis mainly include two parts: one is the recognition accuracy of sample features and the other is the loss function value. Figure 7 shows that, with the increasing number of iterative training, the value of loss function shows a decreasing trend. At the beginning of iterative training, the loss function is about 2.25. When the number of training iterations is less than 100, the loss function decreases exponentially. When the number of training iterations is more than 100, the loss function value decreases and gradually maintains a relatively stable state. At the beginning of training, the recognition accuracy of sample features is 0. When the number of training is less than 100, the accuracy rises sharply. With the continuous iterative training, the change range of accuracy rate decreases. By the end of the 2000th iteration training, the recognition accuracy of sample features is almost maintained at about 0.95. On the other hand, the precision and recall of the multilayer self-organizing neural network, as well as the change of F1 score of precision and recall, are shown in Figure 8 . According to Figure 8 , in the process of continuous increase of network training times, the recognition accuracy, recall rate, and F1 score of the multilayer self-organizing neural network for Chinese language feature samples show different degrees of increase. When the number of network training is less than 20, the growth rate of the three is larger and the growth trend is significant, increasing to about 0.65. When the training times reached 50 times, the accuracy rate, recall rate, and F1 score all reached about 0.85, and the change range was small and gradually remained stable. e influence of different feature selection on feature analysis results is shown in Figure 9 . In Figure 9 , a to m denote the number of feature items from less to most. Looking at Figure 9 , we can find that the addition of a single feature will promote the improvement of accuracy and recall to a certain extent. e main reason for this phenomenon is that when the feature set is small, the multilayer self-organizing neural network algorithm cannot fully learn and analyze the sample feature semantics. M is the feature set that includes all the feature items. At this time, the accuracy rate, recall rate, and F1 score all reach the highest values, which are 68.69%, 80.21%, and 70.19%, respectively. In order to comprehensively analyze the application effect of multilayer self-organizing neural network in Chinese language feature analysis, this experiment applies this algorithm and other algorithms to the network training of sample set at the same time, including the backpropagation (BP) neural network and self-generating (SG) neural network. e comparison results are shown in Figure 10 . Figure 10 shows the feature analysis results of multilayer self-organizing neural network, BP neural network, and SG neural network under the influence of different noises. With the increasing influence of noise, the effective analysis rate of BP neural network and SG neural network on Chinese language features shows a significant decline, and the decline trend of SG neural network feature analysis rate is more significant. When the influence degree of noise is enhanced to 80%, the feature analysis rate of the two neural networks is almost zero and they cannot play the role of feature analysis continuously. e feature analysis rate of multilayer self-organizing neural network is always at a high level. When the noise influence is less than 50%, the feature analysis rate is always higher than 95%. When the influence degree of noise is more than 50%, the characteristic analysis rate decreases obviously, but it is still significantly higher than that of BP neural network and SG neural network. is shows that the multilayer self-organizing neural network can realize the analysis of Chinese language features and has high recognition efficiency and accuracy. Chinese includes different words, emotions, and sentence patterns, so it is of great practical significance to extract and analyze its language features. In order to achieve effective Chinese language feature extraction, this experiment will focus on multilayer self-organizing neural network, combine it with the corresponding data mining technology, and apply it to the network training of Chinese language feature sample set. e results show that, with the increasing number of network training, the model constructed by multilayer selforganizing neural network has strong convergence, the recognition accuracy and loss function value are in the normal range, and the former is almost maintained at a high level of 0.95. When multilayer self-organizing neural network is used to extract and analyze Chinese language features, its accuracy, recall rate, and F1 score will continue to rise with the increase of network training times and feature items. When it meets the full feature coverage, the three can reach the highest values, which are 68.69%, 80.21%, and 70.19%, respectively. Under the influence of different noises, the feature analysis ability of multilayer self-organizing neural network is significantly better than other neural network algorithms. is shows that the multilayer selforganizing neural network can play a superior performance in Chinese language feature analysis, showing a strong ability of feature recognition and analysis. In this experiment, the comparative analysis of multilayer self-organizing neural network is less, only compared with the two kinds of neural networks, hoping to achieve more comprehensive and diversified comparisons in the future work. e data used to support the findings of this study are available from the corresponding author upon request. e authors declare that they have no known conflicts of interest or personal relationships that could have appeared to influence the work reported in this paper. Computational Intelligence and Neuroscience SuperSpike: supervised learning in multi-layer spiking neural networks Estimating the scouring depth of bridge pier using self-organizing neural networks A scalable multi-signal approach for the parallelization of self-organizing neural networks Intelligent wood machining monitoring using vibration signals combined with self-organizing maps for automatic feature selection Pattern recognition of messily grown nanowire morphologies applying multi-layer connected self-organized feature maps Escherichia coli's water load affects zebrafish (Danio rerio) behavior Early breast cancer detection utilizing artificial neural network Proposing a method for combining monitored multilayered perceptron (MLP) and self-organizing map (SOM) neural networks in prediction of heat transfer parameters in a double pipe heat exchanger with nanofluid A proposed self-organizing radial basis function network for aero-engine thrust estimation On improving generalization of classifiers using feature extraction Experimental study of direct bio-electricity generation from municipal waste-activated sludge simultaneously with its stabilization and modeling the process by KSOFM and MLP artificial neural networks Heart murmur detection based on wavelet transformation and a synergy between artificial neural network and modified neighbor annealing methods Coronary heart disease diagnosis through self-organizing map and fuzzy support vector machine with incremental updates Corrosion defect modelling of aged pipelines with a feed-forward multi-layer neural network for leak and burst failure estimation Regularizing extreme learning machine by dual locally linear embedding manifold learning for training multi-label neural network classifiers Development and evaluation of an artificial intelligence system for COVID-19 diagnosis Inference in artificial intelligence with deep optics and photonics More reliable predictions of clear-water scour depth at pile groups by robust artificial intelligence techniques while preserving physical consistency Artificial intelligence for sustainability: challenges, opportunities, and a research agenda Drug discovery with explainable artificial intelligence Exploring 3D wave-induced scouring patterns around subsea pipelines with artificial intelligence techniques Human-centered artificial intelligence: reliable, safe & trustworthy A survey on the development of self-organizing maps for unsupervised intrusion detection Remote tracking of Parkinson's disease progression using ensembles of deep belief network and self-organizing map Self-organizing map artificial neural networks and sequential Gaussian simulation technique for mapping potentially toxic element hotspots in polluted mining soils Acknowledgments is work was supported by the Shandong University of Finance and Economics.