Heterogeneous belief formation method based on BP neural network

. The BDI model has always been the focus of subject modeling research, which includes three kinds of thinking states of the rational subject: Belief, Desire and Intention. Belief is the cognition of agent to the world; it is a collection of environmental information, other agent information, and its own information that the agent has; and it is also the basis of the agent's thinking activity. Due to differences in the individual's living environment and experience, the formation of heterogeneous beliefs is an important issue in the BDI model study. This article divides individual belief set into two parts: knowledge belief and achievable belief. This article proposes an overall framework for the formation of individual heterogeneity beliefs: First, the individual's knowledge experience is modeled, and the empirical knowledge is structured and quantified into binary propositions; then the BP neural network learn and memory propositions of different combinations to form heterogeneous beliefs. Experiments show that this method can simulate the heterogeneity of individual beliefs caused by the individual's own experience, and can realize the belief generation mechanism of gradual information flow, limited attention and heterogeneous priors.


Introduction
The study of agent rationality has two different methods: based on logic and countermeasure decision theory [1]. The logic-based approach generally uses the agent's state of mind model (usually the BDI model). The philosophical foundation of the BDI model is Bratman's analysis of rationality and intention. It consists of the three components of the subject's beliefs-wishes-intentions, and their interrelationships are further elaborated [2,3,4]. The research of BDI model in artificial intelligence focuses on the formal description of BDI. The Agent modeling method does not solve the problems of logic omniscience, inaction, and side effects [5]. In the method based on countermeasure theory and decision theory, the belief is described by the probabilities of different behaviors causing different consequences, which is commonly used in economics. From the concept point of view, the logic method realizes rigorous reasoning, and the decision-making method achieves rational decision by optimizing the subjective utility. From a technical point of view, the logic rationality using symbolic reasoning cannot optimize the utility, and the decision theory using numerical analysis neglects the link of rational reasoning.
Most scholars believe that belief describes the agent's assessment of the current state of the world and the possible course of action to achieve an effect [1]. It is a collection of information about the environment, other agents, and their own information that an agent has. In order to solve the problems of logical omniscience and other issues, Liu Yong and others proposed the idea that "belief includes not only the characteristics of knowledge, but also the characteristics of the goals and aspirations to be achieved." They expanded belief into knowledge belief (BLEK) and achievable belief (BLEA). This distinction and semantic interpretation of the belief concept can avoid the logical omniscience problem and the side effects of logical implications, and there is no problem that the consciousness model does not have a corresponding relationship in practical applications [6]. In summary, the existing research on belief issues adopts abstract and formal formulas for induction, deduction, reasoning and analysis. There is less research on the formation of individual's belief, and the research is not deep enough for the agent to generate different heterogeneous beliefs according to the different living environment and experience.
At present, the study of belief formation is mostly focused on the Agent modeling process in the financial field. Researchers call the different judgments of different investors on the conditional expected return or change in the same holding period of the same stock as heterogeneous beliefs [7,8], and on this basis, the asset pricing problems such as stock [9,10] are extensively studied. Due to the differences in investors' personality characteristics and knowledge backgrounds, investors have great differences in market and investment beliefs, which affect the difference of their investment behaviors, which is the basis of the formation of heterogeneous beliefs [11]. Hong and Stein [12,13] put forward three formation mechanisms of heterogeneous beliefs, namely, gradual information flow, limited attention and heterogeneous priors. The research in the financial field mainly focuses on the use of mathematical probability analysis instead of logical proposition to describe belief. This kind of quantitative belief is easy to use for the research in the financial field. However, the belief description is simple, it can't express the subject's complex decision-making process and it is difficult to use in other fields.
The true state of mind of human beings is formed by the brain through its special neural network structure and continuously predicting the future based on a large number of memories. This is the key to intelligence [14]. In order to study the formation of heterogeneous beliefs in the state of mind, this article defines the individual heterogeneity beliefs as: the different selection and preference of individuals in the same event, resulting from the differences in the learning and memory of the same input information under the different conditions of social experience and knowledge background. In the study of existing heterogeneity beliefs, the logical and formal expression methods, though rigorous, cannot well reflect the subject's choice of behavioral line and probability estimation of the results; the method based on countermeasure theory is simple and easy to understand , but lacks rigorous reasoning similar to the human brain. In order to be able to thinking like a human brain, this study proposes a framework for heterogeneous belief formation based on BP neural networks. Knowledge modeling transforms individual experience knowledge into binary propositions, and uses the classical neural network algorithm to convert propositional learning, memory and inference into individual beliefs and join belief set. By examining the experiments of heterogeneity differences, this paper verifies the feasibility of the method of heterogeneity belief formation and reproduces the three formation mechanisms of heterogeneous beliefs.

Design idea
The formation of individual belief consists of two processes: the formation of knowledge belief and the formation of achievable belief. Each process includes knowledge modeling of individuals and learning, memory, and prediction of belief. The overall design idea is as follows: (1) The formation of belief depends on the experience of heterogeneous individuals and external environment. Individual heterogeneity is reflected in their own experience, the different content and quantity of information obtained, and then leads to the individual will have different views and behavioral decisions. This article will consider three mechanisms for the formation of heterogeneous beliefs: gradual information flow, limited attention and heterogeneous priors.
(2) Individuals have the ability to learn, remember and predict. Memory is the embodiment of intelligence. Individuals can not only learn knowledge from historical experience and memory, but also apply the learned knowledge to predict the outcome of similar events in the future. This article uses BP neural network to simulate the individual's memory and prediction ability.
(3) Structured and quantified knowledge modeling. The individual learning knowledge (experience) appears in the form of propositions and is structured and binary quantified.

Overall framework description
The overall framework diagram is shown in Figure 1.
From the overall framework diagram, it can be seen that heterogeneous individual belief set consists of knowledge belief and achievable belief, and the former plays a guiding role for the latter. Both belief formation processes require learning and memory through knowledge modeling and BP neural network, transforming individual experience knowledge into belief and adding to individual belief set. In the process of belief formation, the belief set is in the process of dynamic change: in the formation of knowledge belief, the individual's different historical experience and knowledge will form a heterogeneous individual belief set; in the formation of achievable belief, the new belief is added to the original belief set to make the individual belief set expand continuously. • The formation process of knowledge belief Individual knowledge belief refers to the knowledge currently possessed, and has the evolution and inheritance characteristics of knowledge [3]. This paper uses different individuals to randomly learn different experiences and knowledge to represent the initial belief set of the current individual heterogeneity. These experiences and knowledge are expressed in the form of knowledge propositions, and they are converted to fixed-form binary propositions through knowledge modeling. Then these binary propositions are learned and memorized by BP neural network, converted into individual belief, added to belief set to form the knowledge belief of heterogeneous individuals.
• The formation process of achievable belief The individual's achievable belief refers to what is not established but will be established in the future. This belief represents the individual's tendency and goals. For things that have not yet taken place or that are not yet established, individuals only grasp certain conditions (including status, information, etc.) that may occur, and it is unclear what will happen (in this case, individuals only have incomplete knowledge or experience). In view of this situation, people will form thinking judgment based on their own experience analysis and prediction. In order to reflect this process, this article divides the formation of achievable belief into two phases: first learning stage and re-learning stage. In the initial learning stage, the individual adopts the BP neural network trained in knowledge belief process to reason and predict the incomplete knowledge propositions formed through knowledge modeling, and then combine these incomplete propositions with the corresponding prediction results to form complete knowledge propositions. In the re-learning stage, the learning and memory function of the neural network is used to add the complete knowledge proposition training formed in the previous stage to the original belief set of the individual to achieve the purpose of expanding the belief set.

Knowledge modeling of belief
There are many forms of knowledge expression. In order to facilitate the learning and memory of BP neural network, we use structured and quantified propositional forms to express knowledge, called knowledge proposition. Structured knowledge proposition include four parts: action set, natural state set, external information set, and result set. Its formal description is shown in Formula 1.
Knowledge proposition P = {action set A, natural state set Ω, outside information set X, result set C} (1) ① Action set A: Also called program set, denoted as A = {a 1 ,a 2 , … , a n }, representing the set of actions that an individual may take.
② The natural state set Ω: Also known as the state space, is used to represent all possible states, denoted as Ω = {θ 1 , θ 2 , … , θ n }.
③ External information set X: Also known as a set of uncertainty, it can be written as X = {x 1 , x 2 , … , x n }. Due to a large number of uncertainties, several external factors that have a large impact on the result set are usually selected for analysis.
④ Result set C: The various possible consequences that the individual will produce after making a choice, which can be recorded as C = {c 1 , c 2 , c 3 , … , c , where i = 1,2, … , m}.
The structured knowledge proposition still does not meet the needs of neural network learning, and this article continues to transform it into a structured fixed binary form knowledge proposition, as shown in Equation 2: P' = action set⊕natural state set⊕external information set⊕result set (2) The process of conversion is to quantify each subset of the knowledge proposition. The quantized rule is to use the encoding of 0/1 to correspond to the actual meaning of the elements in the subsets. After all subsets are converted to 0/1 coding, all 0/1 codes are combined into a binary data stream with fixed number of digits and clear meaning according to the fixed order of formula 2. This data stream is a quantified knowledge proposition.

Learning memory and predictive reasoning model of belief formation
• A brief introduction to the neural network Neural network is a mathematical model that simulates the neural structure of the brain for information processing. In this paper, BP neural network is used to simulate individual belief's learning and memory and predict reasoning process. The BP algorithm is an error inverse propagation algorithm [15]. Its advantage is to encapsulate the complex mechanism of human brain operation. The user only needs to focus on the functions of learning, memory, predictive reasoning, and so on. In this way, when BP network is used to study the problem of belief formation, not only the reasoning process can be realized, but also the prediction and analysis of the results can be realized and the decision can be made. The number of input layers of the network is determined by the structured subset of the belief knowledge modeling process. In the above example, 5 subsets are structured, and the input subsets are four, that is, the input is composed of four different eigenvectors, so the number of nodes in the input layer is four. The output layer is determined by the result set. The result set contains 4 different results, that is, 4 different feature vectors, so the number of nodes in the output layer is four.
• Learning memory and reasoning prediction model Human memory and intelligence are not stored in a single brain cell, but in a network of interconnected brain cells. This is called distributed storage. Through continuous learning, people change the way of connection between brain cells, so that the function of the network is improved. The BP neural network is constructed by imitating the structure of nerve cells.
The basic principle of neural network is: every neuron, supposes it has n inputs (x 1 , x 2 ,…x n ), its output value is: Where f is a function and w i is an indeterminate weighting factor. The principle of learning memory is: The input layer neurons receive the input of the learning sample data, the hidden layer and the output layer neurons process the input information, and the output layer neurons output the actual result. The weight is continuously corrected based on the difference between the actual output and the ideal output. After repeated training, the actual output and the ideal output tend to be consistent. The essence of learning and memory is the dynamic adjustment of each connection weight.

Overall experimental design and process
This section design experiments to illustrate the effectiveness of BP neural network based heterogeneous belief formation methods, and at the same time verify the formation mechanism of three heterogeneous beliefs: gradual information flow, limited attention and heterogeneous priors.
The limited attention mechanism in the formation of heterogeneous individual beliefs means that the individual does not deal with all the information in life due to factors such as time and energy. The limited information that different individuals pay attention to is different, so the conclusions they draw from focusing on the information are not completely consistent. In the experimental design of knowledge belief formation, learning samples are the knowledge proposition after modeling, which is equivalent to the information and knowledge that individuals have learned. When the learning sample sets are different, it means that the information acquired by different individuals is different, and the differences in the content of the learning samples of the different individuals lead to the heterogeneity of the individual in the limited attention.
The gradual information flow mechanism means that the same information does not reach all individuals at the same time, that is, the flow of information is a gradual process. Different individuals have different channels to receive information and have different time to obtain information, and individuals who receive new information are expected to make changes to previous events. In order to verify this mechanism, in the experimental design of the belief formation, it is possible to compare the prediction results of the prediction samples when the new learning samples are added to the original belief set. In the experiment, the original belief set is equivalent to the information possessed by the individual, and the expanded belief set is equivalent to the individual gradually receiving new information.
Heterogeneous priors refer to differences in individual experiences, occupations, education levels, etc. Even if they accept the same information, their own reasons will lead to different ways of selecting and processing information. Therefore, different individuals will not have the same judgement on things. The above two experimental designs implicitly verify the prior heterogeneity of individuals. The heterogeneity priors are simplified into the inconsistency of individual decision making caused by different experiences. In the two experiment designs, the individual experience is equivalent to the learning sample set. When the individual learning sample set is different, it is equivalent to the individual's experience.
In order to illustrate the difference of heterogeneity among individuals, this paper defines the relative proportion of individual heterogeneity δ it : Where i={i 1 ,i 2 、i 3 …i n } represents the experimental group, i 0 represents the control group; t={t 1 、 t 2 、 t 3 …t n } represents the sample to be predicted; k={k 1 、k 2 、k 3 …k n } (k is a binary form) indicates the predicted result classification. P itk represents the probability that the individual H i obtains a classification result of k when predicting the sample t. δ it is the sum of the absolute values of the difference between the probability values of the individual H i and the control individual H i 0 on each classification result k when predicting the predicted sample t. MAX( δ it ) represents the maximum critical difference value. The sum of these differences reflects the difference in prediction between the experimental group and the control group, that is, the standard of individual heterogeneity δ it is a quantitative way to measure the difference in belief heterogeneity between different individuals. δ it % represents the proportion of individual heterogeneity differences, it can more accurately and conveniently measure and compare the size of the differences between individuals, and also quantify the three formation mechanisms of heterogeneity.
In the experimental design of knowledge belief and achievable belief formation, the previous steps are the same, that is, the previous historical experience must be screened and transformed into the knowledge possessed by the individual, and then to construct and quantify the propositions in binary form. The quantified proposition represents the sample to be learned and predicted in the experiment.

Experiment of the formation of knowledge belief
➢ Experimental Design In this experiment design, we explore the influence of different learning samples on the formation of individual beliefs. The difference between the individual learning samples can be divided into two cases: the difference in the number of samples and the difference in the content of the samples, and these differences are caused by the individual's own experience and the different information they are concerned about. Therefore, this experiment validates the feasibility of the mechanism of limited attention and prior heterogeneity in the formation of knowledge belief.
According to the actual situation, 30 knowledge propositions were selected from all historical experience knowledge, then they were structured and quantified into binary propositions as individual learning samples, and some propositions that individuals have not yet learned were selected as prediction samples.
In the experimental design of knowledge belief formation, in order to study the influence of the difference in the number of learning samples on the formation of beliefs among individuals, the experiment was divided into experimental group and control group. The control group was 0 , and the sample set he studied was the first five individuals ( 1 , 2 , 3 , 4 , 5 )which have different learning sample sets. On the basis of the 0 sample set, three, six, nine, twelve, and fifteen learning samples were randomly selected from the last fifteen learning samples to replace the samples in the control sample set, so as to show that each individual learns different knowledge. A schematic diagram of the differences in learning samples between different individuals is shown in Figure 2. In the above figure, each square represents 3 learning samples, and the gray-white square represents the difference between the experimental group and the control group. This indicates that the number of samples in the experimental group is the same as the control group, but the difference in the content of each individual sample is gradually increasing compared with the control group.
Each individual predicts three different kinds of samples (A, B, C) by using the sample set that he learns. A total of 18 experiments were performed (in which a control group had an additional experiment, using a sample that had been studied as a predictive sample to predict the outcome and verify the integrity of the belief formation process). The experimental results are shown in Table 1. The error sizes of the corresponding groups in each prediction sample are shown in Table 2. The relative differences in heterogeneity between the experimental group and the control group are shown in Table 3. ➢ Experimental result Table 1  According to the formula of relative proportion index of individual heterogeneity difference, Max { } = 2.0, Min { } = 0.0, that is, the theoretical maximum value of relative proportion index of heterogeneity difference after quantification is 2.0, and the theoretical minimum value is 0. The larger the value, the difference between individuals is greater. According to the experimental results in Table 4, the relative heterogeneity index of the individual can be obtained as shown in Table 3: Based on the above experiments and results, the following conclusions can be drawn: ① The experimental results from the first row of Table1show that when the predicted samples are the samples that have been studied, the predicted result is exactly the same as the learned result, which indicates that the framework of individual belief formation constructed in this paper is correct and the individual has formed belief knowledge.
② The horizontal comparison of the data in Table 3 reveals that the relative proportion of heterogeneity differences among individuals is different when the individual learning samples are different, which reflects the heterogeneity of beliefs among individuals. In this experiment, the difference in the number of samples learned by individuals is reflected in the actual life, that is, different individuals pay attention to different information and individual factors (such as occupation, education level, etc.) lead to different experiences. Individual heterogeneity has different relative proportion indicators, indicating that the conclusion of information processing is different. Therefore, the experiment verified the limited attention mechanism and the prior heterogeneity mechanism of individuals.
The above experiments validate the effect of differences in the number of individual learning samples on individual heterogeneity beliefs. Further refinement of the experiment will verify whether individual beliefs are affected by the same number of learning samples but different contents.
Individual 0 was used as a control group, and sample C was selected as the prediction sample. Individual 6 was selected, 6 and 1 have three samples different from 0 . The new three samples of individual 6 are replaced by the same position as 1 , but the replacement content is different. The schematic diagram of three individual learning samples is shown in Figure 3. The three individuals were predicted and the results were compared with the following table 4. Using the individual heterogeneity difference relative proportion index calculation method of formula 4, the difference index between 0 and 1 is 1 = 1.0294, and the difference index between 0 and 6 is 6 = 1.1710 , the difference index between 1 and 6 is 1−6 = Max{ } = 2.0. Formula 4 show that the larger the relative proportion of heterogeneity indices between individuals, the heterogeneity difference between individuals is greater. The results of the above three indicators are all greater than 1. It can be concluded that the difference of learning sample content among individuals will affect the heterogeneity between individuals. The difference of individual learning content is due to the individual's own experience or attention to different information, so the heterogeneity differences between individuals in this experiment also verify the limited attention mechanism and the prior heterogeneity mechanism.
Combining the above two experiments, we can see that the difference of learning sample content and the number of learning samples will have an impact on individual heterogeneous beliefs, but the former has a greater impact than the latter. The two experiments verify the individual prior heterogeneity and the limited attention mechanism.

Experiment of the formation of achievable belief
➢ Experimental design In the experimental design of achievable belief formation, it is necessary to study the effect of adding new beliefs to the individual belief set on the predicted sample results. The new belief is equivalent to the individual's new learning knowledge. Individuals receiving knowledge is a gradual process. With the increase of knowledge, it may affect the individual's view of something. Therefore, this experiment validates the mechanism of gradual information flow in heterogeneity.
In the experimental design, the 0 individual and its learning sample set are selected as the individual's original belief set, and the original belief set is equivalent to the knowledge that the individual has mastered. The individual re-learns a new incomplete knowledge proposition and predicts its possible outcome under the influence of the original belief set. The result and the incomplete proposition form a complete knowledge proposition through the neural network learning and memory. The individual transforms the proposition into belief and expands the belief set. Then the original belief set and the expanded belief set are used to predict the same prediction sample and compare the prediction results. If the results are different, it shows that the heterogeneity of individual beliefs also exists in the formation of achievable belief, and it also verifies mechanism of gradual information flow in heterogeneous beliefs. In this experiment, sample E is chosen as an incomplete knowledge proposition, and sample B as the sample to be predicted.
First, the prediction results for sample B were known in the above experiment without sample E. In the new proposition formation stage in which achievable belief formation, BP neural network uses the initial belief set reasoning to predict the prediction result of the newly added incomplete knowledge proposition, and combines the incomplete proposition with the result to form a complete proposition. The experimental results are shown in Table 5. The most likely prediction result of sample E is'01', and sample E forms a complete knowledge proposition'1000000 000 01'.
During the formation stage of new beliefs, the new complete knowledge propositions are transformed into individual beliefs after learning and memorizing by BP neural network and added to the initial belief set. Individuals use the expanded belief set to re-predict sample B to get the prediction results, and use the expanded belief set to predict incomplete knowledge proposition E. The experimental results are shown in Table 6. The experimental results in the first row of Table 8 show that the expanded belief set does not change the prediction results of incomplete knowledge proposition E, which verifies the feasibility of the proposed achievable belief formation method. The experimental results of the last two rows of Table 8 can be obtained from Equation 5. The heterogeneity difference index between the new belief set by adding sample E and the initial belief set prediction result is δ i 0 E =0.7570.

➢ Experimental result
As can be seen from the above table, the new belief set after adding sample E will change the prediction result. Sample E is equivalent to the new beliefs learned by individuals. The experimental results show that the differences in new beliefs among individuals also affect the heterogeneity of individuals. This is reflected in the actual life is individuals' attitude towards things will change as they continue to accept new knowledge. This experiment verifies the feasibility of individual progressive information flow mechanism in the formation of achievable belief.

Summary and prospect
This paper redefines individual heterogeneous beliefs in combination with previous research results, so that its scope of application is not limited to the financial field, but also used in other fields. This paper summarizes the research status of belief definition and belief formation in the field of Agent computing and finance, and proposes a new research method of belief formation based on the three formation mechanisms of BDI model and heterogeneous belief. This method combines the advantages and disadvantages of belief description in previous studies, and proposes a basic framework of belief formation based on agent modeling, structured and quantified knowledge propositions, and individual learning and memory prediction function. This belief formation framework not only has rigorous reasoning similar to the human brain, but also uses the probabilistic method in the strategy of decision making to quantify the beliefs of heterogeneous individuals. This quantitative analysis process is simple and practical, and easy to understand. The experimental example not only verifies the feasibility of the belief formation method, but also verifies the feasibility of the three mechanisms of heterogeneous belief formation under the method. This paper focuses on the formation of individual beliefs from the individual's own internal situation to consider, in the follow-up study will focus on the interaction between individuals and the environment and the interaction between individuals on the formation of beliefs; at the same time, this study also paves the way for the expansion of beliefs.