A two-layer framework for activity recognition with multi-factor activity pheromone matrix

With rapid population aging and increasingly indoor sensing technologies, mining effective information in sensor data is in need that we can analyse individual behaviour semantics, or even predict intentions. The model for indoor activity recognition (AR) is usually based on statistic while sensor data can impliedly reflect abundant information in order. Behaviour will trigger environment perception sensors. Inspired by information transmission in nature, persistent action keeps activity pheromone accumulating and inactive action keeps it volatilizing along with time shift. Different from statistic model, our framework proposes a method to construct multi factor features named activity pheromone matrix (APM). It has a double-layer model for recognizing daily activities include the high-overlapping. The experimental results show that our method can effectively promote the accuracy of activities recognition compared with the existing statistical models, even the high-overlapping activities in small areas.


Introduction
Ambient Assisted Living system is designed to help person get life supports and medical care by living in smart home. The activity analysis of daily living (ADLs) is an important part of the systems and divided into Activity Recognition (AR) and behavioural patterns discovery. Data is collected to model the behaviour patterns of target users, or guess the user`s behaviour intention in order to facilitate the system to adjust the response in advance. Sensor data are continuous time series. Commonly, artificial features can be divided into two main categories: statistical features and frequency-domain features. Statistical features are calculated by mean, variance, and covariance. Frequency-domain features are calculated by Fourier transform or wavelet transform. Traditional AR model features are often based on statistic features. However, a lot of hidden activity information is ignored and activity has been distorted. The accuracy of AR declines to a certain extent. How to fully excavate the hidden information of data ontology and construct effective features are difficulties of human indoor activity recognition. Generally, activity is contextually perceived and consists of sub-activities. Researchers regard sub-activity as atomic-level action [1,2]. The diversity of track makes it accidentally to generate new cluster during the training process and the ambiguity makes similarity between two activity clusters higher. Smaller inter-cluster distance increases the probability of behaviour confusion. How to solve activity ambiguity is another difficulty of AR.
The wireless trigger-sensor network system is the experimental data collection environment of the target user daily activities in this paper. One record represents environmental change caused by a sub-activity. A complete activity has an active fragment, which is regarded as activity ontology and retains all original information. Researchers need to dig hidden data structures and relativity to construct explicit structures that can be used to maximize expression of activity. Hidden information contains behaviour-semantic related information, triggering order and temporal sequence. Most activity ambiguity refers to activity confusion in similar region. We define that high-overlapping activities have fragments with the same triggered sensors in the same region. For example, both washing and cooking will touch off the sensors in water channel or cupboard. Majority papers in ADLs based on sensor data show that traditional processing methods usually calculate the statistic distribution value. Feature vectors are constructed by some statistical numeric values, such as sensor-triggered count by space or time zone [3,4]. This model could have an effectively recognition, but simple construction of feature vector makes some identifiable information lost from the activity ontology.
In this paper, we propose a new framework based on the event trigger sensors data for AR. A novel method of feature construction is proposed. Multi-channel set different parameters for different activities, which makes the framework have a superiority to deal with high overlapping activity. The contribution of this paper about human activity recognition can be concluded as follows:  Activity Pheromone Matrix. This paper proposes method of feature extraction from sensor data, to increase the inter-cluster distance for improving accuracy.  Time attenuation factor and time enhancement factor. The method makes use of sensors spatial position, trigger frequency and trigger duration to construct model factors. Factors facilitate the distinction between high-overlapping activities.
 Two-layer model with multi-channel. According to the activity division, model sets personalized features to improve whole AR accuracy. It has reference significance.

Related works
With the trend of intelligent computing and ubiquitous computing, the paper number keeps increasing for AR based on location data or sensor-triggered data stream [5]. Tapia [6] proposed that Naive Bayes model could be used to classify and identify activities in tagged sensor data. Z Zhao [7] proposed an activity recognition self-adaption model for creating a personalized model for target users AR. K Rieping [3] drew on the experience of the textlanguage classification algorithm. Latent Dirichlet Allocation is a document-topic generation model consisting of words, themes and documents in three-layer structure. One record is regarded as one word in the special documents, and the activity sequence fragment could be seen as the topic of the documents. He segments the dataset into shape-let by a certain time window, construct time series pattern. In paper [8], researchers use sequential patterns to discovery the latent structure also through the very model. L Liu [9] change the ubiquitous sensors data into time series shape-let. W Jiahui [10] adopts the same dataset as we used in this paper. His major research is focused on implicit structure and association analysis between data through Hierarchical Dirichlet Process and Adaboost-HMM (Hidden Markov model) algorithm. He [11] also adopts the statistic vector to process the binary data. Like Johnson N [12] promoting in 1995, models calculate feature vector statistically from sensor data sequences in most paper, which. Our framework innovatively uses activity pheromone matrix as feature, which is described in section III.
High-overlapping activities share major triggered sensors and frequency so that the sequence form is similar. Motion overlapping causes motion self-occlusion [13], which is a hard question due to trajectory randomness. After comparing motion history image with directional motion history image, Mar Ahad [14] adopts seven higher order Hu moments to calculate feature vector in one action. V Nen [15] presented on-line version of the Minimum-distance classifier to achieve higher classification accuracy than static off-line one. Zhu C and Sheng W [16] process dataset ingeniously in multilevel and multidimension. One event sensor frequency differs from another, which was used to distinguish activities in same region.

FSDAR for sensor data to activities recognition 3.2 F1SDAR resume
High-overlapping activities data could be excavated to distinguish by distinctive frequency and duration time. This paper proposes a novel multi-granularity and multi-dimensional framework. As shown in figure 1, dataset is sliced into independent activity fragments called ontology to uniquely represent activity. In FSDAR, it prefers to combine location information and spatio-temporal factors to construct activity pheromone matrix as model features, rather than the feature vectors based on statistical rules in traditional methods. It also emphasizes context semantic relevance on sub-activities, which mapped to the data structure is between adjacent data. There is a two-layer model for coarse-grained and finegrained classification. The coarse-grained classification model is used to identify whether the activity belongs to the high-overlapping. High-overlapping activity uses the feature which is the activity pheromone matrix with enhancement and attenuation time factors (APMTA) with signal TAF in figure 1. And activity pheromone matrix (APM) features if not. After changing the fragments into feature matrixes, those will be input fine-grained classification model, which is used for classifying activity category.

Classification for high-overlapping activities
Coarse-grained classification model identifies whether the activities are highly overlapping. We add temporal constraint into an activity pheromone matrix of high overlapping activity to improve activity specificity. Without losing generality, sensors record is in formula 1.
where d represents date and ti represents time. v is binary value and �h represents the context-awareness application location. Environment-embedded Sensor is fixed position and trigger, which means an active behaviour in the vicinity. Activity ontology slice constructed represent as: (2) T is trigger duration time. Activities in one category are composed of specified sensors. First extract fragments from activity ontology. In text analysis, model extract keywords and feature vector though TF-IDF algorithm, Vector Space Model (VSM) respectively. The basic idea is to simplify documents as vector, value being weight of keywords. Similarity calculation between two documents uses cosine similarity formula in formula 3. Activity ambiguities are discriminant through activity similarity.

APM and APMTA construction
Animals will secrete an external hormone in order to leave some information to other individuals of the same species called pheromones. Traditional data process usually constructs statistic features via regional information and time shapelet, missing abundant information. Sequential fragments contain triggering duration time, rough position and sensor ID. Information has a specific order. The sequence transformation matrix of data structures and data characteristics can be viewed as information left over from the activities called active pheromone matrix. Data structure is not sensitive to the isolated point. Similarity is a common barrier for AR. Both sequence and continuous sensor-triggered duration increase the self-characteristics between various categories. It effectively improves the recognition accuracy for high-overlapping activities. The value "1" returned indicates that the sensor triggering state starts, while "0" indicates that the sensor trigger state is over. We consume that i-type sensor was triggered in the k1 record and ended in k2 record, � �� � � k� � k� k�t � k� h k� , � �� � � k� � k� k�t � k� h k� . From  k� � t to  k� � t the difference value between time-stamp represents a complete sensor-triggered duration. For high-overlapping activities in small region, they have similar sensor set and frequency while sequence and duration obvious difference. Extracting event data from the raw sequence ontology, we add the temporal coefficient to preserve triggering pheromone. Figure 2 shows the time factors how to play in pheromone matrix generation. For the t moment, �䁨 represents historical pheromone and represents synthesized pheromone.
represent newly generated pheromone from the real-time container that temporarily save the triggering sensor type. When remains in triggering state, sensors in container will continuously accumulate pheromone with the time coefficient combined factor σ. As triggering state have been over, position will be removed from such a container and the corresponding pheromone begin to attenuate with the time coefficient combined factor ρ. Both of the two temporal factors enhance the impact of main triggered sensor. This structure looks like a simplified version of Long Short-Term Memory (LSTM) with a forget gate to reduce historical memory. The specific formula is as follows: � ��� t σ� �� t � � � �� �䁨 (4) �� � � � � � ��� (5) Fig. 2. The mechanism of enhancement factor and attenuation factor.
The APM with time factors increases distance between samples. Experimental results of validation effectiveness are shown in next section. We assume that represents the pheromone before time t and the represents the immediate pheromone matrix at time t. At the next moment, APM is shown below: And the APM with time factors is shown below: After n moments, � has changed into APM and APM-TA respectively shown below after iteration:

Dataset
Some well-known datasets for AR are open on the Internet, such as CASAS [17,18], KASTEREN-dataset [19], WSUCASAS [20] and so on. We model on Aruba dataset. All data pre-process and model training is based on Python 2.7 language with the Core(TM) i5−6400T processor. Three sensor categories are in the dataset with the respectively symbols M, D, T. Records are sensor data triggered by an individual`s behaviour for 220 days. The numerical difference of samples and records from different activity categories is quite large. The active category named Resperate only has 6 samples while Relax has 2919 samples in total. On the other hand, the type of triggered sensors is also quite different. Data imbalance is an important reason to affect accuracy of AR.

High-overlapping recognition
We first extract the activity fragments from the active ontology. In text analysis, TF-IDF algorithm extracts the keyword of the article, and the Vector Space Model(VSM) model constructs the feature vector of keywords. the VSM model is used to extract uniquely feature representation vector from the active fragment. The smaller the angle between the vectors is, the more similar the active fragments are. Key words response sensor types, values response sensor-triggered frequency. Activities similarity is shown below in figure 3.
As we can see from the similarity matrix, Enter_Home is highly similar to Leave_Home in 0.869, and Wash_Dishes is highly similar to Meal_Preparation in 0.785. From the perspective of life experience, it is logical because activities are similar in its scope and significance.

Fig. 3.
Similarity matrix of training samples. This is a symmetric matrix to describe the similarity.

Time factors
The purpose of using two-layer model is to classify the easy confusing activities on the coarse-grained model, and accordingly decide whether activity fragment is charged to be APM or APMTA as the fine-grained model feature. Compared to APM, APMTA is added enhancement and attenuation time factors. As shown in figure 4, Meal_Preparatrion and Wash_Dishes are used as confusing activity group in the experiment. Compare numerical values to see whether distance between two similar activities have increased through temporal factors. Active specificity increased so that active similarity decreased. Enhancement time factor σ and attenuated time factor ρ are constant of the exponential function with time as variable. Factor σ and ρ restraint each other. If σ is too large, the current pheromone effect is also large. It causes prophase distortion. If ρ is too large, preexisting pheromone have been excessively volatilized, and that easily leads to post distortion. Whether it is prophase distortion or post distortion, features cannot accurately express activity.

Experiment analysis
Our paper proposed a novel feature matrix APM compared to traditional feature vector, its effectiveness stemming from its more implicit active information to enhance the identification performance of activities. We first extract a multi-dimensional column vector as features. Five dimensions represent the total number of spatial triggering sensors in different regions of the active fragments. Another dimension is the number of sensor types triggered and the last dimension is the duration time.
In the comparative experiment, all the active ontology fragment was transformed into APM feature matrices with different models. Activities completed in overlapped regions generally have an inferior performance in figure 5(a) while the activity recognition accuracy has been significantly improved intuitively in figure 5(b). However, the performance of activity Wash_Dishes is still very low because the positive and negative samples are out of proportion. Its totality is 0.04% in the proportion of Meal_Preparation. Selecting algorithm non-sensitive to the unbalanced dataset or oversampling minority activity is effective to reduce the impact of dataset unbalanced. Both Naive Bayes algorithm and Decision Tree algorithm will have better performance on minor-sample events than others.    Compared to the two-layer model, the single-layer model cannot adaptively identify high-overlap activities and adopt multi-channel processing. As shown in figure 6, if all the features are in the form of APMTA, the recognition accuracy of some activities with highly discrimination turns into lower. The reason is the time factors will lead to the former distortion, weaken the specificity of the isolated activity. Model is sensitive to attenuation factor and enhancement factor. As results shown in figure 6, the accuracy of easy confusing activities do improve. When the time factor is added, the effect promotion is universal. For different parameter combination of attenuation factors and enhancement factor, the model effect is different. The framework model performance evaluation according to the parameter settings is shown in table 1. The APMTA is equal to APM as both ρ and σ are zero. Obviously, the experimental results of APMTA are superior to the APM on precision, recall, and F1 score.

Conclusion
In this paper, a novel framework FSDAR is proposed for activity recognition in WSN. It can apply to wearable devices or indoor location-based for the method of feature matrix construction being general. Traditional statistic feature vector can be extract easily with abundant information losing. Pheromones matrix combines spatial and temporal information to distinguish activities, even the high-overlappings. Results show that feature matrix has a common improvising than feature vector. Temporal factors emphasize the tiny specificity so that it effectively increase the sample distance between similar activities. The framework chooses a two-layer model with multi-channel to avoid time factor weakening specificity of simple activity with few core sensors. Through datasets effective test, the experiments show that novel feature extraction improve accuracy of recognition. The appropriate time factor effectively increases the cluster distance. The experiment results emphasize the effectiveness of our new framework comparing with traditional AR model.