Execution and analysis of classic neural network algorithms when they are implemented in embedded systems

. Many algorithms related to neural networks are used in a large number of applications, most of them implemented on computational equipment that have great processing and storage capacities, however, new communication schemes such as the Internet of Things, need that neural algorithms can be executed from small electronic devices, devices that do not have large storage or processing capacities, but they can function as intelligent control centres for the different "things" connected to the Internet. Currently, there are various electronic devices that allow generating low-cost intelligent technology projects that permit interaction within the Internet of things, such as the Arduino UNO, Tiva-C, and BeagleBone development boards. In this project, we present the analysis of the Perceptron, ADALINE and Hopfield neural network algorithms, when they are executed within the three mentioned development boards, in order to define the best tool to be utilized when using such neural schemes and few data are processed. Economic cost, temporary response and technical capabilities of electronic devices have been evaluated.


Introduction
Nowadays, it is of great scientific and technological interest the development of engineering systems that can interact with their environment, obtain information from it, process such information intelligently, define an output behaviour, and send and receive data through the Internet, looking for these developments to be low cost so they can be accessible for any user.
Several development boards that can be found at low commercial cost, are used for the creation of various technological projects, such as the Raspberry Pi, Arduino (in its various forms), BeagleBone, Tiva-C, FPGA (field-programmable gate array), Jaguar One, Parallax Propeller, among others. One of the main characteristics of this type of boards is its small size, which is ideal for applications that require mobility. Additionally, through its input and output pins, projects developed with this kind of boards can interact with their environment in an easy and fast way. However, one of the main disadvantages of these embedded systems is that they do not have large processing neither storage capacities.
Currently, it is not only looking for development projects to be mobile, cheap, efficient, but also to make them smart. However, algorithms that process and analyse information intelligently, such as Artificial Neural Networks (ANN), usually require large processing capacities [1], which limits the use of the aforementioned development boards. In some cases, these boards are only used as a bridge between the data acquisition and more powerful computing equipment that processes such data [2], and in some other cases, the processing is divided between a development board and a data processing system of high capacity [3,4].
This project seeks to analyse the algorithms of Perceptron, ADALINE (Adaptative Neural Network), and Hopfield neural network, when these are implemented in the Arduino UNO, Tiva-C LaunchPad and Beaglebone development boards, analysing their behaviour with respect to time, and the capacity to respond efficiently when there are not large amounts of data to be processed.

Embedded systems
An embedded system is an electronic device considered to be of specific purpose, these devices have all the necessary elements to achieve their goals. They can be seen as small computers that have a processing unit, RAM and ROM memory elements, input and output ports, internet connection, and in the specific case of the embedded systems that we are going to work with, input and output pins that allow to obtain information from their environment through the control of sensors and different types of actuators. Currently there are different types of embedded systems, among which we can find development boards based on FPGA, based on microcontroller and based on microprocessor; systems that may contain their own Operating System (OS) or depend on a development environment running from a PC. This type of systems can be programmed directly in assembly language, or by means of a high level language such as C / C ++.
The development of this type of systems has allowed the design of a countless number of technological projects [5][6][7], as well as they have served as platforms to bring students of different levels closer to scientific and engineering design [8,9].
Although there is a great variety of this type of platforms, in this study only 3 of them will be treated, Arduino UNO, Tiva-C LaunchPad and BeagleBone. In figure 1, you can see the three embedded systems presented in this project. In table 1, you can see the technical characteristics of each of these development cards. BeagleBone, c) Tiva-C Launchpad.

Arduino UNO
The Arduino platform is perhaps the best known of all the existing development boards at this time. Arduino was developed at the Interaction Design Institute Ivrea in Ivrea, Italy. This project was born with the intention that beginners and professionals could develop technological projects in an easy and economical way. It is currently managed under the open hardware license. Among all the platforms used in this project, Arduino UNO is the development board with the most developed scientific and technological applications, some of them can be seen in [10], [11] and [12].

BeagleBone
BeagleBone is a board of low consumption, but of great computational power. Developed by Texas Instruments in association with DigiKey and Newark element14. This board was designed to develop open source software, and like other boards of this type, it was also designed with the intention that it could be used by students to develop projects based on free software and hardware. It is currently marketed under the share-alike license of Creative Commons. It is a board that contains its own OS, which can be Debian, Android or Ubuntu. There are several versions of this development board, to visualize its characteristics and research areas of each of them, can be read [13]. BeagleBone has been used for the development of different technological projects, for example as a control system for mobile navigation of a differential unit using a single camera connected to the robot platform [14]. This board has also been the core of schemes related to the Internet of Things (IoT), in [15] an intelligent multisensor system was developed, where a multipurpose hierarchical network composed of intelligent nodes capable of obtaining and managing heterogeneous data and spaces is created according to the IoT paradigm. This board has also served for the development of pollution monitoring in a city, measuring the level of carbon dioxide and carbon monoxide in the air along with the location of the Global Positioning System (GPS) as can be seen in [16], among other projects [17,18].

Tiva-C LaunchPad
This development platform focuses on the design of electronic prototypes, is based on the Texas Instruments microcontrollers family. It uses the M4C123G microcontroller. It can be programmed through the C language, or through the Energia project, which is an Open Source variant of the integrated development environment (IDE) of Arduino. The characteristics and manner of use it can be seen in [19].
Tiva-C has been used for projects such as the development of a multispectral information acquisition system to analyse vegetation cover, where the development board is coupled with an unmanned aerial vehicle, which allows the manoeuvrability of latitude and longitude, as can be seen in [20]. It has also been used for the construction of a laboratory prototype of a wireless sensor network (WSN) [21], among other projects [22,23].

Neural networks
ANNs are computational schemes that try to imitate the way in which the human brain learns. These schemes have become very popular tools for solving different types of problems, mainly recognition and search of patterns, as well as imaging processing. We can see some applications and different schemes in [24] and [25].
Although this type of schemes has mostly been developed in projects dependent on high-performance computing equipment, they have also been implemented in various embedded devices such as Arduino and FPGA [26,27]. An attractive proposal to be able to implement neural algorithms in embedded systems of low processing capacity is the C-Mantec algorithm, proposed in [28] and later implemented in an Arduino UNO board to analyse its characteristics [29]. However, in this study we will focus on the classic algorithms of the neural schemes: Perceptron, ADALINE, and the Hopfield network.

Perceptron
The perceptron, was proposed by Frank Rosenblatt in 1958 [30], this scheme can be visualized as the basic neuron within an ANN. It can only be used as a linear discriminator, that is, only linearly separable problems can be solved by a simple neuron. However, the interconnection of different neurons, manages to solve problems with complex behaviour such as can be seen in [31][32][33][34]. Figure 2 shows the basic scheme of a perceptron, while figure 3 shows a multilayer network, with several interconnected neurons.

ADALINE
Like the perceptron, the ADALINE is a simple neuron but with a linear response with respect to its inputs, it was designed to work with memistors [35].
This neuron was mainly differentiated from the perceptron by its learning rule, since the error is not adjusted after crossing the activation function, but the error is adjusted with respect to the sum of its inputs. ADALINE has been used in many applications, including [36][37][38][39]. The ADALINE scheme can be seen in figure 4.

Hopfield neural network
The Hopfield neuronal network was proposed in 1982 [40]. This network seeks to learn a set of patterns, and thus identify new input cases and associate them with one of the previously learned patterns. The neurons of this network, generally has only two states, activated and not activated. The Hopfield network is a single-layer network, but with a recurrent connection, which means that each neuron is connected to all the other neurons, except itself. Some of the applications of this network can be found in [41][42][43]. The basic scheme of the Hopfield network can be seen in figure 5.

Development
This section shows the experiments carried out to obtain the time measurements when each of the aforementioned algorithms is executed in each of the development boards described above.
In the case of Arduino UNO and Tiva-C, there is a development environment from the PC in which the informatic program is made, and such program is loaded to each of the boards, so Arduino UNO and Tiva-C execute only the program related to the implemented algorithms.
On the other hand, the BeagleBone board is preloaded with the Debian OS, so it has a graphical interface. To access it, you can do it from its mini HDMI (High-Definition Multimedia Interface), or through a remote connection using a VNC server (Virtual Network Computing). In this case, the card has been used by means of the command line, using an SSH (Secure Shell) connection, for which the Putty client and the IP address 192.168.7.2 have been used.
For the implementation of the neural algorithms, each one of the mentioned schemes was executed in each one of the development boards, and the processing times of each executions were obtained.

Perceptron implementation
For the first experiment, a single execution of the Perceptron algorithm was performed on each of the development boards, although the algorithm stipulates that it starts with random synaptic weights, for this first experiment, the initial synaptic weights were defined equal for each execution. This was done with the intention of having reference points. For the training, the logical AND function was performed, which can be seen in table 2.
Once we had a single execution, the second experiment was performed, in which an average time of 100 Perceptron performances were obtained with the same initial values and with the synaptic weights defined by the user.
As a third experiment, the same training scheme was presented, but with random , obtaining the average time of 100 executions.
In Table 3, the execution times of each of the embedded systems can be observed, when the perceptron has been implemented once with defined , the average time of 100 executions with defined, and the average time of 100 executions with random .

ADALINE implementation
For the implementation of the ADALINE, the calculation of a binary to decimal encoder was made, in table 4 the training scheme can be seen, where there are 3 inputs and one expected output Y. The synaptic weights have been defined by the user. For the first experiment, a single execution of the algorithm was performed to measure the base time. In the second experiment, the average time of 100 executions with the same problem and with the same synaptic weights defined in the previous case was considered. And in the third experiment the average time of 100 executions performed with random initial values was considered. In Table 5, we can observe the response time of this neural scheme within the three embedded systems.

Hopfield implementation
For the implementation of this algorithm, the neural network was trained with the figures shown in table 6. In the first two columns are the examples with which the network was trained, and in the third column is the element that was subsequently tried to rebuild it through the Hopfield network. It is a simple example, and the algorithm correctly reconstructed the test element. Table 6. Samples with which the Hopfield network was trained, and the last column contains the element that was reconstructed.

Sample 1 Sample 2 Proof
For this case, only two experiments were carried out. In the first, the execution time of an execution is obtained, and in the second experiment the average time of 100 executions is obtained. The execution times can be seen in table 7.

Analysis of results
Although Arduino UNO is the most commonly used board in different technological and development projects, it can be seen in tables 3, 5, and 7, Arduino has a much longer execution time with respect to the Tiva-C and BeagleBone boards. If we compare the Arduino UNO board with the Tiva-C, it can be seen in table 1, that the approximate prices of both boards are close to each other, however, the characteristics of processing speed and storage are very far from each other, comparing the prices, the speed of the board, and the results presented in tables 3, 5 and 7, it can be concluded that for the development of this type of neural network algorithms, Tiva-C board can be a better option than Arduino. One of the characteristics that makes the use of Arduino popular is the ease of programming, however, when using the Energia programming interface, the programming of Tiva-C and Arduino basically is done in the same way, as seen in figure 6. On the other hand, it can be seen that the execution times of the algorithms implemented between the BeagleBone and Tiva-C boards are not so far apart. In Table 1, it can be seen that the processing speed of the BeagleBone far exceeds the processing speed of the Tiva-C board, however, when the algorithms have been implemented, there is no such large difference between their execution times, we must mention that the BeagleBone is a board that runs an OS within itself, in this case Debian, so it performs the processing related to the OS, as well as the processing of the algorithms executed at the same time.
If processes are not very complex, considering speed and ease, Tiva-C can be a very good option. If more complex processes have to be executed, BeagleBone can be used.

Conclusions
As it has been presented throughout this paper, neural algorithms for processing data are widely used in different types of projects, many of them carried out in systems that have high processing and storage power. However, the new schemes of communication and interaction between electronic devices through the internet, require that the obtaining, processing and transmission of information be carried out in devices that can be portable, such as the different development boards that exist in the market. One of the main disadvantages of these development boards is their limited processing and storage capacity. However, such boards can be efficiently used to execute algorithms with few data.
In this paper we have presented the implementation of the Perceptron, ADALINE and Hopfield neural network algorithm in the Arduino UNO, Tiva-C and BeagleBone development boards, to measure their execution times and determine which of the boards presented in this study will be used as the best option for the generation of communication and processing systems when you have a limited amount of data.
If the comparison of the Arduino board with Tiva-C is made, it can be seen that the Tiva-C LaunchPad development board presents a better response at the execution time of the implemented algorithms, better technical characteristics, and has the same programming ease when the environment Energia is used.
In the carried out experiments, it was found that the Tiva-C development board is a good option for the design of projects that may be within the IoT scheme, since it has a much shorter execution time compared to the Arduino UNO, currently the most used.
The BeagleBone board has a longer processing speed with respect to the Arduino UNO card, however, BeagleBone processes the calculations related to the implemented neural algorithms, and the Debian OS with graphic interface at the same time, ie , its computational resources are not only destined to the processing of the input data, but also they are destined to the OS.