A novel electronic gate that identifies and counts bees based on their RGB backscattered light

In this work, we present an electronic gate that aims to extract a deeper representational signal of the color characterization of the main body of an insect, namely: a) we record the backscattered light and not the extinction light as commonly done, b) a color sensor analyses backscattered light to individual RGB channels independently to grasp the melanization, microstructural and color features of the wing and body of the insects passing the gate. We present all the necessary details to reproduce the device and we analyze many insects of interest like the bee Apis mellifera and the wasp Polistes gallicus. The electronic gate is attached to the entrance of the beehive and counts foraging activity. The backscattered light intensity can quantify the size of the incoming insect and discern a drone and a worker bee from a queen bee while the color measurements aim to recognize invasive species so that the gate closes and the beekeepers are alerted.


Introduction
For the past 25 years, European beekeepers have been reporting weakening bee numbers and colony losses and the situation is worsening. According to the EU Reference Laboratory for honeybee health, some countries in the EU are losing up to a third of their colonies every year [1]. This problem also afflicts North American countries and Asia. At present, the honeybee population in the USA is less than half of what it was in the 1940s [2]. Honeybees are essential in the pollination of many agricultural crops. In [3] it is estimated that in EC the number of beekepers is 700.000, keeping around 15 million hives and honey production is estimated to be close to 200.000 tons/year. Apis mellifera is the only managed honeybee species in Europe providing food, which has been domesticated by beekeepers to produce beekeeping products [4] (honey, pollen, wax). When wild bees do not visit agricultural fields, managed honey beehives are often the only solution for farmers to ensure crop pollination. Indeed, A. mellifera remains the most economically valuable pollinator of crop monocultures worldwide [5]. The direct value of honey produced in the EU is estimated about €140 million [6], while the value of insect pollination for European agriculture has been estimated to be around €20 billion per year, €153 billion worldwide [7]. Pollinator declines result in loss of pollination services, that have negative ecological and economic impacts significantly affecting the maintenance of wild plant diversity, wider ecosystem stability with potential knock on effects on crop production, food security and human welfare. There is clear evidence of recent declines of managed bee colonies, correlated with parallel declines in the plants that rely upon them [8,9]. To help prevent the decline of bee numbers we support the effort of developing IoT sensor applications that can automatically assess the health and threat status of colonies. Forager traffic is related to food availability and demand [10] and therefore, it is an important variable to monitor. Sudden changes in the traffic level indicate a possible threat at the colony level. Forager activity is described in terms of the number of bees entering/exiting the hive over time. The bee counter is a device that fits over the entrance of a beehive [11][12][13][14]. Bees pass through any one of a series of adjacent tunnels. Each tunnel is equipped with light beams, one at the entrance and one at the exit of each tunnel. The direction of the bee's travel is denoted by the order in which these beams are broken. The novelty of this work is that the bee counter we present decomposes the backscattered light stemming from the insect's main body into Red-Green-Blue (RGB) channels and capture the colour of the incoming insect. Therefore, the novel gate is able to assess forager activity, quantify the size of the insect based on the intensity of the backscattered light and discern insects based on their coloration. When integrated with wireless communication abilities, the system will wirelessly transmit health and threat status results to a cloud server, making data open for running prediction models and risk assessments, issue warnings and make historical analysis. This will allow beekeepers and public authorities to be active participants in colony surveillance programmes, overcoming labor costs of manual inspections. As a result, unhealthy or threatened colonies will be remotely detected in time with greater precision and treated accordingly.

Materials and Methods
The LED emits white light that includes the three basic colors (RGB). The color sensor decomposes backscattered light to individual RGB channels independently. The backscattered RGB color can capture differences in the coloration of insects' main body and wing veins of incoming insects and backscattered light intensity can quantify the size of the incoming insect. The coloration of the insect can reveal different aspects of insects' characteristics. Two identical insects with differences only in the melanisation of the wings and coloration of the main body will produce different reflected light patterns. The reflected light stemming from insects, relates to the refractive index of the wing membrane and the glittering of the insect [15,16]. Different spectral bands carry complementary information on the insect's main body and wings coloration [17,18]. In Fig. 1 we see the main components of a single cell that monitors one tunnel of the e-gate. The LEDs and the receiving photodiode are on the same side so that when the light is emitted, it is subsequently backscattered by the main body of the insect and received from the photodiodes of the color sensor in the center. The photodiodes have several filters that decompose the white light of the LEDs into different spectral bands corresponding to the RGB light.
In Fig. 2 . 3). The I2C bus requires two lines, the serial clock (SCL) and serial data (SDA). In one I2C bus it is possible to connect many devices with different address. In our case, all devices (colour sensors) have the same address therefore, it is not possible to connect the devices in the same bus. For this reason we made a modified I2C bus.    In Fig. 6 one can see the recorder and a single channel for better visibility. It also has an analogue to digital converter (ADC) for each photodiode, and, therefore the design is simplified. The communication with the main processor takes place through the I2C Bus. The colour sensor is located in the upper part of the each cell in the lid next to the LED so that it reads the reflected light.
Regarding the software, the embedded microprocessor runs a constantly looping program that processes data captured by the sensors. The board is programmed in C/C++. The data from the colour sensors is copied to sixty two circular buffers. The first thirty one buffers are used to monitor the backscattered colour values using a window of 64 samples for each colour. The other thirty one circular buffers of 256 samples each (256 samples of each colour), store the recoding of each gate. If any of the first thirty one buffer exceeds a common threshold, it triggers the recording process for the corresponding gate. The recordings of the signal are coded in 16-bit resolution. The first 20% of the samples are drawn before and up to the triggering point and 80% after that point in order to ensure that the data of an event is not lost. The software is written in C language using the IAR Embedded workbench. The programming of the flash memory was carried out using the ST-Link V2 programmer. The code initialization was done using the STM32CubeMX of ST. For programming the peripheral sub-components such as the SD & I2C bus we made use of the STM32 HAL Drivers.

Results and Discussion
In order to monitor the presence and density of bees and wasps in particular, as well as design policies and apply measures, one augments beehives with sensors so that their health status is assessed remotely [19][20][21][22][23][24]. This work belongs to a broader context of applications that relate to automated insect surveillance of insects of economic and hygienic importance [17][18]. Forager traffic is closely linked to colony food intake and to pollination, it is particularly useful variable for researchers, beekeepers and growers. Monitoring hive traffic prior to pollination would allow beekeepers to observe hive health and a record of quality control [10]. Hereinafter, we present and examine preliminary results by placing dead bees, wasps, beetles and flies on a rail and passing the rail through the sensor. A large evaluation plan in vivo will be carried out in the near future but for experimenting with and tunning the system and because of the difficulty of performing experiments with life specimens. Regarding the insect specimens, we collected the insects A. mellifera and P. gallicus from the area Gouves, Chersonisoss Crete, on February 2019. The bees have been found dead around apiaries while wasps have been terminated in situ with acetone and all insects transported to an entomological laboratory.  . 7. The gate is first tested with dead specimens of bees, wasps, beetles and various type of flies. The movement process is simulated by placing them in a rail that is passed through the gate to get the backscattered light from the sensor.
All insect specimens (bees, wasps, beetles and flies) are placed on a rail that passes through the sensor. The backscattered light is recorded and stored to the SD card of the recorder (see Fig. 6). Hereinafter, we present and examine preliminary results that look very promising and lead the way for large in-vivo assessment.
In Fig. 8 we see four different bees passing through the gate as depicted in Fig. 7. We observe that the light intensity shape has a repeatable shape among different bees and the ratio between bands holds a consistent pattern. In Fig. 9 we observe four cases of the wasp P. gallicus. One can observe the repeatability of the light intensity pattern regarding wasps as in the case of bees and that the blue colour is lower than the green and this is not the case for the bees. Therefore the ratio of intensity between colours changes consistently for different insect species. We will pursue this direction and try for different species using different specimens than the ones used in Fig. 8 and Fig. 9. In Fig. 10 the first two peaks correspond to the pattern of the backscattered light stemming from a bee and the subsequent two belong to wasps. Note that, by observing Fig. 8 and Fig. 9 we could classify correctly the cases in Fig. 10. In Fig. 11 we see the differences between a bee, a wasp and a bumblebee. Moreover in Fig. 12 the beetle and the black coloured domestic fly have completely different patterns that than of bees and wasps.
The sensor system presented hosts the optoelectronics signal acquisition set-up, which for being multispectral will be optimized to facilitate a reliable registration of the presence of bees and beehive pests, without the possibility of having false alarms due to ambient interferences. The design and set-up is optimized with respect to various technical aspects (data quality, signalto-noise ratio, etc.)     To our point of view, optical sensors are the suitable choice for use in electronic insect gates and automatized insect traps working in the field because they record intermittently, i.e. on per event basis, and only if their probe volume (that can be shaped with proper lenses) is interrupted in contrast to the continuous recording of microphones. Microphones receive continuous input from an uncontrolled and unknown number of audio sources in the field and are not generally suitable for field applications. The proposed multispectral sensors do not require the bandwidth of a vision camera and do not face the difficulty of a photograph of a pile of insects that are not easily discernable in detail. Non-technical considerations (cost, feasibility for mass production, endurance of materials, etc.) is also in favour of optical solutions. Using the sensor presented in this work, walking insects (e.g. bees and wasps) are efficiently detected and their presence is registered in the power of low frequencies around the DC level. The power level of the received light is suitable to rank insects according to their size. Multispectral signatures look richer than the ones provided by simple one-band sensors but their advantage on classification improvement needs to be clarified and quantified with large-scale experiments (see [25][26][27] for related work).