Automatic Image Annotation for Small and Ad hoc Intelligent Applications using Raspberry Pi

The cutting-edge technology Machine Learning (ML) is successfully applied for Business Intelligence. Among the various pre-processing steps of ML, Automatic Image Annotation (also known as automatic image tagging or linguistic indexing) is the process in which a computer system automatically assigns metadata in the form of captioning or keywords to a digital image. Automatic Image Annotation (AIA) methods (which have appeared during the last several years) make a large use of many ML approaches. Clustering and classification methods are most frequently applied to annotate images. In addition, these proposed solutions require a high computational infrastructure. However, certain real-time applications (small and ad-hoc intelligent applications) for example, autonomous small robots, gadgets, drone etc. have limited computational processing capacity. These small and ad-hoc applications demand a more dynamic and portable way to automatically annotate data and then perform ML tasks (Classification, clustering etc.) in real time using limited computational power and hardware resources. Through a comprehensive literature study we found that most image pre-processing algorithms and ML tasks are computationally intensive, and it can be challenging to run them on an embedded platform with acceptable frame rates. However, Raspberry Pi is sufficient for AIA and ML tasks that are relevant to small and ad-hoc intelligent applications. In addition, few critical intelligent applications (which require high computational resources, for example, Deep Learning using huge dataset) are only feasible to run on more powerful hardware resources. In this study, we present the framework of “Automatic Image Annotation for Small and Ad-hoc Intelligent Application using Raspberry Pi” and propose the low-cost infrastructures (single node and multi node using Raspberry Pi) and software module (for Raspberry Pi) to perform AIA and ML tasks in real time for small and ad-hoc intelligent applications. The integration of both AIA and ML tasks in a single software module (with in Raspberry Pi) is challenging. This study will helpful towards the improvement in various practical applications areas relevant to small intelligent autonomous systems.


Related Work
The state-of-art Machine Learning (ML) algorithms are contributing to several sectors of human life through Big Data [01]. However, explosive growth of Big Data generates several types of Big Data, in which visual data (images) is much informative and complex to perform analytics using ML algorithms. Image data requires more systematic pre-processing steps (Image Annotation) for analytics then non-imaging data [02]. In the literature, Image Annotation techniques can be categorized into three (3) different approaches.
1. Text-based approach 2. Content-Based Image Retrieval (CBIR) approach 3. Automatic Image Annotation (AIA) Text-based approach manually annotates (through human) the images and hence inappropriate in the current digitization scenario [3][4]. Content-Based Image Retrieval approach automatically retrieves and index  [5][6]. The need for large-scale image dataset annotation introduced the concept of Automatically Image Annotation (AIA) [7][8][9][10]. The AIA technique contains the good characteristics (advantages) from both traditional (text based and CBIR) annotated techniques through the keyword searching based on image content. In AIA, the semantic concept model automatically learns from the large number of visual data. Several studies already discussed AIA advantages over traditional approaches [11][12][13][14]. . Few recent surveys, also discussed the cutting edge Deep Learning approaches for AIA [15][16]. The issue of AIA is it requires a massive (expensive) infrastructure to annotate the large-scale images in real time because these approaches mainly use high computational systems Therefore, in this study, we proposed a cost-effective AIA approach to generate and annotate the massive amount of dataset.

Research Motivation
The current Machine Learning(ML) architecture is centralize in nature for real-time AIA and ML task. Typically, centralized server is responsible to execute all AIA and ML tasks and forward response (output) to point of action. This approach is quite optimal when using a moderate or large type of Intelligent Systems, as shown in Fig. 1. Beside high cost this approach also upfront security-related issues too [17]. On contrary, for the small and ad-hoc intelligent systems, we need quick and on spot, data pre-processing to perform intelligent tasks. Therefore, there is need of automatically annotate data (images) on spot (for creating a database) and utilize them for quick prediction or classification. In this study, we present the idea of real time AIA for ML tasks using Raspberry Pi. We propose single node approach for representing the small autonomous intelligent systems and multiple nodes integration approach towards decentralized distributive small and ad-hoc intelligent systems (which could behave as a single network) to perform some high computational processing jobs through object migration, as shown in Fig. 4. 2 AIA for small and ad-hoc intelligent applications.

Research Methodology
In this research study, our aim is to propose a novel approach for small and ad-hoc based intelligent system architecture through cost-effective and less computational resources. However, our main concern is to develop the Automatic Image Annotation Systems which will capture the real-time images and automatically annotate them for ML analytics. To achieve our required goal, we followed below objectives; Along with a systematic literature review (to achieve the first objective). This study will follow Design Science (DS) research process to achieve the goal. Design Science is being considered as the most efficient process for the development of successful models or artifacts.
Proposal and evaluation of model can include a variety of algorithms, techniques, methods, tools or framework, which could create various hindrances in terms of strategy, creation, and control which could be avoided by adopting Design Science process.    Fig. 02 shows the single node approach, using a small camera (for capturing real-time images of the natural environment), attached with a small Raspberry Pi (hardware). This cheap and efficient hardware infrastructure can capture and annotate the real-time images automatically through the machine learning algorithm. For that purpose, we design the software module (for each raspberry pie) as shown in Fig. 3. This software is responsible for performing basic and advanced level annotation task using a machine learning algorithm (3 channel pixels to matrix conversion, object identification, and object labeling). Fundamentally, this software is based on two major components;

Automatic Image Annotation Module 2. Image Analytic Module
The Automatic Image Annotation module is capable to receive the raw image data (real-time capturing through the camera) and automatically convert it into an annotated format (by tagging and indexing). Annotated image transfers to the image repository. Image repository keeps track of all the indexed and tagging images with their respective labels to train Image Analytic Module, as shown in Fig. 3. The single node performs some less computational processing tasks due to its hardware limitations. However, multiple single nodes may share their computing resources and form a decentralized distributed network environment to perform some high computational ML tasks, as shown in Fig. 4. These architectures will be less expensive, fast and feasible for on spot decision making.

Conclusion and Future Work
Instant preparation (annotation) of image dataset from the real-time data collection is critical and cost-effective in typical AIA (centralized) approach. Whereas the decentralized approach possesses the potential to substantially reduces the cost (through low-cost devices) and distributes the computational processing (through multiple nodes) among various small units (single nodes). Interestingly, the proposed small ad-hoc intelligent systems contain the capability to run the machine learning algorithms to perform automatic image annotation and classification. Certainly, the further advancement in these individual small and ad-hoc intelligent systems towards the sharing of computational resources (through migration of objects) can make it feasible on the situations where large-scale infrastructures are necessary. In this paper, we present the idea and propose the framework for Automatic Image Annotation for small and ad-hoc intelligent application using Raspberry Pi. In the future, we will implement the proposed approach to validate this approach.
In the implementation phase, we will design the necessary tools and applications (for Raspberry Pi) using MATLAB (Machine Learning toolbox). For evaluation purpose, we will first implement the proposed solution in a single Raspberry Pi based intelligent node as shown in Fig. 2. (For validating the autonomous behavior of a single node). Later we will setup the Raspberry Pi intelligent network with multiple intelligent nodes, as shown in Fig. 4. (To validate the combined behavior of multiple nodes).