Design and implementation of children’s gesture education games based on AI gesture recognition technology

In order to cultivate children's imagination and creativity in the cognitive process, combined with the traditional hand shadow game, a children's gesture education game based on AI gesture recognition technology is designed and developed. The game uses unity development platform, with children's digital gesture recognition as the content, designs and implements the basic functions involved in the game, including AI gesture recognition function, character animation function, interface interaction function, AR photo taking function and question answering system function. The game is finally released on the mobile terminal. Players can recognize gestures through mobile cameras, interact with virtual cartoon characters in the game, watch cartoon character animation, understand popular science knowledge, and complete the answers in the game. The educational games can better assist children to learn digital gestures, enrich children's ways of cognition, expand children's imagination, and let children learn easily with happy educational games.


Introduction
With the rapid development of video games, serious games, which mainly teach knowledge and skills, provide professional training and simulation, are gradually favored by researchers and applied in many fields, such as military, medical, industrial, education, scientific research, training and so on. Serious game is a branch of video games. This kind of game does not take entertainment as the chief object, but adopts the game form of edutainment, so that users can acquire some information, training or treatment in the process of the game [1][2][3].
In serious games for children, good human-computer interaction is the key of this kind of game research [4][5]. At present, in human-computer interaction, gesture recognition as a simple and intuitive way, and with the development of technology, recognition accuracy is gradually improving. Integrating gesture recognition technology into children's serious games can not only enrich the interaction mode of children's games, but also inherit the traditional gesture games with the help of video games, so that children can enjoy the interesting gesture culture while simulating gestures [6][7][8][9].
This paper designs and implements children's gesture education games based on AI gesture recognition technology, which combines AI gesture recognition technology with children's serious games. It can not only innovate the interaction mode of children's serious games, cultivate children's imagination in the cognitive process, but also inject high and new technology into children's education and enrich the form of children's education.

Game design
The design of children's gesture education games based on AI gesture recognition technology is mainly divided into three segments. One is the design of game content, which is mainly based on the collected gesture recognition data in the early stage to complete the design of game level. The other is the design of game function, including the design of gesture recognition function, animation playback function, model interaction function, video playback function, AR camera function and question answering system. In addition, there is the design of human-computer interaction in the game.

Game structure and flow char
The structure of the game is shown in Figure 1, and the flow chart of the game is shown in Figure 2.

Human computer interaction design in games
x Interactive design of game interface. When the player clicks the audio playback button on the start interface, there will be audio playback. After the touch and click of the element pattern on the game interface, there will be corresponding animation effect. When entering the game, there will be a text reminder, which plays a guiding and prompt role in the playing process, which is a reflection of interaction. x Interaction design of gesture recognition. Gesture recognition technology itself belongs to an important field of human-computer interaction. In this system, when players make different gestures, the system will match the gesture shape with the information in the gesture recognition library set in the system, and display the result of gesture recognition on the interface. According to the result of gesture recognition, the character animation related to gesture is generated [10]. x Interactive design of game character model. After the character model appearance, users can click on different parts of the model to trigger different animations of the model, so that players can gain fun and surprise in the process of experiencing the game x Answer interactive design. In the game interface, when player completes a gesture, the circle outside the gesture will be colored, indicating that the experience has been completed. When the user completes all the gestures, the system will pop up a prompt on the interface, whether to enter the question and answer game, the player can choose to continue gesture recognition, or enter the segment of question and answer game. After entering the answer level, the player uses finger touch to drag the options in the answer interface to complete the answer.

Design and implementation of gesture recognition function
After entering the game, the user needs to make a specific gesture under the camera. The game starts to recognize and display information according to the user's gesture. The prompt panel of the user's previously selected gesture will appear on the interface. The user only needs to make the corresponding gesture according to the prompt. The prompt panel disappears, the hand shadow appears, and the animation starts to play. Based on unity game engine, SenseAR plug-in is used to realize gesture recognition. The key codes are as follows.

Design and implementation of AR camera function
After the model appears, the camera button is displayed on the interface. Click the button to take photos of the current display interface. Let users record the moment with photos when they interact with the model. The system automatically refreshes the photo album and stores it in the mobile phone photo album to meet the user's requirements. AR photographic effect, as shown in Figure 4.

Design and implementation of question and answer game function
After the user completes the ten gestures set by the system, the interface will prompt whether to enter the Q & A game interface. When the user enters the question and answer MATEC Web of Conferences 355, 03043 (2022) ICPCM2021 https://doi.org/10.1051/matecconf/202235503043 game function interface, the user needs to drag out the animal model corresponding to the hand shadow from the hand shadow image on the interface to collide with the obstacles in the question, so as to answer the correct question. After answering all the questions, the game ends.
In the game, if all the answers are not correct within the specified time, the interface will pop up. If the answer is right or wrong, the corresponding prompt interface will pop up. And produce the corresponding particle effect and sound effect.The effect of correct answer prompt interface is shown in Figure 5.

Game experience results
In order to verify the experience effect of children's gesture education games based on AI gesture recognition technology, five preschool children were selected to experience the games with the help of their parents. In the process of experience, five children can learn with the help of their parents, complete the imitation of gestures, interact with cartoon animal characters in the game, watch story videos, and complete the answers in the game. According to the performance of five children in the experience game and the feedback from their parents, we get the following experimental results. x Children's gesture education games based on AI gesture recognition technology can strengthen the communication between parents and children, promote emotional exchange and understanding, and make parents better understand children's psychology. x The interactive way of gesture recognition has a certain attraction to children, which is conducive to children's experience of the fun of the game and the deepening of the memory of the content of the game. x There are few kinds of gestures in the gesture recognition library, and the gesture model needs to be further expanded.

Conclusion
With the development of serious games, educational games with rich interactive forms will play an important role in children's education. Children's gesture education games based on AI gesture recognition technology, combined with gesture recognition technology, enrich the interactive form of children's education games. The game integrates traditional digital gestures with game elements, through the role animation, interface interaction, question answering system and other technologies in the game, helps children comprehend the implication of different gestures while experiencing the game. According to the results of the actual game experience, it is confirmed that this kind of game plays a certain role in attracting children's attention and cultivating children's imagination.
I would particularly like to acknowledge my game studio members, for their wonderful collaboration and patient support.