Development of Intelligent Terminal-based Signal Acquisition System for Cochlear Implant

. Present speech recognition of cochlear implant is still low for situation of noisy environment or under mismatch condition, and more researches focus on improving front-end signal acquisition and speech recognition. To simplify signal acquisition and algorithm research, we develop an intelligent terminal-based signal acquisition system for cochlear implant, in which the electric relay and many sensors are adopted to implement system monitoring function. The proposed system platform is helpful to actualize algorithm research and intelligent monitoring, adding to its value of further research of speech recognition improvement.


2System design and function
The proposed hardware system adopts intelligent terminal as control center and information processing center, which is connected with the input ports through network router and Zigbee. The output ports use electric relay to communicate with intelligent terminal, and the transmitting hardware can send modulated signal to cochlear implant for algorithm research. Connection diagram of each function module of the system is presented in figure 1. In figure 1, the voice signal, video signal and other information are collected through sensors, and are further transmitted to intelligent terminal through network router and Zigbee. The recorded video signal is used in real-time monitoring, and voice signal is used for subsequent signal processing in front-end signal acquisition and further algorithm processing for cochlear implant. Intelligent terminal collects and stores information, and then implants control instructions, and use electric relay to control corresponding output ports. In signal acquisition and algorithm research, intelligent terminal modulates the recorded signal and is connected with the cochlear implant through transmitting hardware, aiming to develop new speech processing strategies. 3System control and parameter setting

System software control and setting
Serial communication of electric relay includes 1 start bit, 8 data bits and 1 stop bit, with default initial address of 0x01. The serial communication uses switch to choose RS485 interface or RS232 interface, and ModBus-RTU communication protocol is used in this communication. Control instructions of electric relay, reading state, controlling all electric relays, controlling one electric relay, reading switch data and setting device address are 0x01, 0x0f, 0x05, 0x02 and 0x06 respectively, and the CRC check includes 2 bits, containing CRCH and CRCL.
For reading the output status of electric relay, exemplified by reading the electric relay with address of 0x01, the required data for sending is 0x01, 0x01, 0x00, 0x10, 0x00, 0x04, 0x3c and 0x0c. To realize CRC checking, a checking algorithm needs to embed into the desired position, based on the crc>>1, crc^=0xa001 and other shift operation to achieve CRC checking.
In the RS485-based data communication, the host computer sends data into slave device by a fixed address, and the data format is listed in the following sequence: address of slave device, function code, start address, number of registers, number of byte, data 1 for storage, data 2 for storage and CRC checking code. And corresponding return data format from slave device is: slave device address, function code, start address, number of registers and CRC code.
In this system, RS485-based data communication interface uses dual-wire (A+ and B-) shielded twistedpair cable. This twisted-pair cable adopts half-duplex working mode and Modbus communication protocol. In this communication mode, for reading multiple register data (for example, reading 2 register data from slave device), data needs to be sent from the host computer, and the corresponding format is listed as follow: address of slave device, function code, start address, data length and CRC code. Correspondingly, return data format is: address of slave device, function address, bite number of return data, register 1, register 2 and CRC code.
The beeper module connects with the electric relay, and the corresponding connect address is 0x01 in the output 1 position in the open interface of electric relay. To open this beeper module, instruction of "AT+AA_RS485TX=01050010FF008DFF<CR>" needed to be sent from the intelligent terminal into the electric relay.
Network camera in this system connects with computer by network router, which is accessed through LAN network. Search software "BSearch_cn.exe" is required to find the local network camera and set its device address. Detailed setting procedure is summarized in the following steps: find in the search software, choose the desired network camera in the LAN network, modify the camera IP information, input the user name and password, information update and finally choose the desired camera after updating. When finishing this setting steps, picture and video information can be captured.
In this system, wireless communication adopts Zigbee to send and receive data. When the ISP switch in the Zigbee communication node is turned on the programming position and the serial debugging tool is open, we can achieve the communication module setting, data transmission and parameters acquisition.
In addition, to facilitate speech training and other rehabilitation training for CI recipients and simplify the information contract with control room, an external call telecontroller is added. Working frequency of this telecontroller is 315 Hz, which can send 8-bit binary data, with more than 100 meter effective distance in open environment or long distance in room environment. When the telecontroller button is press down, instruction of "AT+AA_C315=address of telecontroller" is sent to the intelligent terminal. The wireless network router in this system provides five work modes of AP, Client, Repeater, Bridge and Router, with 150 Mbps transmission rate.

Signal coding and transmission to CI inner coil
Desired signal is modulated by CI speech processing strategy and then a set of stimulating pulse sequence with signal amplitude information and other parameters are generated. Information of electrode parameter, stimulating mode, pulse width parameter and parity bit check code together needs to be combined and sent to CI inner coil wirelessly. All the parameters can be constructed to a completed stimulating sequence as a transfer data frame, which is coded and then sent to CI coil. Coded signal transmission rate in this system is 10 Mbps, based on PWM (Pulse Width Modulation) coding mode.
For the wireless transmission of coded data in CI device, a fixed frame structure is required. A 36-bit frame structure is needed in CI wireless transmission to CI coil, which contains 1-bit frame head, 3-bit stimulating mode, 12-bit stimulating amplitude, 8-bit pulse width, two 1-bit parity bit check codes and two 5bit electrode parameters. Frame head is the start position of one data frame, setting to be 1. And data sequences of the aforementioned 3-bit stimulating mode are 000, 010, 011, 100, 101 and 111, corresponding to six work modes of synchronous frame mode, single-electrode general mode, single-electrode superimposed mode, dual-electrode general mode, dual-electrode superimposed mode and test frame mode respectively. There are two electrode parameters in this 36-bit frame structure, for single-electrode mode, parameter of Electrode 1 needs to be transmitted, but for dual-electrode mode, both information in Electrode 1and Electrode 2 need to be transmitted.

UI display interface and feature analysis
To simplify the algorithm research in CI speech processing strategy and provide researchers intuitive display for desired signal feature, mobile phone-based UI program is embeded in the research platform. Signal recorded by microphones and modulated signal by CI speech processing strategy can both be extracted their parameters and features for UI display. The proposed UI platform allows researchers obtain intuitive signal feature and other effective signal parameter to analyze the algorithm performance in CI speech processing strategy.
To analyze the signal, microphone records signal from the loudspeaker (voice material: "Beautiful", American English pronunciation). The 16-channel CI CIS speech processing strategy is adopted in the following signal analysis. Autocorrelation method was applied to extract signal fundamental frequency (F0). The range for F0 is between 50 and 500 Hz, and the signal time-frequency spectrum and F0 curves are shown in figure  2.  Comparison of the spectrum of original signal and processed signal, energy distribution changes obviously. In panel (a-2), signal concentrates its energy at the 16 frequency positions; contrastively, in panel (a-3), at a fixed frame, energy concentrates at one position. In pitch curve, zero-value amplitude means no fundamental frequency at the corresponding position. In panel (b-1), amplitude of F0 curve for original signal is fluctuant, which means signal fundamental frequency changes in real time. But in panel (b-2), F0 curve for modulated signal with all channels selected is almost straight, which means signal fundamental frequency is an approximate constant. As F0 represents voice pitch, for situation of modulated signal with all channels selected, CI users can only obtain one pitch, which results in pitch distortion. For daily communication, it would be difficult for CI users to distinguish male voice and female voice. Comparison of panel (a-3) and (b-3), correlation between the pitch changing and spectrum distribution can be observed. However, the pitch curve and its changing trend in panel (b-3) is different from the those in panel (b-1), which shows that modulated signal with channels continuous interleaved sampling would distort the pitch information.
Energy intensity is also calculated to analyze the detailed signal feature, which is an important parameter to characterize signal real-time energy changing at each data frame, as shown in figure 3.    (c)). The comparison of intensity curves in panel (a) and (b) presents that energy intensity of modulated signal with all channels selected almost matches that of original signal. The coincident results in energy intensity shows that, CI users can accurately perceive sound intensity changing, as the modulated signal sent to CI devices can match the desired signal in real-time energy changing. However, when comparing the curves in panel (a) and (c), the energy intensity feature is very different from each other. As signal in panel (c) is the modulated signal with channels interleaved sampling, and the channel with most energy will not always be selected. Therefore, the modulated signal with all channels selected matches the original in the energy intensity feature, but the modulated signal with channels interleaved sampling distorts the energy intensity feature.
Except for signal feature of waveform, spectrum, pitch and intensity, UI platform was constructed to display and analyze more feature and parameters. The proposed platform is helpful in developing new speech processing algorithms in further research.

4Conclusion
We propose an intelligent terminal-based signal acquisition system for cochlear implant, and adopt electric relay and many sensor interfaces to improve monitoring functions. The proposed system platform is helpful in algorithm research and intelligent monitoring, adding to its value in subsequence research of speech recognition. Actual speech signal is recorded through this platform and is further analyzed to obtain parameters of spectrum and energy intensity, aiming to verify the system performance and parameter feature of signal acquisition and speech processing strategy.