Automatic Puncture System Based on NIR Image and Ultrasonic Image

In view of modern clinical treatment and care, venipuncture has a very important position, but in order to train a medical staff with high level skill of puncture need to spend a lot of time, human and material resources. For this reason, in this paper, an automatic system that can achieve venipuncture is proposed, which can replace the medical staff and achieve the puncture process. The automatic venipuncture system, combined with NIR imaging and ultrasound imaging technology, the overall plane information is obtained by NIR image, depth information and fine positioning of the plane information is obtained by ultrasound image. By way of image processing, the NIR image is enhanced and segmented, getting the spatial location of the vein. And the most suitable blood vessel segments were selected by piecewise straight fitting. Then the location ultrasonic probe should be placed is determined. Then the ultrasound image with the blood vessel in the vicinity of the longitudinal centreline can be obtained. All the information of blood vessel have been obtained, then the machine can be driven to achieve venipuncture.


Introduction
Venipuncture as one of the basic methods of clinical treatment, is of important significance in the present medical examination and treatment.It is a critical step in blood sample collection, infusion and interventional therapy.In order to ensure rapidity and accuracy, medical staff should go through long-term training to get acceptable level of venipuncture.To obtain high level of puncture technology, at least several years of clinical training is needed.And for patients with no obvious blood vessels (such as obese patients), it is difficult to puncture successfully.[1] However, an auto puncture system, can be quickly applied in the clinical fields once in mass production.Despite of saving manpower, it also has the advantage of avoiding reduce human error from the medical staff and other external interference.
Currently the clinical products related to puncture are mainly auxiliary equipment, which enhances the contrast, so that the medical staff can find the blood vessels easily.As infrared projection equipment and ultrasound-guided peripherally inserted central catheter (PICC).[2][3][4][5][6][7][8][9] In the laboratory research, has been able to carry out venous puncture automation equipment.In 2006, Western University successfully developed an automatic puncture device using ultrasound guidance similar to the PICC, however, different from the PICC, this system used machine instead of manpower to insert a needle.[10]In 2013, Rutgers University successfully developed a Nearinfrared(NIR) and ultrasound-based automatic venipuncture system, which allows the puncture needle into the blood vessels automatically, [11]and in 2015 a comprehensive upgrade of the system has been realized.[12] 2 System design Venipuncture in this paper is not only aimed to collect blood samples, but also aimed to administrate intravenous infusion.Therefore, after the needle has entered the blood vessel at an angle of about 45°, it is necessary to adjust the direction of the needle so as to coincide with the direction of the blood vessel and proceed a predetermined distance.This requires the needle can turn around the tip after it has entered the blood vessel in the mechanical design.Also, the blood vessel segment selected for puncture should be longer than the puncture needle, and it should be as straight as possible to avoid being pierced by the needle.As shown in Figure 1, the system obtains the plane information of vessel by NIR camera and then move the B-ultrasonic with the puncture unit fixed together onto the vessel obtained to get the depth information, completing the three-dimensional positioning of blood vessels.At this moment, the vessel is under the center of B-ultrasonic probe, the relative position of the needle to the vessel is shown by the dotted

Unit design 3.1 NIR unit design
Near infrared spectroscopy is an imaging technique using the spectrum which wavelength range of 750nm~1500nm.
Compared to other bands, NIR has a strong ability in penetrate human tissue, the distance can be up to 10mm.[13] And as shown in Figure 2, the absorption characteristic curve for the NIR of different material, [14] it can be seen that when the wavelength range is 760nm~850nm the absorption coefficient of reduced hemoglobin reaches a peak.Considering the spectral characteristics of NIR camera and other effects, eventually using a wavelength of 850nm.The CRG value is defined as the following equation Where ∆x represents the width of single pixel, and N V represents the count of pixel that a vessel model contained.Figure 4 shows the original image with the result from CRG algorithm and histogram equalization.It shows that the vessel information obtained by CRG algorithm is more obvious.After obtained the centre line of vessel, it should be identified that which vessel segment is the most suitable one for puncture, it contains the following features: 1) Length: longer than the needle; 2) Straightness: needle can go through without touching vessel wall.Based on the constraints above, a piecewise linear fit was made to the found centerline of the blood vessel (the segment length is greater than the length of the needle), and then the centerline segment with the largest correlation coefficient is selected as the destination location for puncture.As shown in Figure 5, linear correlation coefficient is 0.996398, the maximum value in the same length line segments.

Ultrasound unit design
Using an external image acquisition box, the ultrasonic signal acts as an external camera input, so the ultrasound image can be called directly for a real-time processing.Unlike NIR, ultrasound imaging is a contact imaging process.Coupling agents are required to make the ultrasonic probe in contact with human tissue intact.So as to introduce ultrasonic waves into the human body to obtain an echo image of the human body tissue.When the ultrasonic probe is operated manually, medical staff can find the suitable location, according to their experience.So that it is possible to introduce the ultrasonic wave into the human body tissue without causing the extrusion deformation.Due to different measurement targets have different surface shapes, when using automation equipment, easily lead to poor ultrasound signal or squeeze the blood vessels that can not be identified.As shown in Figure 6.
Because plane position information has obtained by NIR unit, compared with feedback control using sensors, increasing the thickness of the coupling agent to make the unevenness of the skin surface compensated is more suitable and simple.However, considering the fluidity of the coupling agent, how to concentrate it between the ultrasound probe and the skin is a tricky problem.The method presented in this paper is that placing a soft rubber film on the ultrasonic probe, in order to perfect fit with the skin.Ultrasound can be good import without squeezing the skin.The operation process and the acquired image is shown in Figure 7.After obtaining the ultrasound image, the following work is extracting vessel information.Since the plane location has been got by NIR image, the ultrasound image can be processed relatively easy, as following: 1) Based on the position and width of the blood vessel obtained from the infrared image, combining with the error of aligning the ultrasound probe with the vessel, determining the transverse position range in which the blood vessels appear in the ultrasound image.2) Find the boundary between the skin and the coupling agent, and the vessel position is below the boundary.3) In the previously determined interval, through the threshold segmentation method to find the darkest point at the top.4) Output vessel coordinates.

Conclusion
This paper designed a venipuncture system to perform vessel localization combining NIR imaging with ultrasound imaging, and proposed methods in image acquisition and processing.Compared with the existing research results, the trajectory of puncture needle is more in line with clinical practice.However, considering the conditions of laboratory, we were unable to carry out a complete validation test, some problems may not be exposed.In addition, during the selection of the final puncture of vessel segment, we only consider the straightness of the center line, the width, direction, and bifurcation were not considered.In the follow-up work, these problems will be further investigated in order to find a resolution.

Figure 2 .
Figure 2. The absorption characteristic curve for the NIR of different material

Figure 3 .
Figure 3.The visible light image (left) and the NIR image (right)

Figure 4 .
Figure 4.The origin image (left), the CRG model image (middle) and the histogram equalization image (right)

Figure 5 .
Figure 5.The centre line (left) and the target vessel segment (right)

Figure 6 .Figure 7 .
Figure 6.The condition that vessel been squeezed(left) and poor ultrasound signal(right)