Deep convolutional neural networks for cardiovascular vulnerable plaque detection

. In this paper, an accurate two-stage deep learning method is proposed to detect vulnerable plaques in ultrasonic images of cardiovascular. Firstly, a Fully Convonutional Neural Network (FCN) named U-Net is used to segment the original Intravascular Optical Coherence Tomography (IVOCT) cardiovascular images. We experiment on different threshold values to find the best threshold for removing noise and background in the original images. Secondly, a modified Faster R-CNN is adopted to do precise detection. The modified Faster R-CNN utilize six-scale anchors (12 2 ,16 2 ,32 2 ,64 2 ,128 2 ,256 2 ) instead of the conventional one scale or three scale approaches. First, we present three problems in cardiovascular vulnerable plaque diagnosis, then we demonstrate how our method solve these problems. The proposed method in this paper apply deep convolutional neural networks to the whole diagnostic procedure. Test results show the Recall rate, Precision rate, IoU (Intersection-over-Union) rate and Total score are 0.94, 0.885, 0.913 and 0.913 respectively, higher than the 1 st team of CCCV2017 Cardiovascular OCT Vulnerable Plaque Detection Challenge. AP of the designed Faster R-CNN is 83.4%, higher than conventional approaches which use one-scale or three-scale anchors. These results demonstrate the superior performance of our proposed method and the power of deep learning approaches in diagnose cardiovascular vulnerable plaques.


Introduction
Vulnerable plaques are unstable plaques which prone to form thrombus. They closely related to the occurrence of acute cardiovascular events such as thrombosis, acute coronary syndrome, coronary heart disease and even sudden death. About 19 million patients die each year due to acute coronary syndrome (ACS) and sudden cardiac death, patients who suffer from myocardial infarction caused by the rupture of unstable coronary atherosclerotic plaques account for 70% of the total number [1,2]. Therefore, accurate detection of cardiovascular vulnerable plaques is significant in preventing cardiovascular diseases.
Developed in recent years, Optical Coherence Tomography (OCT) is a new coronary artery imaging technology both at home and abroad. OCT not only has a resolution rate of 10μm, 10 times the resolution rate of IVUS, but also has the ability to identify organizations (e.g. the main components and characteristics of plaque can be showed in OCT images [3]), hence Intravascular Optical Coherence Tomography (IVOCT) has become preferred method in clinical detection of vulnerable cardiovascular plaques.
In recent years, with the fast development of deep learning and computation power, computer aided diagnosis has been applied to some medical fields, making medical image diagnosis more effective [4]. However, they yield some distinctive drawbacks when it comes to cardiovascular vulnerable plaques detection: 1. Cardiovascular vulnerable plaques adhere to the inner wall of blood vessels, which are more difficult to detect than common object detection targets [5].
2. IVOCT cardiovascular images belong to ultrasonic image. Owing to the imaging process of ultrasonic image, image contents and the interference of instrument catheters, IVOCT cardiovascular images have a lot of speckle noise [6].
3. Medical images require manual annotation by professional physicians, which is more difficult than common object annotation. Besides, the number of available images is limited.
In order to solve these drawbacks mentioned above, we propose a novel cardiovascular vulnerable plaque detection method using deep convolutional neural networks to detect vulnerable plaques. In the proposed method, we first use a Encoder-Decoder structure neural network named U-Net to segment the IVOCT cardiovascular image dataset, removing background and instrument catheters. Then, state-of-the-art object detection framework Faster R-CNN is applied to detect vulnerable plaques. We experiment different threshold values on U-Net and choose the threshold which has best segmentation performance. We also design six-scale anchors for cardiovascular vulnerable plaques detection, so it takes account of a wider range of plaques' size than conventional methods which use one-scale or three-scale anchors.
We evaluate our method on CCCV2017 Cardiovascular OCT Vulnerable Plaque Detection Challenge dataset. The Recall rate, Precision rate and IoU (Intersection-over-Union) rate are 0.94, 0.885 and 0.913 respectively, higher than the 1 st team of CCCV2017 cardiovascular OCT vulnerable plaque detection challenge. AP of the designed Faster R-CNN is 83.4%, higher than both conventional approaches and the 1 st team of the challenge. Thus the method presented in this paper has great value for clinical application.

Methodology
In this section, we first give a description of data preprocessing and data augmentation, then describe the two-stage deep learning method presented for cardiovascular vulnerable plaques detection. In stage one, we use U-Net to do segmentation, removing background and instrument catheters in the original images. In stage two, we apply Faster R-CNN to do precise detection. The framework of our proposed two-stage deep learning algorithm is illustrated in figure 1.

Data preprocessing and data augmentation
The dataset used in this paper is provided by the CCCV2017 cardiovascular OCT vulnerable plaque identification challenge official 1 . The dataset consists of 2000 images, with 1000 positive samples and 1000 negative samples, the size of each image is 720*352 pixels. Ground Truth of the dataset are manually labeled by pathologists. We rotate, translate, shear and zoom the images in a proper range to enlarge the dataset using Keras [7].

Formatting cardiovascular image segmentation based on U-Net
Image segmentation is a crucial step in medical image diagnosis. It can remove unrelated background, instrument catheters and other irrelevant information from the original images, only keeping body organs or tissues which will be detected. Image segmentation is often the first step of computer aided diagnosis algorithm, preparing for subsequent accurate detection [6].
We mentioned in introduction that there are three problems in cardiovascular vulnerable plaque detection, in order to solve these problems, we compare several approaches and adopt U-Net to do segmentation. Proposed by Olaf Ronneberger et al in 2015, U-Net is a Fully Convolutional Neural Network based on encoder-decoder architecture [8]. U-Net won the first place in the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks in 2015 and has been popular since then. Though U-Net has a relatively simple structure, it's able to have an excellent performance even on small datasets [8]. Besides, U-Net has been proved to have better performance on medical image segmentation than any other currently existing convolutional neural networks [9,10].
In this paper, we use U-Net to segment IVOCT cardiovascular images. The insight of U-Net is to classify every pixel in a given image. Suppose the segmentation task require pixels to be classified into N class, when the label of original image is made, each pixel of label should belong to one of the N classes. In the presented method, we classify pixels into two classes. One is human cardiovascular area, the other is unrelated background and instrument catheter area. All labels used in this paper were made by pathologists, they sketched the contours of human cardiovascular area and filled unrelated area into black. Figure 2 gives an example of IVOCT image and it's label. Set p to represent an arbitrary pixel in an image, we normalize p to get p using the equation: . Set threshold range as : , we set the label as 1, otherwise the label is 0. Notice that threshold value is directly related to the quality of segmentation results, we make several experiments adjusting threshold values to find the best value. The curve of threshold and validation loss is shown in figure 3. From figure 3. we can observe when 2 . 0  threshold the presented method attain the minimum loss on validation set. We further illustrate the results of U-Net segmentation using different threshold in figure 4.

Precise cardiovascular plaque detection using faster R-CNN
In this section, we present how we utilize state-of-the-art detection algorithm Faster R-CNN to detect vulnerable plaques in IVOCT cardiovascular images which have been operated by U-Net. In detection algorithm, vulnerable plaque regions are marked with rectangle boxes, coordinates of the four corners can be obtained from the label. Since vulnerable plaques vary greatly in size, we utilize six-scale anchors (12 2 ,16 2 ,32 2 ,64 2 ,128 2 ,256 2 ) and compare our six-scale approach with the conventional onescale and three-scale approaches.
Faster R-CNN algorithm consists of two modules: 1. Region Proposal Network.
Second, in order to train the RPN network, we set class labels for each candidate area object} non {object, . The label settings for RPN network are described as follows: Object: candidate regions which have overlap ratio larger than x with arbitrary Ground Truth bounding boxes ) ( x IoU  .
Non Object: candidate regions which have overlap ratio equal or smaller than ) 1 ( x  with all Ground Truth bounding boxes.
Since RPN network extracts a large number of proposed boxes and there are lots of overlapping regions between these proposed boxes, we use NMS (Non-Maximum Suppression, NMS) to merge and delete proposed boxes. To be specific, all candidate boxes are sorted by score and box with the highest score is selected, then we delete boxes which have overlap ratios larger than ) ( x IoU x  . We then sort proposed regions and take top N proposed regions for detection.
Third, After RPN network got the proposed regions, Fast R-CNN is used to realize the final detection and identification. We train RPN network and Fast R-CNN to let them share the convolutional layers of ResNet101 [13].

Experimental results
The goal of CCCV2017 Cardiovascular Vulnerable Plaque Detection Challenge is to detect cardiovascular vulnerable plaques in IVOCT images precisely and instantly. Performance of detection algorithms are evaluated by 3 indicators, they are Precision rate, Recall rate and IoU (Intersection-over-Union) rate. We evaluate our presented two-stage deep learning method on the official dataset of CCCV2017 Cardiovascular Vulnerable Plaque Detection Challenge. Let TP be the positive class which the algorithm also predicts as positive class, FP be the negative class which the algorithm predicts as positive class, FN be the positive class which the algorithm predicts as negative class, g S demotes the area of Total score: S=0.5*(2*P*R/(P+R)) + 0.5*IoU We compare our detection algorithm with other submitted algorithms, the result is illustrated in table 1 and table 2. Table 1 shows the performance comparison of top3 team and us, we can see our method has the highest Precision rate and IoU rate, thus we get the highest total score. Table 2 compares AP value of our method and top3 team. To better demonstrate the performance of our presented method, we draw ROC curve of our method as it shown in figure 5.

Conclusion
In this paper, we propose a novel cardiovascular vulnerable plaque detection method based on deep convolution neural networks. In the first stage we experiment on different thresholds to attain the best segmentation result, in the second stage we design an improved Faster R-CNN which has six-scale anchors to do precise detection. Experimental results on the CCCV2017 Cardiovascular Vulnerable Plaque Detection Challenge demonstrate the proposed method has superior performance than the 1 st team of the challenge. The Recall rate, Precision rate, IoU rate and Total score of our method are 0.94, 0.885, 0.913 and 0.913 respectively, AP of the designed Faster R-CNN is 83.4%, higher than conventional onescale and three-scale approaches. We believe our proposed method would be a very powerful tool for clinical cardiovascular vulnerable plaques diagnosis.