Region of Interest Selection Interface for Wide-Angle Arthroscope

We have proposed a new interface for an wide-angle endoscope for solo surgery. The wide-angle arthroscopic view and magnified region of interest (ROI) within the wide view were shown simultaneously. With a camera affixed to surgical instruments, the position of the ROI could be determined by manipulating the surgical instrument. Image features acquired by the A-KAZE approach were used to estimate the change of position of the surgical instrument by tracking the features every time the camera moved. We examined the accuracy of ROI selection using three different images, which were different-sized square arrays and tested phantom experiments. When the number of ROIs was twelve, the success rate was best, and the rate diminished as the size of ROIs decreased. The experimental results showed that the method of using a camera without additional sensors satisfied the appropriate accuracy required for ROI selection, and this interface was helpful in performing surgery with fewer assistants.


Introduction
As minimally invasive surgery has become popular, the demand for arthroscopic surgery has also increases.Contrary to patient preferences, most surgeons would agree that video-endoscopic surgery causes more stress and task-related difficulties than open surgery [1].
The most common complication of arthroscopic surgery is the frequent arthroscope maneuvers due to the small field of the arthroscopic view.Due to the narrow field of the arthroscopic view, a camera assistant is required to manipulate the arthroscope consistently within the joint to provide a wider image of the arthroscopic surgery to a surgeon.Therefore, the camera assistant and surgeons spend a long time keeping pace with each other to solve communication problems.
Various studies on an endoscope system with a wide FOV have been conducted to overcome the drawbacks caused by the conventional endoscope with a narrow FOV.There are two primary approaches: One is attaching the supplementary camera to the existing endoscope [2,3], and the other is improving the optic system of the traditional endoscope with a wide-angle lens, prisms, or movable lenses [4,5].Communication problems between a camera assistant and surgeons can be reduced by utilizing a wide-angle arthroscope, because wider arthroscopic images make the surgery possible without frequent arthroscope maneuvers.
However, even though the wide-angle arthroscope is affixed to a manipulator, an operating surgeon who holds the surgical instruments in both hands requires additional assistants during surgery.Because the size of the region of interest (ROI) becomes small in the image of the wideangle arthroscope, additional aids for selecting the ROI within the wide-angle image and zoom-in functions are necessary.
Therefore, we have developed a hands-free ROI selection interface for a wide-angle arthroscope for solo surgery.An wide-angle endoscopic image for the intuitive perception of global anatomical structures is obtained by the wide-angle arthroscope, and a region of interest is magnified by digital zoom for a detailed operative view (Fig. 1).The position of the ROI is movable by handling the surgical tool itself without releasing the instrument from the surgeon's hands.The movements of surgical instruments with marker sensors attached are mostly tracked by optical or magnetic trackers in the computerassisted surgery utilizing surgical navigation.Optical tracking provides high resolution and accuracy, but it is expensive.The cost of a magnetic tracker is lower than that of an optical one, but the accuracy diminishes with distance [6].Thus, we attached a camera to the surgical tool to estimate the position changes and controlled ROI position in real time through image processing steps.

ROI selection interface for wide-angle image
An enlarged image from a wide-angle arthroscope could provide more information about the operative space to surgeons than a conventional scope, which is beneficial for understanding the camera position in the patient's body and improving hand-eye coordination.However, showing only one wide-angle image is insufficient, because the main image that surgeons generally need for surgery is concentrated on the target organ.Therefore, we have developed an interface in which both the wide-angle image and the ROI image that is selected by handling the surgical instrument itself are displayed simultaneously.

ROI selection method
Several devices (a motion recognition system, a foot pedal, and a tablet mouse) have been taken into account to select an ROI within the wide-angle image.However, these are not suitable, because both the motion recognition system and tablet mouse need additional hand motions or an assistant to operate the device, and a foot pedal was already employed to turn on the surgical electrocautery or to circulate the water in the patient's body.
A method of selecting an ROI by manipulating the surgical instrument eliminates the need for releasing the surgical tool.This means that the surgical instrument is utilized as a surgical mouse.For tracking the position of the tool, we affixed a camera perpendicularly to the direction of the surgical tool, and the camera images were analyzed to identify the rotational movement of the tool.A hand switch was used to turn the ROI selection mode on or off.

Relation between surgical instrument and ROI movements
After a surgical instrument passed through a portal, the motion of the tool was determined with four kinds of degrees of freedom (DOF).The first DOF regarded forward and backward motion; the second DOF was rotation movement in a longitudinal axis; the third DOF was left and right motion; and the fourth DOF was up and down motion.The third and fourth DOFs were matched with ROI movements (up and down, left and right).We determined the relation between the surgical instrument and the camera in formula 1.From the relation, when the tool was moved along the direction of the third DOF, the camera was rotated in the Y-axis.When the tool was moved along the direction of the fourth DOF, the camera was rotated in the Z-axis.The ROI movements could be controlled by using the rotation angle (Fig. 2).

Feature extraction and matching algorithm
The motion of the instrument with a camera attached was estimated by matching the features of two consecutive scene frames.The estimation accuracy of the tool movement depended on the image processing capacity, such as feature extraction, feature matching, and essential matrix calculation.In particular, because we used the pure scene images without specific vision markers or pattern images, acquiring the features from the background image was an essential step.
In this paper, the features were extracted using the Accelerated-KAZE (A-KAZE) algorithm, which exhibits a good performance and fast multiscale feature detection and description [7].The features were matched by considering a nearest neighbor distance ratio approach [8] (Fig. 3).The rotation of the endoscope was derived by decomposing the essential matrix, which was calculated using matched feature points and camera intrinsic parameters.Our interface system was programed using C++ and OpenCV 3.0 library beta version.

Experiments
International Symposium of Optomechatronics Technology (ISOT)

Resolution test of ROI selection
In order to assess the resolution of the ROI selection, we prepared three kinds of images (640×480) with different sizes of colorful squares with a black background (Figure 4).The square was regarded as a ROI and its position was changed every 4 seconds.Success was counted when the square was selected by instrument control within 4 seconds in 20 attempts, and the experiment was repeated five times.

Phantom test
Two tasks were employed to evaluate the usability of the ROI selection interface qualitatively in phantom tasks (Fig. 5).The first task involved marking the dot point within multi-size circles in the order of numbered groups.The second task involved pulling only the blue sutures, which were mixed with white sutures, with a cannula.

Results
In the resolution test, when the number of ROIs was twelve, the success rate was perfect, and as the number increased, the success rate became lower.Standard deviations of 25 ROIs and 48 ROIs accounted for less than 5.5 % (Table.1).In the phantom test, the qualitative assessment of ROI selection interface was implemented by a surgeon who manipulated the interface.

Discussion
We have proposed the idea of attaching a camera to the surgical tool to overcome the limitations of existing trackers.This could be applied as a ROI selection method for a wide-angle endoscope interface for solo surgery.The reason of decreasing success rate with number of ROIs is that selecting the more divided ROI require the higher accuracy for selection.Although the success rates of 25 ROIs and 48 ROIs are not perfect, we expect that the resolution of 12 ROIs is sufficient for the view selection.
In the results of the phantom test, the first strength of our interface is that one surgeon can control the surgical image without other assistants, which means solo surgery is possible.When the participant performed the phantom tasks, they used only one hand.The second strength is reducing the collision ratio between the scope and surgical tool, because a wide-angle arthroscope can show a wider arthroscopic image, although a wide-angle scope is placed farther than a conventional arthroscope.
In future studies, we plan to address the ability to recognize differences between image changes by the camera movement and object movement within a camera FOV.This study could be one step toward the development of a wide-angle endoscope interface.

Figure 1 .
Figure 1.Comparison between the conventional arthroscopic view and dual view by hands-free interface.

Figure 2 .
Figure 2. Relation between movement of a surgical instrument passing through a portal, camera coordinates, and ROI movement.

Figure 3 .
Figure 3. Feature matches in consecutive image frames when a camera affixed to the surgical tool moved.

Table 1 .
Resolution test of ROI selection.