A Holographic Displaying System for Gastroscopy Diagnosis

Gastroscopy is a common procedure for diagnosing the diseases in upper GI tract, in which a large number of localized and sometimes unstructured images on different parts of the upper GI tract are captured. While it is effective, the procedure also requires experienced doctors to carefully exam these images in real time. As a result, errors may occur causing miss and/or false diagnosis. This paper presents a holographic displaying system that can display the images in a 3D structured manner. It consists of four steps. First, the gastroscopy images are acquired from the standard gastroscopy procedure. Second, a 3D image is generated by means of panoramic mapping. Third, the 3D image is transferred into four images. Finally, the images are displayed in a 4-side pseudo 3D holographic system. The new system is capable of showing 3D structured view, based on which the doctor can carry on diagnosis more effectively. With further development, it is expected that the new system will find practical applications in hospital soon.


Introduction
Gastroscopy is a common procedure for diagnosing the diseases in upper GI tract [1].Everyday millions of gastroscopy procedures are practiced.While it is effective, it requires experienced doctors to exam large number of localized and sometimes unstructured images in real time.In particularly, the limited Field Of View (FOV), usually less than 2 cm in diameter, on a 2D flat screen may create confusion and limit its effectiveness.As a result, miss and/or false diagnosis may occur.
To solve this problem, various methods have been developed.In Reference [2], a method for harvesting 3D features is proposed by means of following the flow of images and the hierarchical structure.In Reference [3], a method for generating global / local panoramic view from the gastroscopy images is described.The book [4] presents a number of other methods.Though, while the stomach is 3D, few uses holographic displays.
It is known that a holography displays a 3D image in the form of hologram.Instead of the stereo view on a 2D flat screen, it gives the volumetric view in 3D.In the past decade, a number of different holographic methods have been developed.The simplest method is the 4-side pseudo 3D holography system [5].It consists of a set of 4 holographic plates, a large monitor (or 4 small monitors), and a computer software system.The 4 holographic plates are assembled as a pyramid which can reflect the light field from the monitor(s) and generate a composite holographic image.The holographic plats reflect approximately 50% of the lights and hence, the holographic display is not very bright.Moreover, it has no images in 45 0 , 135 0 , 225 0 , and 315 0 directions, which are the connecting areas of the two adjacent holographic plates.The former can be resolved by using better materials while the later can be improved by rotating the images in small increments.An improved method is the so-called "Transpost" presented in reference [6].The idea is to project the images on a moving anisotropic screen to generate the 3D holography, though the projected image could be burry.It can also be done by means of spinning LCD [7,8].The other method is the 360 0 curved mirror display system [9,10].It uses a series of mirrors to generate the holographic image and has no moving parts.Its imaging quality is about the same as that of the Transpost.In recent years, the interactive feature has also been added [11,12,13].In short, although the existing holography methods still need much improvement, the futuristic holographic display is very helpful for gastroscopy diagnosis as it can give a 3D structured information.
This motivates us to develop a holographic display system for upper GI tract diagnosis.The rest of the paper is organized as follows: Section 2 briefly describes the conventional gastroscopy procedure.Section 3 shows the building of the 4-side pseudo 3D holography system.Section 4 shows the implementation of the new

The Gastroscopy Procedure
The cancer in the upper GI tract, or more specifically the stomach cancer, is the second most common cancer in the world, especially in Asia [14].To encounter this terrible disease, early diagnosis is the key.Presently, the most reliable method for diagnosing the stomach cancer is gastroscopy.As shown in Figure 1, a gastroscope is a thin, flexible telescope that passes through patients' mouth, esophagus and gets down towards the stomach and duodenum.The gastroscope usually contains a light and a camera by which the doctor can see the inside of the upper GI tract.Upon starting the procedure, the doctor can capture images and/or videos from different positions / viewing angles, based on which the doctor can detect possible cancer sites and make diagnosis decisions.It is noted that the images may contain structural information, such as pylorus (as shown in Figure 1).Though, because the gastroscope only has limited FOV, usually about 2 ~ 5 cm in diameter, many images do not have structural information.As a result, the doctor may not be able to get the full picture of the whole stomach.Consequently, miss or false diagnosis may be made.To address this problem, we propose to use the holographic display system.

The Holographic Display System
As discussed in Section 1, a number of different holographic display systems have been developed.In this paper, we use the 4-sided pseudo 3D holographic display system.The 4-sided pseudo 3D holographic display system consists of a set of 4 holographic plates, a large monitor (or four small monitors), as well as a software system.The four holographic plates are assembled as a pyramid.Its basic operation principle is the light refraction.The holographic plates are made of acrylic sheets.When the image is displayed in the monitor(s), the light field enters the holographic plate.As a result, the light refraction occurs.Because of the refractive index of acrylic sheet, the light field partially refracts to the center of the pyramid, at which the 4 images are fused forming the pseudo 3D image.Figure 2 shows the light field directions of the 4-side pseudo 3D holographic display.The 4 acrylic holographic plates are easy to make and assemble as shown in Figure 3.This makes the 4side pseudo holographic display rather inexpensive.In order to drive the 4-sided holographic display system, a computer software system is needed.The computer software system would read the images and generates a fused image to display in the monitor(s).This can be done using a number of different ways such as MATLAB® and Adobe Photoshop®.In this paper, we use MATLAB®.The procedure is as follow: 1. Read the images; 2. Sort the images; 3. Generate the four view images, each view is 90 o apart; and display the four view images on the monitor(s) An illustration example is shown below.The object is a half stomach model.First, four images are taken from front (0 0 ), left (90 0 ), back (180 0 ) and right (270 0 ) respectively.Next, the images are sorted and displayed on a monitor as shown in Figure 4.Then, adding the 4sided holographic display on the monitor, the holographic image is shown as presented in Figure 5.One can also take more images from different angles, for example, 0 0 , 45 0 , 90 0 , 135 0 , …, 270 0 , and 315 0 .Then, by display the images alternatively; a moving holographic display can be generated.As discussed in Section 1, in the gastroscopy procedure, a large number of images are acquired.These limited FOV images are usually taken in a sequence.In order to generate the holographic display, the key is to fuse the images and generate a panoramic image, and then generate the 4 view images.There have been a number of methods for generating a panoramic image from multiple images [16].In this paper, we adopt the invariant feature method.This method uses the invariant features of the images to stitch the images together to generate the panoramic image.It does not the ordering and orientation of the images and hence, is easy to implement.Its procedure is as follows [17]: 1. Load images; 2. Detect and match features for each image; Calculate and run geometric transformation; 3. Register successive images; 4. Compute the minimum and maximum output limits for transformations to calculate the size of panorama; and 5. Create panoramic image by mapping.
Two demonstration examples are shown below.In the first example, the object is a bottle with a painting inside as shown in Figure 6.First, a number of images are taken by rotating a camera.Next, a panoramic image is generated as shown in Figure 7.Note that at the two ends of the image, there is an overlap, which shall be removed to ensure no redundancy.Then, the four view images are created as shown in Figure 8. Finally, the holographic image is displayed shown as presented in Figure 9.In the second example, the object is a stomach model.First, as illustrated in Figure 10, in diagnosing the upper GI tract, the gastroscope shall go through a specific trajectory, along which a large number of images are acquired covering the entire stomach.Figure 11 shows a sequence of images.Note that the model stomach is made of two pieces and hence, for simplicity we use a simple camera to the images of the two pieces separately.As a result, two panoramic images are obtained as shown in Figure 12.In practice, when using the gastroscope, it could be just one, like that of the first example.Figure 13 shows the two view images and Figure 14 shows the holographic display.Note that it maps the entire stomach inside out and hence, is very easy to read.

Conclusions and Future Work
This paper presents a holographic system for displaying the gastroscopy information.Based on the discussions above, following conclusions can be drawn.
(1) The presented 4-side pseudo 3D holographic system can effectively show the image of the entire stomach in 3D.It eliminates the ambiguity from looking at a large number of images with limited FOV on a 2D display, and hence, is valuable for diagnosing the upper GI tract.(2) The system is rather simple and can be easily integrated to the existing gastroscopy system for clinical applications.
Currently, the presented system is tested only on simple subjects such as a bottle and a stomach model.In the near future, ex vivo testing using a pig stomach will be carried out.The future work also includes automating the entire procedures, improve the quality of the holographic display and adding the user interactive control.

C
Owned by the authors, published by EDP Sciences, for upper GI tract diagnosis.Finally, Section 5 contains the conclusions and future work.

Fig. 2 :
Fig. 2: The light field direction of the 4-side pseudo 3D holographic display system

Fig. 4 :Fig. 5 :
Fig. 4: The 4 images of the half stomach model displayed on a monitor

Fig. 13 :Fig. 14 :
Fig. 13: The two view images of the stomach model