Rapid Automatic Lighting Control of a Mixed Light Source for Image Acquisition using Derivative Optimum Search Methods

. Automatic lighting (auto-lighting) is a function that maximizes the image quality of a vision inspection system by adjusting the light intensity and color.In most inspection systems, a single color light source is used, and an equal step search is employed to determine the maximum image quality. However, when a mixed light source is used, the number of iterations becomes large, and therefore, a rapid search method must be applied to reduce their number. Derivative optimum search methods follow the tangential direction of a function and are usually faster than other methods. In this study, multi-dimensional forms of derivative optimum search methods are applied to obtain the maximum image quality considering a mixed-light source. The auto-lighting algorithms were derived from the steepest descent and conjugate gradient methods, which have N-size inputs of driving voltage and one output of image quality. Experiments in which the proposed algorithm was applied to semiconductor patterns showed that a reduced number of iterations is required to determine the locally maximized image quality.


Introduction
Intermediate inspection processes to find defective products in semiconductors and flat panel displays (FPDs) have become more stringent because the loss cost when a product is found to be defective in a final process is greater. Image-based inspection using machine vision is currently widespread and the image quality is critical in automatic optical inspection (AOI).The image quality depends strongly on the lighting, which is adjusted such that it is maximized. Image quality is usually evaluated according to deviation in pixels of the inspected image, which is called sharpness. The lighting properties are intensity, color or spectrum, angle, uniformity, polarization, and collimation. Intensity control is widely used, and this function is currently called automatic lighting (auto-lighting). Chen proposed an active lighting control function that uses fuzzy for a robot manipulator. [1] In most inspection machines, the light intensity from a single color source is varied from 0 to maximum in small intervals in an equal step search (ESS), and then the maximum image quality can be determined during the variation. [2] Auto-lighting using a mixed color source generates a mixed light by adjusting multiple color inputs. A mixed color source can enhance the inspection performance, in particular, for agricultural products. [3] Optimal illumination can increase the color a e-mail: htkim@kitech.re.kr b e-mail: yong00@kaist.ac.kr c e-mail: stkim@kitech.re.kr d e-mail: jongseok@kitech.re.kr e e-mail: kcjin@kitech.re.kr f e-mail: memslee@kitech.re.kr contrast in machine vision. [4] An RGB mixing source was used to align a surface mounting device in Pedro's study. [5] In addition, spectral approaches have been applied for bio-medical research. Park constructed a fivecolor illumination system to detect artificial drawings on humane skin, [6] an endoscope has been developed with a mixed light source that allows discrimination between the skin and blood vessel, [7] and Wang proposed an optimal illumination system for surgery, which was verified using a pig's heart. [8] These spectral approaches were aimed at reproducing the optimal combination of light intensities determined in manual experiments, and thus, they are not easily applied to apply AOI systems. When using ESS, the number of iterations usually amounts to a few hundred when using a single color source, and therefore, the number of iterations required when using a mixed-color source is large. The number of iterations can be reduced by employing optimum search methods for a mixed light source, as well as for a single light source. [9,10] Derivative optimum search methods are well-known, simple, and easy to implement. [12] The steepest descent (STD) and conjugate gradient (CJG) methods were derived for arbitrary N color sources and image quality, that is, sharpness. Experiments to validate the proposed auto-lighting algorithm were conducted using an RGB source and a machine vision system. The results show that the auto-lighting is practically applicable for inspection systems in manufacturing processes. The inspection system considered in this study comprises a mixed light source, industrial camera, framegrabber, controller, and digital-to-analog (DAC) board. Figure 1 shows a typical construction of a mixed light source and a machine vision system. Different color LEDs are driven by electrical current which is proportionally amplified using the voltage level from the DAC board. Lights from the different color LEDs are directed into a mixing chamber -with an optical mixer, from which they are emitted as a mixed light. The mixed light is directed at a target object and the industrial camera acquires a digital image. The digital image is stored in a controller and the voltage level of the DAC board is determined after optimum search process has been conducted. The voltage level has N dimensional vectors V because they are N color LEDs: The steps in the procedure from current amplification by V to camera image acquisition involve electric, spectral and camera responses. This procedure cannot easily be described using a mathematical model, and therefore, it is reprensented by a black box function, f . The respective pixel on the image coordinate (x,y) in the camera acquires a grey level, a digitized value of the light intensity detected in a pixel, I. Then the grey level can be obtained from the voltage level as shown.
Image brightness constitutes the average grey level of an image and the image quality can be evaluated by the variation in the grey level. The image quality is usually evaluated according to sharpness, which is calculated from the grey level difference among the pixels. The sharpness, [13] denoted by σ for an m × n pixel image, is defined as: Optimum search methods find minimum of a cost function; however, for the purpose of this study, maximum sharpness is required. Thus, the negative sharpness is applied for a cost function: find min ρ = −σ for ∀V (4)

Derivative Optimum Search Methods
The derivative methods calculate the tangential vector of a test point and follow the direction of a curve and a surface, as shown in figure 2. The tangential vector is obtained from a gradient as : The derivative methods are iterative and we denote the number of the current iterations by k. Thus, we represent the current voltage level by k V, negative sharpness by, k ρ, and the gradient by, k c. The direction vector d is defined as in the derivative methods and the negative gradient d = −c is used in the case of STD. Next the test point is calculated using a step size coefficient α: α is determined at ∂ρ(α)/∂α = 0, which is difficult to compute when the cost function is complex. In practical cases, the step size is usually replaced with a convergence constant η. The CJD calculates the direction vector using : The next test point may be located at a far point as a result of numerical error, and therefore, the step size should be limited using a boundary value τ. [11] The iteration is terminated when the gradient is reduced to a small value as shown in : where is a small value for the terminal condition.

Experiment and Results
The sharpness and derivative methods were applied to a test system which was constructed in our previous study. [9] The test system was composed of a 4M pixel camera, coaxial lens, CAMLINK framegrabber, multi-channel DAC board, and RGB mixing light source. EPROMs were used as sample targets A and B (EP910JC35 and Z86E3012KSES) ,as shown in the figure  3(a) and (b). The ESS method was implemented to find the global maximum; the total step number of combinations for RGB was 50 3  The paths shaped curve lines, which is compared to direct and non-differentiation optimum search methods. Although the maximum sharpness value determined by the derivative methods was a little lower than that determined by ESS, the number of iterations was much smaller. Therefore, the proposed auto-lighting algorithm can reduce the number of iterations, while the image quality remains almost the same. Figures 6 and 7 show the color variation in chromatic diagram during a search using the proposed optimum search methods. Figure 6 shows the color variation obtained when the STD method was applied. The search path of pattern A moves toward the left at the beginning and then changes direction toward the bottom left, while that of pattern B starts in a forward right direction and then approaches a final value at the bottom-right. The optimal colors determined using the STD method were close to blue for pattern A and red for pattern B. The movement is in one direction, and there are no oscillating curves in the paths. Figure 7 shows the results of the CJD. The shapes of the moving paths produced by CJD are similar to those by STD. The reason for this is considered to be that the methods are composed of similar equations. The color coordinates at the final values of CJD closely approach those produced by STD.

Conclusion
In this study, derivative optimum search methods were applied to maximize the image quality of a machine vision system in the shortest time, thus providing a function called auto-lighting. In our system, the image quality is defined as the standard deviation of the grey level in pixels of an inspected image and is proportionally adjusted using multi-color LEDs using the voltage level from a DAC board. The voltage level is determined using STD and CJG methods. The STD and CJG methods are used to determine the near maximum sharpness in a reduced number of iterations. Our experimental results indicate that the application of the proposed method in image-based inspection machines can be useful and practical.