Closed-Loop Autofocus Scheme for Scanning Electron Microscope

. In this paper, we present a full scale autofocus approach for scanning electron microscope (SEM). The optimal focus ( in-focus ) position of the microscope is achieved by maximizing the image sharpness using a vision-based closed-loop control scheme. An iterative optimization algorithm has been designed using the sharpness score derived from image gradient information. The proposed method has been implemented and validated using a tungsten gun SEM at various experimental conditions like varying raster scan speed, magniﬁcation at real-time. We demonstrate that the proposed autofocus technique is accurate, robust and fast.


Introduction
For high accuracy during manipulation tasks or micro-nanoscale measurements under a scanning electron microscope (SEM), high quality and sharp images are always required.For this purpose, an efficient and reliable SEM autofocus algorithm has to be executed before the manipulation process.In general terms, autofocus is a process of maximizing the image sharpness by regulating the device focus sets.There are two types of autofocus techniques: active methods, using a different subsystem to modify the lens position and passive methods, which solely rely on the image sharpness information.Out of the two, passive methods are commonly employed for microscopic devices.Since the geometry and projection model of a SEM are different to optical systems [1,2], the autofocus process is different.Most of the autofocus methods are based on evaluating the image sharpness score i.e., the score should reach a single optimum of a selected sharpness function at the in-focus image.For this purpose, many sharpness criteria such as image variance, autocorrelation, wavelets, Fourier transform were discussed for microscopic applications [3].A comparison of these criteria regarding electron microscopy was discussed in [4].
To perform passive autofocus process with SEM, a former method is to obtain a sequence of images within a defocus range and to compute their sharpness scores.The optimal SEM focal length that corresponds to the maximum of sharpness score is then obtained [5].The main drawback in this approach a e-mail: le.cui@univ-rennes1.fr is that it requires the acquisition of many images, which is practically time consuming due to the high focus range of SEM.Alternatively, a later method is to start with an initial set of SEM imaging parameters that correspond to a defocus image.Then an iterative algorithm is used to search for the best focus position [6,7].Even though these methods are effective, they are highly dependent on the search history.Rudnaya [8] has proposed to use Nelder-Mead method for searching the optimum of image variance.An alternative method has been proposed in [9], based on fitting the sharpness function to a quadratic polynomial approximatively using some initial measurements.In [10], the autofocus has been achieved by computing the derivative of sharpness function numerically.Finally, statistical learning-based autofocus methods were studied for SEM [11], but were never implemented in real-time.
In this paper, we consider the autofocus issue as a control problem and propose a direct closed-loop control scheme to solve it.The objective is to control the device focal length (working distance) iteratively based on the time variation of the gradient information of acquired image.An analytical formulation of the relation between the displacement of the working distance and the variation of the gradient information is proposed.The considered method advances the available methods in different ways.First, It overcomes the problem of hysteresis since it is independent of the search history and directly reaches the optimal focus position.Next, the method automatically circumvents unnecessary defocus positions that makes it faster than the search-based techniques.Finally, since analytic formulation of derivative has been computed, the algorithm is more robust w.r.t numerical computation-or regression-based method.We deem these important, if the method needs to be integrated with any other real-time tasks.
2 Background on SEM Focusing

SEM Focusing Geometry
In general, the SEM images are formed by scanning a sample surface by means of a focused beam of high energy electrons.Different sets of electromagnetic lenses in the SEM electron column are responsible for performing the focusing task.The first are the condenser lenses that control the beam diameter and the second are the objective lenses that focus the spot sized beam on to the sample surface.Apart from them, an objective aperture is present in between them to filter out the non-directional electrons.The distance measured electronically between the final pole piece of the objective lens and the focal plane is the electronic working distance W (focal length), which plays a vital role in the focusing process.This distance depends on two factors: the beam acceleration voltage and the current passing through the objective lens.In this work, we assume the former remains constant and the main focusing is performed only using the latter.The total focusing process is illustrated in the Fig. 1(a).For any selected magnification, at a distance D 2 on both sides of the focal plane, the beam diameter is two times pixel diameter.This results in the images that look to be acceptably in-focus.

SEM Image Formation
The general image formation model, which is commonly used in the case of optical microscopes [12], can be extended to use with a SEM [11].Considering an extremely small area element dxdy centered on (x, y), the secondary electron (SE) current emitted from this area is ds(x, y) = δ(x, y)ϕ(x, y)dxdy, where, ϕ(x, y) is the incident current density at the point (x, y), δ(x, y) is a yield coefficient which is as- signed in a way that δ is the average number of resultant secondary electrons emitted.The total SE current emitted from the specimen s is, Considering an approximative linear relation between emitted SE current s and the result signal i (i.e.i(x, y) = ks(x, y), k is a constant), the SEM image formation can be seen as a linear convolution of a specimen-dependent component and a systemdependent point-spread function (PSF) [5,11].Here, the PSF is the scaled and reflected electron beam current density passing through the origin.Considering this, the result image i(x, y) can be expressed as the convolution of an in-focus image i * (x, y) with a defocus kernel h(x, y) and is given by Previous studies state that, a Gaussian kernel can be used as an approximation to the defocus kernel [13].
In this case, the probability density function (PDF) is given by where, σ is the standard deviation of the Gaussian kernel.

Closed-Loop Control Scheme for SEM Autofocus
As mentioned before, autofocus can be achieved by computing the maximum value of the sharpness function (see Fig. 1(b)).In the proposed approach, the autofocus process is regarded as a closed-loop control problem where the focal length of SEM is updated at each iteration until the optimal focus is reached (see Fig. 2).In this section, we will first show how to use the image gradient for a closed-loop control scheme (in case of parallel projection).Then we derive the control law to perform the autofocus task.

Sharpness Function for Closed-Loop Control
Previously, it has been shown that the gradient-based sharpness measures perform well with electron microscope imaging [4,9].The underlying reason to use image gradient is that it shows a good compromise in the case of unstable image contrast, which has to be considered with SEM [14].Besides, in [15] it has been proposed to use image gradient information to perform vision-based positioning tasks and a similar process could be envisioned for an autofocus task.However, due to the parallel projection model at high magnification which is inherent to a SEM [2], the perspective projection-based control law proposed in [15] cannot be employed for the SEM.To tackle this problem, we propose in this section a direct projection model-free approach to derive the control law for full scale autofocus.
Considering the fact that the image gradient varies when the image focus changes i.e., when the working distance varies, we aim to update the working distance by a closed-loop control law to obtain the maximum of image gradient.For an acquired image i(x, y), the squared norm of the gradient g(x, y) at a point (x, y) is expressed by where, ∇i 2 x (x, y) and ∇i 2 y (x, y) represent the squares of gradient in x and y directions, respectively.Considering the squared norm of the gradient G for a whole image (size M × N ) as the sharpness function to be maximized by varying W , we have where Here instead of extracting any local features, a global feature G is defined over a whole image.Fig. 3 shows the variation of image gradient G for a series of SEM electronic working distances.In fact, it is not necessary to take into account any local information(i.e.shape, texture and uniformity of the sample), since the image gradient for most of the pixels increases (or decreases) when changing working distance, resulting a smooth sharpness function which has one evident optimum (at the desired working distance).Considering the autofocus task as a closed-loop control law, the objective is to compute the variations of the working distance Ẇ from the variation of G for achieving an optimal SEM focus set.
In order to employ image gradient information as the sharpness function for full scale in a SEM, the relation between the temporal variations of working distance W and image gradient G is considered: The Jacobian J G that links the variations of working distance W and image gradient G in ( 8) can be expressed by where, σ is the standard deviation of Gaussian kernel given in (4).For a small displacement of W , considering a proportional relation between σ and W , ∂σ ∂W = k, leading to From ( 7), ∂G ∂σ can be expressed as 2(∇i x (x, y) ∂∇i x (x, y) ∂σ Next, (3) can be rewritten as follows: From (4), we get the derivative of the Gaussian kernel: Considering ( 12) and ( 13), we have and Replacing ( 14) and ( 15) in (11), we can finally compute J G , which is used to derive the control law.
The Journal's name

Control Law
The objective of our approach is to maximize the G by controlling the working distance W to obtain an optimized focus of SEM.In order to maximize G, we are going to minimize a cost function given by where, α, β ∈ R + are adaptive gains that control the variation of working distance and the speed of convergence.γ is a small positive value that can be considered as a threshold to determine if the optimal focus is reached.The algorithm will stop when αe −βG(W ) γ. Considering an exponential decrease of the error i.e., ε = −λε, the control law is: where, ξ is the velocity along the focal axis and J ε is the Jacobian and can be expressed by Rewriting ( 17) using ( 18), leads to Subsequently, the W displacement (working distance) to be set with the SEM has been computed as follows where, ∆t is the time taken between the two acquired images.For each iteration, the working distance is updated as given by ) where W new is the working distance to be updated, W prev and W 0 are previous and initial working distances, respectively, |∆W | is the magnitude of ∆W , W max = 50 and W min = 1 are the factory provided maximum and minimum values for the electronic working distance (in mm) of the employed SEM, respectively.In our experiments, (21) is used to control the direction of displacement computed by the control law.For the initial working distance close to a middle value between 1 and 50, according to the single maximum in the evolution of image gradient with respect to working distance (see Fig. 3), the direction can be obtained by comparing G(W 0 ) with G(W 0 + dW ), where dW is a small change in working distance.

Experimental Set-up
In order to validate the proposed method, different experiments have been realized.Fig. 4 shows the experimental set-up architecture used for this work.The SEM used is a Jeol JSM 820 tungsten gun SEM equipped with a conventional Everhart-Thornley SE detector.Its electron column is equipped with different sets of electromagnetic lenses and an objective aperture strip containing 4 changeable apertures of different diameters.The magnification of the SEM varies from ×10 to ×100, 000 and the maximum allowable electronic working distance is 50 mm.A beam control and image acquisition system, DISS5 (from point electron GmbH) has been interfaced with the microscope.It is mainly responsible for sending the scan parameters to SEM and to acquire the data coming from SE detector.Later this data is amplified, digitized and saved as an image in the computer to which the DISS5 is connected.All the autofocus experiments are performed in this computer using the SE images of size 512 × 512 pixels and are monitored using the developed special purpose graphical user interface program.Besides, DISS5 provides a user interface control for the device focus by linking the working distance with a range of focus steps i.e., each step corresponds to a specific working distance.For real experiments with the system, a model given by ( 22) has been obtained by approximating the curve using least squares fitting.This model will be used to compute the corresponding focus step for a working distance given by (21) to modify the device focus.
where, p i=1...6 are the coefficients of the model and F is the focus step of the SEM.As the acceleration voltage used to excite the electrons vary the focusing model, for each voltage used in this work, a corresponding model has been derived.However, for the experiments, the voltage is fixed for all the tests performed with a specific sample.

Validation of the Method
Initial test is performed to validate the performance of the proposed method.The acceleration voltage used to generate the electron beam is 10 kV and has been fixed through all the experiments performed with this sample.The magnification used for this test is ×300 and the images are acquired with a raster scan speed of 720 nanoseconds/pixel, which provides a frame rate of 2.2 frames per second.The sample for the experiments is a microscale calibration rig containing chessboard patterns (Fig. 5).The brightness and the contrast are set to optimal values for the image acquisition process.The evolution of focus step and image gradient are shown in Fig. 6(a) and the variations of velocity and working distance are shown in Fig. 6(b).
From the obtained results, it is evident that the velocity decreases to zero when the image gradient reaches its maximum, which points out that the best focus has been accomplished successfully.

Validation under Different Conditions
Different sets of experiments have been conducted to validate the proposed method at various experimental conditions that include the variation in scan speed and magnification.Normally with SEM, usage of higher scan speeds or increasing magnification degrade the useful image information by increasing the level of random noise [16], which slightly affects the image gradient.However, any such influence can be readily compensated by the closed-loop control scheme.Apart from that, the performance of the method has also been evaluated by comparing it with an iterative search-based method [6].It is a three fold technique that operates in three different iterations by varying the step size (distance between working distances) to search for the best focus position that provides the For the experiments, the step sizes used are 50, 5 and 1, respectively in each iteration.In both the cases i.e., for proposed and search-based methods, the optimum working distance estimated by a skilled human operator has been used as the reference in computing the error.
As a pre-processing step, the images were filtered using a Gaussian filter of size 5 × 5 to reduce the level of noise.The magnification varies from ×300 to ×1200 with a step change of 300.The acceleration voltages used for the sample are 10 kV .Besides, all the experiments are performed in four different trials for any particular condition.The results shown are the average values of all these trials.
Table 1 and Table 2 summarize the obtained results at different magnifications using scan speeds of 720 nanoseconds/pixel (optimal) and 180 nanoseconds/pixel (high), respectively.From the results, it can be noticed that the accuracy of proposed method is better than search-based method under both conditions, with an improved average accuracy of 60% in comparison with the search-based method.This is mainly due to the fact that the proposed method is not affected by the lens hysteresis.
From analysis, the obtained results clearly show the efficiency and the repeatability of the proposed method of autofocus regardless of the sample surface as well as the experimental conditions.Some of the images acquired during different experiments are shown in the Fig. 7.

Discussion
The obtained experimental results show the accuracy and efficiency of the proposed method in the case of various real world scenarios.Since the Jacobian is computed analytically, the autofocus procedure is more robust w.r.t. the alternative methods (i.e.numerical computation of Jacobian or approximating the sharpness function to a simple function).Since the sum of squared norm of gradient has been considered as a global sharpness function to be optimized by a closed loop control law, the efficiency of the method with high scan speed at ×300 (g) to (i) with optimal scan speed at ×900 (j) to (l) with high scan speed at ×900.Last column depict the in-focus images.
will not be affected by changing the sample or the magnification of the SEM.However, there are few limitations where the performance of the method could be affected.It should be mentioned that in our experiments the sample is flat.If the sample is far from perpendicular to the vision sensor, there is a risk that the sharpness function could feature multiple optimum.Generally in a SEM, the support of sample can be set to be perpendicular to the electron gun by the SEM software.In this case, the tilt is normally smaller than the field of view, keeping the sample in-focus when the autofocus procedure achieve the optimization.Next, similar to the other autofocus techniques (using any imaging device), the proposed method also requires the objects with sufficient texture information.

Conclusions
In this paper, an efficient and robust closed-loop control scheme has been proposed for a full scale autofocusing of SEM.The image gradient information is considered as the sharpness score in designing the vision-based control law.The optimum value of focus i.e., the maximum image sharpness has been obtained by updating the device working distance iteratively.
The method has been validated on different experimental conditions.Different from other SEM autofocusing techniques, the derivative of the cost function is computed analytically, which makes the algorithm robust.Since the designed cost function reduces exponentially, the proposed method quickly converges to the optimal value.

Figure 2 .
Figure 2. Control framework for SEM autofocus

Figure 3 .
Figure 3. Evolution of image gradient with respect to working distance.

Figure 7 .
Figure 7. Screenshots obtained during the autofocus process: (a) to (c) with optimal scan speed at ×300 (d) to (f) with high scan speed at ×300 (g) to (i) with optimal scan speed at ×900 (j) to (l) with high scan speed at ×900.Last column depict the in-focus images.

Table 1 .
Autofocus results using optimal scan speed.
1 fabricated at FEMTO-ST Institute, France

Table 2 .
Autofocus results using high scan speed.