Fast Quality Inspection of Micro Cold Formed Parts using Telecentric Digital Holographic Microscopy

Quality inspection is an integral part of the production process and often part of the quality management agreements between manufacturer and customer. Especially when it comes to safety-relevant parts, i.e. in the automobile or medical industry, often a 100% quality inspection is mandatory. Here, we present a solution comprised of a digital holographic measurement system, as well as fast algorithms for geometric evaluation and surface defect detection that paves the way for the inspection of metallic micro cups in less than a second. By use of a telecentric lens instead of standard microscope objective, we compensate scaling effects and wave field curvature, which distort reconstruction in digital holographic microscopy. Due to limited depth of focus of the microscope objective, depth information from different object layers are then stitched together to yield 3D data of its geometry. The resulting point cloud data is automatically decomposed into simple geometric shapes in order to analyse geometric deviations. Amplitude as well as phase distribution images are then analysed for surface defects. Our approach is demonstrated by inspecting cold formed micro cups. Defects larger than 2 μm lateral resolution and 5 μm depth can be detected.


Introduction
Quality inspection is an integral part of the production process and often part of the quality management agreements between manufacturer and customer.Especially when it comes to safety-relevant parts, i.e. in the automobile or medical industry, often a 100% quality inspection is mandatory [1].
Achieving this goal is especially challenging in the micro-domain, i.e. when measurement uncertainty has to be less than a few μm as there is usually a trade-off between precision and speed [2].Processes like e.g.micro cold forming, however, allow for production rates of multiple parts per second.Presently, there exist no solutions that are fast, precise and suitable for in-process measurements at the same time [3].
Due to these indusial requirements, methods such as tactile measurements, confocal microscopy and phase retrieval [4,5] are not suited for the inspection task since they are too slow.In contrast to these methods, interferometric methods are established for a full field measurement.Examples of interferometric techniques include Digital Holography (DH) [6], White-Light Interferometry (WLI) [7], Optical Coherence Tomography (OCT) [8] and computational shear interferometry (CoSI) [9].These methods are based on the determination of the optical path difference (OPD) of light scattered by or transmitted through a test object.Since available cameras can only measure intensities of light, the OPD must be encoded consequently.Based on the utilized encoding technique and the geometrical complexity of the test object, measurement uncertainties down to a fraction of the illumination's wavelength can be achieved [10].
Compared to other interferometric techniques, DH is considered fast since the OPD can be retrieved from only a single digital hologram.DH is based on the evaluation of the lateral phase distribution (within the object plane) of monochromatic light.For this purpose, light coming from a test object and a reference wave having known characteristics are coherently superimposed to convert the phase information into detectable intensity modulations.From the resulting intensity pattern, information on the phase can be extracted using phase MATEC Web of Conferences 190, 15008 (2018) https://doi.org/10.1051/matecconf/201819015008ICNFT 2018 shifting techniques.Thereafter, the optical path difference is identified and thus the shape of the test object is obtained.However, the measurement of the OPD is ambiguous with respect to multiples of the wavelength.Hence, for optically rough surfaces, the result of the evaluation cannot be used to reconstruct the shape of the test object if the height variations exceed half of the wavelength.One way to solve this problem is measuring two phase distributions associated with two different wavelengths.This approach is referred to as two-wavelengths contouring technique.
Here, we present a solution comprised of a digital holographic measurement system, as well as fast algorithms for geometric evaluation and surface defect detection that paves the way for the inspection of metallic micro cups in less than a second.By use of a telecentric objective instead of standard microscope objective, we compensate scaling effects and wave field curvature, which distort reconstruction in digital holographic microscopy.Depth information from the measured holograms is then stitched together to yield 3D data of its geometry.The depth information is converted to a 3D point cloud, which is automatically decomposed into simple geometric shapes in order to analyse geometric deviations.Amplitude as well as phase distribution images are then analysed for surface defects.
In section 2, hologram recording using the proposed DHM, digital hologram reconstruction and reconstructing the 3D surface of micro cups will be discussed.The geometry and surface inspection of micro cups under test will be discussed in section 3. Finally, a brief conclusion will be given in section 4.

Holographic contouring
Commonly engineering objects' surfaces exhibit peaks and steps larger than half of the wavelength.Since the measured phase values are located within the interval [-, such surfaces are considered to be optically rough.Thus, the evaluation process becomes ambiguous and the measurement is not unique.The solution to this dilemma is referred to as the synthetic wavelength.In this approach, two phase distributions are retrieved from two measurements associated with two different wavelengths 1 and 2.Calculating the phase difference  between the reconstructed phase distributions corresponding to the two measurements, the 3D height map zp of the test object can be directly calculated by utilizing Here,  is the angle between the observation and illumination direction and  is referred to as the synthetic wavelength which is given by The resulting phase difference distribution map given by Eq. ( 1) equivalents a single measurement with .This map contains fringes and it is referred to as the phase contouring map.Adapting the difference between the two wavelengths 1 and 2 is required to enable the investigation of objects with step heights of several millimeters.

Digital Holographic Microscopy
The digital holographic microscope given in Fig. 1 is used to capture the required two holograms with two different wavelengths.The setup contains a long distance microscope objective (LDM) with a magnification of 10, a numerical aperture (NA) of 0.21 and a working distance (WD) of 51 mm.Optical fibers were used to illuminate the test object and to provide the required reference wave.A beam splitter (BS) is used to combine object wave, light diffracted from the surface under investigation, and the reference wave, resulting in a hologram, which is captured across the camera plane.It is noted that there exists an angle  between the observation and illumination direction.Numerically the phase distribution corresponding to each measurement, is reconstructed utilizing the spatial phase shifting method, where the carrier frequency is controlled by shifting the position of the reference wave with respect to the optical axis [3].Thus, the setup was used to register the phase information in the plane, which is close to the surface of the object under test.

Surface measurements of micro components
The micro deep drawn cup shown in Fig. 2 insert is as an example of a micro component.Geometry inspection of such micro parts is needed in order to improve and optimize the micro cold forming process.In the following, the 3D surface measurements of the micro cup based on digital holographic microscopy, which is considered to be the backbone of geometric inspection, will be discussed.
An experimental setup is realized based on the sketch shown in Fig. 1.An image of the setup is shown in Fig. 2. Four laser diodes supplied by the company Thorlabs two with λ = 638.13nm and the other two with λ = 644.08nm were used.According to Eq. 2, a synthetic wavelength of 69.07 µm is used for the evaluation process.Utilizing a fiber switch and a 12 fiber splitter, object and reference waves are realized.The LDM is an object side telecentric objective with a numerical aperture of 0.21, a magnification of 10 and a working distance of 51 mm.From Rayleigh formulae, the latera resolution (0.61/N.A) of each DHM unit is 1.87 m and the axial resolution (1.67/N.A. 2 ) is 24.4 m at  = 644 nm.This limits the lateral size of the defects measured by the DHM.It is noted that the limited axial resolution is enhanced by utilizing numerical focusing properties of digital holography.The camera sensor is an AVT prosilica GT 2750 having 27502200 pixels, where the pixel pitch is 4.54 µm.It should be noted that the test object is illuminated from four different directions and four holograms are recorded on single-shot using four reference waves by applying the digital holography multiplexing principle [3].These holograms are used to reduce speckle noise in two wavelength contouring.Accordingly, each holographic unit from the four units will capture two successive multiplexed holograms.The two successive multiplexed holograms are captured, one for each wavelength, using only one DHM observation unit.This means for each observation direction, eight holograms are captured in two shots for the two wavelengths.In the following, results captured using only one observation direction will be presented and discussed.The time required for the capturing process and for the switching between the two wavelengths is 70 ms.Using the spatial carrier frequency method [11,12] one can numerically reconstruct the phase distributions and which correspond to the two measurements.Errors with lateral extensions from 2 µm and minimal depth of at least 5 µm can be detected [3].An example of the resulting real amplitude reconstructed from the hologram captured for λ = 638.13nm is shown in Fig. 3a.The phase difference  = 1 between the two reconstructed phase distributions across the capturing plane which represents the countering map across that plane is shown in Fig. 3b.As it can be seen in Fig. 3b, fringes are only sharp across the in focus area of the object which can be clearly recognized by the real amplitude shown in Fig. 3a.This is expected since the microscope objective has a limited depth of focus.In contrast to microscopy, digital holography offers the extension of the objective depth of focus, since holograms give access to the complex amplitude, digital refocusing across the whole object by means of numerical propagation is possible.Thus, in order to completely reconstruct a sharp contouring propagation, autofocus algorithms are used.For fast evaluation, an automated process was proposed and implemented within a graphics processing unit (GPU).The autofocus algorithm is implemented by scanning within small windows throughout all propagated planes to define where the object is in-focus by estimating the standard deviations of  which is relatively high within those windows, where the object is out of focus because of the speckle decorrelation.The result of this process is shown in Fig. 4. The contouring phase map shown in Fig. 4b is then unwrapped.Form the quantitative comparison which is based on the standard deviation across different areas of the constructed contouring map shown in Fig. 4b, the height error ranges between 2.5 and 4.6 m.The unwrapped phase values are substituted into Eq. 1 to determine the 3D height map.The result is shown in Fig. 5.Such a height map is converted to a 3D point cloud which is used as the input for both the dimensional and the surface inspection process which will be discussed in the next section.

Dimensional and surface inspection of micro parts
This section describes the evaluation of optically acquired data with respect to geometric deviations (subsection 3.1) as well as regarding surface defects (subsection 3.2).

Dimensional inspection
Optical measurement data contains a high number of surface coordinates of one or multiple observation directions and represents either a free-form surface or a combination of several geometric elements.The evaluation of free-form surfaces, on the one hand, consists in aligning the measurement data to the nominal CAD-data [13] and to calculate and visualize the deviations of each measurement point.For this kind of quality inspection, several commercial solutions exist.On the other hand, in order to analyse the optical measurement data regarding geometric parameters like dimensional or shape deviations, respectively, the measured points have to be segmented.This means assigning the individual measurement points to the approximating geometric elements.However, a manual segmentation is time consuming and not suitable for an automated analysis within a mass production.Furthermore, due to the following context, it is to aim for an automated segmentation.
For reducing the measurement uncertainty, the geometric objects to be analysed should be acquired thoroughly.For instance, a decreased central angle of a circle directly leads to an increased uncertainty of the approximated geometric parameters [14].In case of micro features or measurement data with a low point density, a further reduction of measurement points additionally raises the uncertainty [15].This can happen during the manual segmentation due to a doubtful assignment, e.g. by leaving out points which cannot be reliably correlated to a certain geometric element.Only by an automated segmentation of the measurement data, the individual measuring points can be assigned in a reproducible and optimal way to the corresponding geometric elements.Two approaches are known for such an automated segmentation: 1.
Neighbouring measurement points are rated based on their curvature and assigned to corresponding geometric elements [16].This method can provide accurate solutions, but it is sensitive to noisy data and not able to distinguish between spheres and cylinders with certain radii.

2.
A holistic approximation can evaluate a composed set of data under the present boundary conditions in a single approximation task [17].By the definition of separating functions, an optimal assignment of the measurement points to the corresponding geometric elements (segmentation) can be carried out simultaneously.The method is presented for different applications in [18,19].
It was proved that the holistic approximation with automated segmentation (2nd approach) is only little sensitive regarding the initial values of the approximation and at the same time converges reliably within wide ranges [18].Furthermore, this method was successfully tested for the evaluation of micromeasurements [20], and it allows the automatic detection of outliers by a combination with statistic methods [21].Thus, the 2nd approach is particularly suited for noisy optical measurement data.However, the algorithms were not yet implemented for the evaluation of optical data acquired with DH.
The presented method is described in detail in [19] for the evaluation of tactile acquired geometries of microforming tools.As the geometry of micro-cups is composed in the same way by a combination of a plane, a torus and a cylinder, the holistic approximation can be applied with the same geometric model for the dimensional evaluation of optically acquired micro-cup surfaces.https://doi.org/10.1051/matecconf/201819015008ICNFT 2018 Fig. 6.Result of holistic approximation: The measured points are decomposed into a plane (green), a torus (red) and a cylinder (blue).
The approximately 3 million measured points of the micro-cup's surface were evaluated with the holistic approximation.The decomposition result is presented in Fig. 6.The torus wall radius was calculated to rt = 317.7 µm and the cylinder radius was determined to rc = 542.2µm.A calculation time of 0.9 s could be achieved by using the MATLAB Parallel Computing Toolbox calculating on a graphics processing unit (GPU) and reducing the density of the acquired point cloud.Tests with simulated data showed that a reduction of the measured points by the factor 100 results in deviations of the approximated radii less than 0.09 µm.

Surface inspection
Surface defects like e.g.scratches or dirt might be too small to cause a detectable change in the measured phase distribution.Hence, reliable detection necessitates additional methods, which incorporate the measured amplitude image.
In order to realize fast and precise detection of surface defects, we opted for convolutional neural networks (CNN), as they have drastically increased the abilities of machine vision in many areas, as e.g.object classification [23,24,25], semantic segmentation [26,27] or the generation of text descriptions for images [28].In contrast to methods that rely on laborious feature engineering, CNNs are able to automatically extract useful.CNNs have hence already been successfully applied for automatic surface inspection [29,30,31].

Data and pre-processing
We created a data set comprising of 69 amplitude measurements taken from top view, which we handlabelled for surface defects.All measurements were downscaled to a size of 512×512.Backgrounds were removed by applying a strong low-pass-filter (i.e.convolution with a 55×55 matrix of ones) to the measurement and subsequent thresholding with the mean of the resulting matrix.In the resulting binary matrix the largest contour was detected using methods, provided by the OpenCV software library [32].An example for the pre-processing is shown in figure 7.

Network architecture
We opted for a modified "U-Net" architecture [33], as it achieves pixel-level detection while being computationally efficient.This architecture augments the classical CNN architecture with additional layers for up-sampling the results.Fine-grained spatial information is thereby fed into the up-sampling layers via skip connections from the earlier layers.Figure 8 shows a schematic graph of the network used in this work.In the down-sampling part of the network we used blocks of one separable convolutional layer [34] followed by Max-pooling [31] and batch normalization [35].In the up-sampling we used blocks of one up-sampling layer, one concatenation layer for combining the upsampled result and the corresponding feature layer, and one separable convolutional layer to further process the results.The parameter values at each layer are provided in table 1.As a modification we not only predicted local defect maps but also used the features of the middle layer (block 4 in tale 1) for global defect detection, i.e. deciding if a measured cup is overall defective or not.To achieve this, we used global max pooling, i.e. used the maximal activation of each feature over all positions and fed this into a single sigmoid unit.https://doi.org/10.1051/matecconf/201819015008ICNFT 2018 Fig. 8. Schematic of model architecture.The pre-processed measurement is fed into the network (top left) and downscaled while extracting features for defect detection (left part).These features are then used for global, i.e. measurement-wise defect classification (bottom).To also enable defect localization the features are upsampled again (right part), yielding a map with defect probabilities (blue = 0, red = 1).

Model parameters
As activation function, we used exponential linear units (elu) [36] in all but the output layers were we used sigmoid units to constrain the output to 0-1.Filter sizes where chosen to be 5×5, except for the output layer with 1×1.For all but the last convolutional layer we used depthwise separable convolutions [34].Max pooling was conducted over 4×4 windows, effectively decreasing the result by a factor of 0.25.
The network was trained to minimize binary crossentropy using the Adam optimizer [40] with minibatch size of 5.
All experiments were conducted using the keras library for Python.

Evaluation
In order to utilize as much of the data as possible for training, we opted for a modified leave-one-out cross validation scheme, i.e. we used 68 samples for training and the remaining sample for validation and repeated this process 10 times.Each model was trained until perfect discrimination between intact and defective parts was reached for the training set.We then computed the lowest threshold for achieving perfect discrimination and used it to determine if the validation sample would be correctly classified.

Results
All 10 validation samples were correctly classified.
Examples for defect localization are shown in figure 9.It can be seen that all defects are labelled correctly, although some labels are still rather coarse.We hypothesize that this is due to the relatively small amount of training data and will hence improve as larger amount of data are being acquired.

Conclusions
We demonstrated here a complete method for the automatic measurement, geometric decomposition and surface inspection of micro parts.The experimental configuration is based on two-wavelength holographic contouring technique.Amplitude as well as phase distribution images recovered from the captured holograms are utilized for dimensional and surface defects.The setup can detect defects from 2 μm lateral resolution and 5 μm depth.Algorithms for evaluation are parallelized in order to speed up computing time.The proposed method is demonstrated by inspecting cold formed micro cups.Complete evaluation process for holographic capturing and acquisition is 12 s, for dimensional evaluation is 0.9 s and for surface inspection is 0.065 s.This means in total the evaluation process could be achieved in less than 13 seconds.The next steps would be the characterization and calibration of data acquisition.Stitching of multiple views will likely further decrease uncertainty for incomplete acquired features but will also bring an extra individual uncertainty contribution, dependent on the chosen method.This has to be analysed / characterized.This optimisation and characterization is part of ongoing research.

Fig. 1 .
Fig. 1.Digital holographic microscope setup (DHM): To image the surface of the test object onto the utilized camera sensor, a long distance microscope objective (LDM) having 10 magnification, a numerical aperture (N.A.) of 0.21 and a working distance of 51 mm was used.Fiber technology is used to illuminate the object under test and provide a spherical reference wave.For simplicity, only one illumination and one reference is shown.A beam splitter was used in order to combine the object and the reference waves producing a hologram across the camera plane.

Fig. 2 .
Fig. 2. Digital holographic microscope setup based on the scheme shown by Fig. 1.The setup consists of four DHM units distributed around the test object.Thus each unit delivers part of the test object.These four parts can then be used to reconstruct the whole 3D shape of the test object.The insert is an SEM image of a micro deep drawn cup with a diameter of ca. 1 mm and a depth of 0.5 mm, which is shown as an example of a test object.

Fig. 3 .
Fig. 3. a) Shows the real amplitude of the reconstructed complex amplitude across the capturing plane of the recorded hologram for λ = 638.13nm.b) Shows the phase difference between the two reconstructed  = 1 phase distributions across the capturing plane.The image size is 2200  2200 pixels with a pixel pitch of 4.54 µm.

Fig. 4 .
Fig. 4. a) Shows the real amplitude, which represents a sharp image of the micro cup under test with respect to the observation direction.b) Shows the phase difference distribution which represents a sharp contouring phase map across the whole object.

Fig. 5 .
Fig. 5.The 3D height map calculated after unwrapping the countering phase map shown in Fig 4b.

Fig. 7 .
Fig. 7. Top left: measurement.Top right: measurement after application of a low-pass filter.Bottom left: result of applying a binary threshold.Bottom right: largest contour.

Fig. 9 .
Fig. 9. Top row: input measurements.Middle row: hand-drawn defect labels.Bottom row: the networks predictions.Red indicates high defect probability and blue indicates low defect probability

Table 1 .
Network architecture and parameters