Optical assemblies of an imaging module in medical devices to image objects onto an image sensor include multiple lens units, such as an objective lens. Gastrointestinal endoscopy, one of the electro-optical devices with an assembled imaging module, is medical equipment used to image internal body cavities for visualization [1]. The imaging module of gastrointestinal endoscopy commonly consists of a miniature objective lens and an image sensor, which is configured at the distal end of gastrointestinal endoscopy along with a light guide, water jet, air/water nozzle, and biopsy channel [2]. The endoscopic objective lens consists of several subminiature lenses and is optically designed considering not only the specification of an image sensor but also the required endoscopic performance. A housing to combine several lenses and an image sensor are required to build the imaging module, and the miniature lens and sensor must be accurately assembled to ensure the optimal image quality [3]. Imaging performance can be influenced by the precision of the assembly; However, detailed assembly protocols between the miniature lens and sensor, as well as multiple lens units such as objective lens, including theoretical optical simulation data, are commonly unknown and not well documented.

In this study, we investigated the optimal approach for assembling the imaging module at the distal end of the gastrointestinal endoscopy as well as quantitative performance evaluation methods of the imaging module. An assembly protocol of the imaging module can be established by finding the best way to ensure image quality. One of the traditional methods to determine image quality, such as in regard to resolution and contrast, is to find the modulation transfer function (MTF) of the imaging module. In general, the MTF has been used to examine the ability of optical systems such as cameras, microscopes, projectors, or other imaging modules to distinguish various spatial frequencies. The MTF can be defined in theory as the magnitude of the optical transfer function (OTF), which is represented by the Fourier transform of the point spread function [4, 5]. The OTF describes the response of an imaging system as a complex-valued function of spatial frequency. However, phase effects of the OTF have rarely been considered. Recently, slanted-edge methods to deduce the system MTF have been widely investigated to judge the ability of optical systems to transfer light from one spatial frequency to another [6–15]. The slanted-edge method was mentioned in International Organization for Standardization (ISO) standards [16, 17]. Spatial frequency response (SFR), which quantifies the degree of change in output signals relative to inputs as a function of spatial frequencies, is used to represent the resolution of the entire system from optical lenses to the image sensor, and is comparable with the OTF and MTF based on linear system theory [18, 19]. Thus, when assessing image quality from an assembly perspective, the MTF can be regarded as a function that varies based on the spatial frequency. In general, the imaging performance of commercial gastrointestinal endoscopies is guaranteed at a working distance of 5–100 mm on average. Here, the performance guaranteed by near-focus mode imaging was excluded. Imaging performance at the entire working distance can be determined by the value of the MTF, but this may not be included as part of publicly available information on endoscopy. The specifications of gastrointestinal endoscopic devices typically provide data such as the working distance and field of view.

In addition to measuring the MTF, two approaches were investigated to determine the optimal assembly of the imaging module. The image sensor can be ideally assembled by positioning it at the focal plane of the lens [20]. One of the functions that applies this approach is a laser-based autofocus function that has been implemented to capture sharp images by moving the image sensor until the image of collimated light, which is converted into a spot image at the focal plane, is minimized along the optical axis. We also suggested another approach using three different mathematical algorithms to measure the focus index for best-focused images at a specific working distance of the endoscopic imaging module [21–23]. Each of these approaches provides distinct details regarding the spatial frequency properties of an image and may be useful for different applications in image processing and analysis. For example, autofocus algorithms have been typically used in microscopy to achieve an appropriate distance between the sample stage level and objective lens. For maximum image sharpness, the maximum value of the focus index was obtained based on focus measurements [24, 25].

Regarding the assembly of endoscopic imaging modules, it is important to establish the standard of imaging performance. As mentioned above, the MTF was adopted as one of the standard parameters to determine imaging performance. In terms of image sharpness, the focus index was also adopted as a key parameter for accurate assembly in this study. Considering both parameters, we expected to evaluate imaging performance using double verifications. More specifically, we measured the focus index using a star chart as well as the system MTF using a slanted-edge pattern based on the ISO 12233 standard [16, 17]. The imaging performance of endoscopic imaging modules was validated in two steps: (1) optimization of assembly, and (2) performance evaluation of the assembled imaging modules in terms of the MTF and the focus index. For experimental measurements, we also considered a limited working distance in the range of 5–100 mm, taking into account the average conditions of commercial endoscopies. By performing these two sequential processes, we assembled the imaging module to be inserted into the distal end. Performance parameters of the MTF and the focus index were quantitatively compared, and tried to establish the optimal assembly protocol.

2.1. Optimization of Assembly

For the assembly method, we used a full high-definition (FHD) image sensor (OV2740; OmniVision Technologies Inc, CA, USA) with an optical size of 1/6″. The sensor pixel size was 1.4 μm × 1.4 μm. For an endoscopic lens, we utilized a miniature objective lens (Sumita Optical Glass Inc., Saitama, Japan) with an outer diameter of 3.2 mm. The lens was designed to fit a size of the image sensor. The miniature lens and sensor module were mounted inside a separate metal housing case for precise assembly. Figure 1(a) presents an assembly method based on laser focusing. A collimated laser beam was first used to determine the focal position of a miniature objective lens, which may be regarded as an ideal method. In general, He-Ne lasers with characteristics of very good coherence and small beam divergence have been widely used as an alignment source [26, 27]. In this study, the light of a well-collimated He-Ne laser (05-LHP-991; Melles Griot, NY, USA) was focused on the image plane after passing through the miniature lens. Then, an image sensor was placed at the image plane when the spot size of light was minimized. To improve the accuracy of mechanical assembly processes, the outer housing cylinder of the lens and sensor module had screw threads with a pitch of 500 μm.

Figure 1. Assembly of an imaging module for gastrointestinal endoscopy. Assembly method based on (a) laser focusing and (b) focus index measurement. (c) Photograph of the assembled imaging module of gastrointestinal endoscopy.

The lens was driven forward and/or backward relative to the image sensor using a home-built jig that was designed to allow the lens to rotate 360 degrees after fixing the image sensor. The spot size of the light was measured using LabVIEW software (National Instruments Co. TX, USA) during the assembly process to assemble the imaging module more precisely. A photograph of the assembled module using the miniature objective lens and image sensor is shown in Fig. 1(c). To further validate the degree of optimization, we assembled additional imaging modules based on measurements of the focus index using a star chart of the ISO 12233 resolution test chart as shown in Fig. 1(b). To obtain the optimal method for calculating the focus index, three different mathematical approaches were considered, namely Vollath’s F₄ [21, 28, 29], absolute central moment (ACMO) [22, 30], and Spatial Frequency (SFRQ) [23]. The focus measures by the Vollath’s F₄, ACMO and SFRQ methods were calculated by the following Eqs. (1), (2), and (3), respectively.

(1)$$F_{\text{Vollath}}={\displaystyle \sum _{i=1}^{M-1}{\displaystyle \sum _{j=1}^{N}I(i,j)\times I(i+1,j)}}-{\displaystyle \sum _{i=1}^{M-2}{\displaystyle \sum _{j=1}^{N}I(i,j)\times I(i+2,j)}}$$

(2)$$F_{\text{ACMO}}=\frac{1}{MN}{\displaystyle \sum _{i=1}^M{\displaystyle \sum _{j=1}^N|I(i,j)-\overline{I}|}}$$

(3)$$F_{\text{SFRQ}}=\sqrt{\frac{1}{MN}({\displaystyle \sum _{i=0}^{M-1}{\displaystyle \sum _{j=1}^{N-1}{[I(i,j)-I(i,j-1)]}^2}}+{\displaystyle \sum _{j=0}^{N-1}{\displaystyle \sum _{i=1}^{M-1}{[I(i,j)-I(i-1,j)]}^2}})}$$

Vollath’s F₄ (Vollath, 1987) ,which has been widely used for automated microscopy, is based on the autocorrelation function [31], whereas ACMO and SFQR are based on the measure of standard deviation and frequency level in the spatial domain, respectively. We calculated the focus index for images of the same star chart captured at various distances using three distinct methods. The best focus position was determined by fitting the optimal focus measurement curve. The star chart was located at a preset distance (d_s) of 5, 10, 20, …, 80, 90, and 100 mm from the endoscopic miniature lens. In this study, d_s was defined as the distance between the star chart and miniature lens when assembling. The distance between the miniature lens and image sensor was determined under the condition that the focus index obtained at each location of the star chart showed the maximum value. The focus index was also measured using LabVIEW in real time when the miniature lens was driven back and forth relative to the image sensor.

2.2. Performance Measurement

After the assembly process, the differences of each imaging module were evaluated in performance with respect to the MTF by comparing these two assembly methods. Here, the system MTF was considered, which was defined as the resolving power of an imaging system to present the details of an object. The system MTF can be considered an evaluation of the entire series of processes, which is generally referred to an imaging pipeline, that is, lens-sensor-image processing-image file-monitor [32, 33]. In other words, for assembled modules, we attempt to identify the system MTF, which represents the overall effects of the assembly of optical device modules, pixel architecture specifications of the image sensor, imaging processing algorithm, data transmission circuit and image display. Raw images of a target object obtained by the imaging module are displayed after performing image processing. Every component of the imaging pipeline induces degradation of the resolving power of the imaging module, which means perfect reproduction of the original target object is eventually limited.

To measure the system MTF, the slanted-edge method was employed based on the ISO 12233 standard. An optical diagram to measure the slanted-edge MTF is presented in Fig. 2(a). We built an optical measurement system in the laboratory environment to capture slanted-edge patterns and calculate the MTF by fixing the distal end of endoscopy. A square pattern with slightly (5°) tilted edges relative to rows and/or columns was used, as shown in the inset of Fig. 2(a). This square, made with a chrome coating on glass, was illuminated by an LED light (LEDPanel; Thouslite, Jiangsu, China) with 6,000 K illumination and a uniformity of 96% across the measured field. The distal end of the gastrointestinal endoscopy was mounted to a motorized stage that moved with a resolution of 1 mm in the axial direction. The top surface of the square pattern was set to the baseline [which means the working distance (d) = 0 mm], and the distal end was moved up to d = 120 mm from the baseline. The value of the slanted-edge MTF was measured within the range of d = 5 mm to d = 100 mm. Raw images of a tilted square pattern were measured through a full HD image sensor of the assembled imaging module. Figure 2(b) shows captured images of the tilted square pattern at d = 20, 50, and 100 mm. A preset region of interest (ROI) of 256 × 256 pixels was used to calculated the MTF of each image captured within the whole range of working distance. The MTF mentioned in this paper usually refers to the system MTF unless otherwise specified. Here, we follow a well-known and well-developed process to calculated the SFR [5–16], which is used interchangeably with the system MTF. The tilted edge in the ROI of the slanted-edge image was linearly fitted to obtain the edge spread function (ESF), and oversampling of the ESF caused a reduction of signal aliasing. Finally, as expressed in Eq. (4), the MTF was induced by calculating the discrete Fourier transform of the line spread function, which was obtained after the discrete derivative of the ESF [8, 34–36].

Figure 2. Measurement of the slanted-edge pattern to obtain the system MTF. (a) Optical diagram to measure the slanted-edge MTF. (b) Captured images of the tilted square pattern at d = 20, 50, and 100 mm.

(4)$$\text{MTF}(f)=|\text{FFT}[\frac{d}{dx}\text{ESF}(x)]|$$

We compared the performance of imaging modules assembled using different methods: (1) laser focusing and (2) focus index measurement with the star chart located at d_s = 10 mm, (3) d_s = 20 mm and (4) d_s = 30 mm. These imaging modules assembled by methods (1), (2), (3) and (4) are referred to as models A, B, C and D, respectively. According to the working distance (d) that was preset to 5, 10, 20, …, 80, 90, and 100 mm from the baseline (d = 0 mm), the slanted-edge MTFs of models A, B, C and D were quantitatively measured along the horizontal and vertical axes as shown in Figs. 3(a)–3(d), respectively. To measure the MTF, the focus index calculation based on the Vollath’s F₄ method was adopted when assembling imaging modules in this paper. Model A, based on laser focusing [presented in Fig. 3(a)], showed the best performance under the condition of d = 20 mm, and the performance of the MTF was degraded as the object moved away from the distal end of the endoscopy. MTF values at d = 5 mm and d = 10 mm were lower than those at other working distances. The performance tendency of model C seemed to be similar to that of model A according to the variable working distances, as presented in Fig. 3(b). The best MTF for model C was obtained at a working distance of 20 mm, which is similar to model A. However, different aspects of model B and D are shown in Figs. 3(c) and 3(d), respectively. For model D, the maximum MTF was measured at d = 50 mm. On the other hand, the MTF decreased when the working distance was less than 20 mm compared to when it was 30–100 mm. Unlike other models, the deviation of the MTF was not large within the range of 30 mm to 100 mm. However, the performance of the MTF was significantly degraded at working distances of 5, 10, and 20 mm. Model B shows a different trend in MTF compared to other models, with the best MTF at d = 5 mm and decreasing performance as the working distance increases. In particular, significant degradation in imaging performance was measured at a working distance of 20–100 mm.

Figure 3. Measured slanted-edge modulation transfer function (MTF) according to the varying line pair at working distances of 5, 10, 20, …, 80, 90, 100 mm: (a) Model A, (b) model B, (c) model C and (d) model D. Horizontal and vertical MTF are shown on the left and right, respectively.

For a more detailed analysis of each model, the depth of field according to the various working distances was estimated by assessing the degree of slanted-edge MTF variations in the depth direction along the optical axis, as shown in Fig. 4. In the case of models, A, C, and D, the MTF tended to increase and then slightly decreased as the distance to the object increased in the depth direction [see Figs. 4(a), (c), and (d), respectively]. As mentioned above, models A and C had the maximum MTF at d = 20 mm. From 30 mm to 100 mm, the performance of models A and C tended to be slightly degraded as the working distance increased, but the MTF of model A was slightly higher than that of model C. Up to a working distance of 30 mm, model D showed considerably worse performance of the MTF than models A and C, and the MTF saturated relatively constant at a working distance of 30–100 mm. The MTF performance of three modules of models A, C, and D tended to be degraded for short working distances. In particular, it seemed difficult for close-up imaging at a working distance of 5 mm to achieve an MTF of 50% or more under conditions of the line pair larger than 20 lp/mm. On the contrary, model B exhibited an opposite tendency with respect to the working distance compared to other models [Fig. 4(b)]. Therefore, the optimized imaging performance can finally be achieved by using the imaging module assembled with the laser focusing method (such as model A) when target imaging up to a working distance of 30 mm, including close-up imaging, is important. Model C can be an alternative from an imaging performance perspective since it has imaging performance similar to model A. Otherwise, the imaging module assembled based on the focus index measurement (such as model D) can be adopted if the imaging performance is ensured at greater working distances (30 mm or more).

Figure 4. Estimated depth of field according to various working distances at line pairs of 10, 20, 40, 60, 80, and 100 lp/mm: (a) Model A, (b) model B, (c) model C and (d) model D. Horizontal and vertical modulation transfer function are shown on the left and right, respectively.

Each value of line pairs at 50% MTF (MTF50), 20% MTF (MTF20) according to the working distance in the horizontal and vertical directions is presented in Fig. 5(a). MTF50 and MTF20 at each module are denoted by dotted and solid lines, respectively. For models A, C, and D, MTF50 was measured within the range of 20–84 line pairs over the entire working distance, and MTF20 was measured within the range of 30–144 line pairs. When considering a working distance of 10–100 mm, the minimum line pair for satisfying MTF50 and MTF20 is 38 and 64, respectively. In addition, for models A and C only, the minimum line pair for MTF50 and MTF20 increases to 56 and 96, respectively, at a working distance of 10–100 mm. The average values of the MTF50 and MTF20 ratios at each working distance and each imaging module are shown in Figs. 5(b) and 5(c), respectively. The number of line pairs for MTF50 corresponded to about 60% or slightly lower than that for MTF20, and this ratio was consistently observed not only in models A, C, and D but also across the working distance range of 20–100 mm. A MTF50/MTF20 ratio exceeding 60% was observed in the case of a working distance of 5 mm [Fig. 5(b)] and model B [Fig. 5(c)]. This indicated a significant degradation that satisfies both MTF50 and MTF20.

Figure 5. Measurements of 50% MTF (MTF50), 20% MTF (MTF20), and the ratio of MTF50 and MTF20. (a) MTF50 (dotted lines) and MTF20 (solid lines) in the horizontal and vertical direction, presented in the left and right, respectively. (b) Average of MTF50/MTF20 at each working distance, and (c) average of horizontal and vertical MTF50/MTF20 for each model. MTF, modulation transfer function.

An analysis of the focus index was also explored based on three different methods, Vollath’s F₄, ACMO, and SFQR, for each imaging module, as shown in Figs. 6(a)–6(c), respectively. Focus index measurement was employed with the same star chart as that used during assembly. The focus index of model C derived by the Vollath’s F₄ method was calculated to have the highest value when the working distance ranged from 10 to 30 mm. Otherwise, model D had the maximum focus index at d = 40–100 mm. Different tendencies of the focus index derived by ACMO and SFQR were observed compared to the focus index based on Vollath’s F₄. Although there were slight differences, the focus index of models A, C, and D increased until the working distance reached 40 mm, and then decreased slightly as the working distance increased. The results of focus index calculation showed slight differences depending on the method for focus measurement, and there were also differences when compared to the MTF performance results. For example, in the MTF analysis, the imaging performance of model A was found to be slightly superior to that of model C. Therefore, the most optimal model can be selected depending on the criteria chosen to determine imaging performance.

Figure 6. Measured focus index of models A, B, C and D according to three different mathematical methods: (a) Vollath’s F4, (b) absolute central moment (ACMO), and (c) spatial frequency (SFRQ).

In this study, we assembled an imaging module for gastrointestinal endoscopy, set parameters for imaging performance analysis, and measured each parameter within the working distance guaranteed by endoscopy. An optimal protocol for assembling the imaging module of gastrointestinal endoscopy can be established based on quantitative performance evaluation. Four types of imaging modules were assembled by mounting miniature objective lenses and image sensors based on methods of laser focusing and focus index measurement, and we experimentally evaluated the imaging performance by measuring the system MTF, corresponding depth of field, and the focus index. Within a distance range of 5–100 mm between the distal end of the endoscopy and target objects, we observed changes in the MTF performance for different working distances and line pairs. The imaging performance evaluation based on the focus index measurement was also analyzed. Within the working distance less than 30 mm, imaging modules assembled based on laser focusing and focus index measurement from a star chart located 20 mm away showed relatively superior MTF performance compared to other modules. The laser-focusing-based imaging module had a slightly better MTF performance. However, at a working distance further than 30 mm, the performance of the imaging module assembled by focus measurement using a star chart located 30 mm away was better. All three modules showed significant degradation of performance under the working distance condition of less than 10 mm, but this can be expected to be improved with the implementation of a close-up imaging technique. We examined the performance and assembly within a limited range of the working distance from 5–100 mm in this study, but the specific working distance at which an imaging module of the gastrointestinal endoscopy should have the optimal performance can be defined according to medical opinions. In conclusion, a laser-focusing-based assembly method is preferred to achieve overall good imaging performance at a working distance of 5–100 mm. Alternatively, it can provide better performance to assemble the imaging module aligned with a fixed working distance in capturing objects at a specific distance. Thus, this study can be influential in establishing an assembly protocol based on the performance evaluation for small imaging modules of medical equipment used in the field of biomedical research as well as the development of actual gastrointestinal endoscopy.

The authors declare no conflicts of interest.

Data underlying the results presented in this paper are not publicly available at the time of publication, but may be obtained from the authors upon reasonable request.

A Korea Medical Device Development Fund grant funded by the Korean government (the Ministry of Science and ICT, the Ministry of Trade, Industry and Energy, the Ministry of Health & Welfare, the Ministry of Food and Drug Safety) (Project Number: RS-2021-KD000006).

A Korea Medical Device Development Fund grant, funded by the Korean government (the Ministry of Science and ICT; the Ministry of Trade, Industry and Energy; the Ministry of Health & Welfare; and the Ministry of Food and Drug Safety; Project Number: RS-2021-KD000006).