검색
검색 팝업 닫기

Ex) Article Title, Author, Keywords

Article

Split Viewer

Research Paper

Curr. Opt. Photon. 2024; 8(6): 632-640

Published online December 25, 2024 https://doi.org/10.3807/COPP.2024.8.6.632

Copyright © Optical Society of Korea.

A Fully Electronic Confocal Microscope Using a Digital Micromirror Device and an Electrically Tunable Lens for 3D Measurement

Sung Hyun Kim1, Min Sung Ko2, Jae Hun Shin3, Jin Sun Yu2, Cha-Hwan Oh1

1Department of Physics, Hanyang University, Seoul 04763, Korea
2Research and Development Department, Basler Korea Inc., Seoul 05836, Korea
3Siwon Optical Technology Co., Anyang 14117, Korea

Corresponding author: *choh@hanyang.ac.kr, ORCID 0000-0003-4772-6696
Current affiliation: ZBIT Co., Ltd., Yongin 16864, Korea

Received: September 30, 2024; Revised: November 18, 2024; Accepted: November 19, 2024

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

With its ability to provide high-resolution, 3D images, the confocal microscope has revolutionized various scientific fields. However, conventional confocal systems often rely on complex mechanical scanning mechanisms, making them less cost-effective, less durable, and limited in speed. We present a 3D confocal microscope that uses a digital micromirror device (DMD) and an electrically tunable lens (ETL) for a fully electronic, high-speed 3D measurement solution. By leveraging the parallel scanning capabilities of DMDs and the precise axial focusing of ETLs, our approach offers significant speed, accuracy, and simplicity advantages. This innovative technology could enable new applications in materials science, biomedical research, and industrial inspection.

Keywords: Confocal microscopy, Digital micromirror device, Electrically tunable lens, High-speed 3D measurement, Non-mechanical scanning

OCIS codes: (110.0110) Imaging systems; (110.6955) Tomographic imaging; (150.3045) Industrial optical metrology; (180.1790) Confocal microscopy; (230.6120) Spatial light modulators

Confocal microscopy has long been a cornerstone in high-resolution imaging by offering 3D visualization by selectively blocking out-of-focus light with pinholes. Marvin Minsky first conceptualized this technique in 1955 to improve image clarity by eliminating interference from out-of-focus light [1]. Although Minsky submitted a patent in 1957, the technology did not see widespread use until the development of laser scanning techniques in the 1980s [2]. The first commercial confocal microscopes were introduced in the early 1980s, with systems relying on stage scanning mechanisms to generate high-resolution images [3]. These early systems were innovative but lacked speed and usability due to their mechanical complexity. The advent of laser scanning confocal microscopy (LSCM) enabled more precise imaging by scanning a laser beam across the specimen and collecting light through a pinhole [4].

However, despite this advancement, LSCM systems still face challenges, such as slow imaging speeds due to the reliance on single-point scanning [5]. In the following decades, efforts to improve confocal microscopy focused on increasing speed and reducing system complexity. The Nipkow disk system, developed in 1967, introduced a rotating disk with multiple pinholes for faster image acquisition by scanning multiple points simultaneously. Despite its increased speed, this system suffered from reduced light efficiency [6, 7]. The direct-view confocal microscope (DVCM), based on this system, has continued to face challenges, including light inefficiency and a trade-off between signal intensity and optical sectioning ability [8].

Over the years, various techniques have been developed to enhance the speed and simplify the complexity of confocal microscopy. Structured illumination microscopy (SIM), introduced in the early 2000s, uses spatial light modulators (SLMs) such as digital micromirror devices (DMD) to project structured light patterns onto the sample, enhancing imaging speed and resolution while enabling effective optical sectioning. However, it requires complex pattern generation systems and may exhibit reduced performance under certain imaging conditions [912]. Another approach, spectrally encoded confocal microscopy (SECM), emerged in the late 1990s. It uses SLMs to encode spatial information into different wavelengths, eliminating the need for mechanical lateral scanning. While SECM enhances axial scanning speed with an electrically tunable lens (ETL) and offers unique advantages, it often suffers from reduced light efficiency due to the dispersive nature of its optical components [1317]. Further advancements, such as chromatic confocal microscopy [18], mitigated certain limitations, but the technology still relied on mechanical parts. Consequently, modern confocal microscopes continue to face issues with durability, speed, and cost due to the complexity of their mechanical scanning mechanisms.

This study addresses these limitations by introducing a fully electronic confocal microscope that uses DMD to create virtual pinholes for high-speed lateral scanning and an ETL for rapid axial adjustments. Our novel system can dynamically adjust the size and pitch of the virtual pinholes offering flexibility in optimizing optical sectioning strength and signal intensity. Furthermore, it ensures consistent magnification and minimizes optical distortion across the depth scanning range. To achieve proper pinhole scanning during axial movement, it is crucial to eliminate magnification changes along the optical axis. The proposed system removes mechanical movement and reduces distortion from the ETL optics. This allows for faster and more accurate 3D measurements while decreasing manufacturing costs. This innovation is particularly relevant for industrial and biomedical applications where high-speed and high-precision measurements are essential [19, 20].

The confocal system in this study integrates dynamic lateral scanning using a DMD and z-axis scanning with an ETL to achieve fully electronic, high-speed 3D measurement. The system was designed to eliminate mechanical scanning components, thereby improving speed, durability, and cost-efficiency [12, 21].

In the configuration shown in Fig. 1, light from a high-power LED (90W, CBT-90; Luminus Devices Inc., CA, USA) passes through the DMD, which modulates the light beam for precise lateral scanning by adjusting the tilt positions of its micromirrors. This feature enables dynamic control over the illumination patterns, which is critical for high-speed imaging. The modulated light then travels through a pair of relay lenses. These lenses relay the light beam while preserving beam alignment and image quality without altering the magnification. The ETL between the relay lenses facilitates rapid focus adjustments along the z-axis. The ETL eliminates the need for mechanical z-axis scanning, which significantly improves the speed and precision of depth scanning for high-speed 3D imaging. The light subsequently passes through the tube lens and objectives, which focus it on the specimen. When the light interacts with the specimen, it is reflected through the same optical path and is retraced through the objective lens, tube lens, relay lens, ETL, and second relay lenses. At the final stage, the returning light reaches a beam splitter (BS), which directs a portion of the light to the camera. This light is then captured and processed to generate the final image.

Figure 1.Schematic of a digital micromirror devices (DMD)-based scanning confocal microscopy system integrated with an electrically tunable lens (ETL).

The primary focus of this research is the successful implementation and testing of this optical system, which leverages the unique capabilities of the DMD for lateral scanning and the ETL for dynamic z-axis control. This setup offers a robust solution for high-speed, high-precision 3D measurement, making it a valuable tool for industrial and research applications.

Figure 2(a) shows the initial test concept of a DMD-based confocal microscopy system. This setup was designed to evaluate the core functionality of the system before progressing to final integrations. This system employs a laser source that is expanded through a beam expander to provide uniform illumination.

Figure 2.Initial test setup for digital micromirror device (DMD)-based confocal system. (a) Schematic of an experimental setup for testing confocality of DMD’s pinhole array. (b) An experimental DMD-based confocal microscopy system is set up with a high-speed camera and optical components.

This configuration is shown in Fig. 2(b) was primarily used to test the basic functionality of the system by focusing on the DMD’s ability to control the illumination pattern and maintain optical alignment [22]. This setup initially evaluated image quality, resolution, and overall system performance before implementing more advanced features such as relay lenses, ETL, and further optical tuning.

Lateral scanning is performed using the DMD, which consists of an array of micro-mirrors that can be individually tilted to reflect light. The DMD was programmed to create pinhole arrays to scan the sample in the x- and y- directions [9]. Initially, a micromirror device evaluation kit (DLP5500; Texas Instruments, TX, USA) with a resolution of 1,024 × 768 pixels and a micromirror size of 10.8 µm was used [23]. The DLP6500 (Texas Instruments) with a 1,920 × 1,080 pixels resolution and a smaller micromirror size of 7.6 µm, was later integrated to improve spatial accuracy and resolution [24]. The DLP6500 also provided faster switching speeds, making it more suitable for high-precision 3D measurement applications [17].

This configuration used a green laser (λ = 532 nm, 40 mW, Z40M18B-F-532-pz; Z-laser GmbH, Freiburg im Breisgau, Germany) for illumination during lateral scanning. The reflected light from the sample was captured by a high-resolution camera (4M pixels, acA2040-180 km; Basler AG, Ahrensburg, Germany) to ensure accurate mapping of focus points across the lateral plane. The system processed images in real time by uploading pinhole array patterns (stored as 1-bit bitmap image files) to the memory of the main board. The captured images confirmed the system’s ability to adjust scanning parameters dynamically for flexible and high-speed 2D and 3D measurements [9, 25].

To verify the confocality of the system, a series of images were captured as the z-stage (Travel = 4 mm, LS6H-688; Misumi Co., Tokyo, Japan), previously calibrated using a displacement sensor, was displaced by ±3 µm around the focal plane, as shown in Fig. 3.

Figure 3.Captured images showing the effect of z-stage displacement on the DMD-generated pinhole array: (a) −3 µm movement, (b) optimal focus position, and (c) +3 µm movement.

At optimal focus (0 µm displacement), the pinhole array appeared sharp and well-defined, confirming that the system was correctly aligned with the focal plane. When the z-stage was displaced by −3 µm, blurring was observed in the top-right corner, while a displacement by +3 µm caused blurring in the bottom-left corner of the image. This demonstrated the system’s sensitivity to focal plane misalignment and further confirmed the accuracy of both the lateral and z-axis scanning capabilities.

The bright spots in the images represent the pinhole array generated by the DMD. Each pinhole was designed to be 2 pixels by 2 pixels for a well-defined image. The spacing between the pinholes was set to 50 pixels for a clear distinction between individual spots. The experiment revealed that the depth of focus of the pinhole is at least ±3 µm or less. This test considers these parameters to optimize confocality testing and demonstrate the ability of the system to maintain resolution and clarity in different z-axis positions.

As shown in Fig. 4, the confocal microscope system was designed using Zemax software to simulate light paths and optimize optical components for dynamic 3D scanning. The optical design consists primarily of two key sections: The tube lens assembly, which includes the objective lens, and a pair of relay lenses. This configuration allows the system to maintain high image quality and precise magnification control during z-axis scanning. The pair of relay lenses serves the crucial purpose of implementing the 4f system so that any changes in the focal length of the ETL do not affect the magnification of the system, which is essential to maintain image accuracy during z-axis scanning [20, 26, 27].

Figure 4.Optical simulation of a digital micromirror device (DMD)-based confocal system with constant magnification using an electrically tunable lens (ETL).

The system’s tube lens has a focal length of 200 mm, an entrance pupil diameter of 16 mm, and an overall length of 340 mm when combined with the objective lens. Its modulation transfer function (MTF) performance is 57.8% at 50-line pairs per millimeter (lp/mm), with a distortion of −0.33%. The lens comprises six elements optimized to maintain high image quality and minimize aberrations across the focal range.

The pair of relay lenses maintains a 1:1 optical magnification and ensures telecentricity in both object and image spaces. These lenses are part of the 4f system and are spaced four times their focal lengths apart to prevent magnification changes [19]. When the ETL’s focal length is fixed at 0 diopters, the optical path length, which includes two relay lenses and the ETL, measures 512 mm from the focal plane of the tube lens to the target detector. The MTF for the relay lenses is 53.6% at 50 lp/mm, and they exhibit a slight distortion of −0.51%. This optical setup guarantees accurate z-axis scanning and constant magnification, and preserves image quality throughout the total length of the system.

The detector and DMD are positioned along the same optical path and share the same focal plane via a beam splitter placed after mirror 2. Although the beam splitter is not shown in the optical simulation diagram, it was implemented during the fabrication to ensure proper alignment of the detector and DMD for accurate imaging. This optical design allows dynamic 3D scanning and height measurements with precise z-axis focus adjustments, achieved through the ETL and 4f relay lens system without magnification changes [20, 26, 28].

The system employed an ETL for z-axis scanning for rapid and precise axial adjustments. As shown in the middle of Fig. 4, The ETL used was the EL-16-40-TC-VIS-5D-C model (aperture 16 mm; Optotune Switzerland AG, Dietikon, Swiss), featuring a focal length adjustment range of −2 to +3 diopters. The ETL enabled fast z-axis sectioning without mechanical components, which significantly enhanced the speed and reliability of the system.

Figure 5 illustrates the variation in pinhole intensity as the stage moves along the z-axis. The intensity gradually increases and reaches a maximum y-value of 250 at x = 9.0. The centroid algorithm was used to find this peak. The intensity decreases symmetrically as the stage moves beyond the focal point. This figure visualizes how the pinhole intensity changes with z-stage movement, and the peak represents the optimal focus point.

Figure 5.Intensity distribution curve as a function of z-axis displacement.

The system incorporates the centroid algorithm to enhance z-axis scanning resolution, improving precision in height detection by calculating the weighted average of the intensity values across the focused light spots. This approach allows the system to determine the in-focus position during z-axis scanning accurately. The intensity distribution of light passing through the DMD-generated pinhole array was analyzed for each z-step. The centroid position of each light spot was computed using the formula in Eq. (1):

Zcentroid=Σ Ix, y ×  Zx, yΣ Ix, y,

where I(x, y) represents the intensity at pixel coordinates (x, y), and Z(x, y) is the corresponding height value at each pixel. By applying this algorithm, the system could detect even minor shifts in focal position for better z-axis measurements [2931].

Before evaluating the DMD and ETL confocal microscope, we set up and calibrated the z-axis motion stage to ensure accurate depth measurements. Calibration and height verification were conducted using a Minicom displacement sensor (Tokyo Seimitsu Co., Tokyo, Japan). This high-precision sensor offers a resolution of 0.1 µm for excellent adjustments to the z-stage, which was moved in 1 µm increments to calibrate the stage. We observed maximum measurement errors of +1 µm and −0.5 µm after moving the stage at 100-µm intervals over a 500 µm range following the stage calibration.

Figure 6 shows the linear relationship in the 10× objective working distance as the focal power of the ETL increases from −2 diopter to +4 diopter. As the ETL’s diopter increases, the working distance from the last surface of the objective lens to the specimen consistently decreases. The graph is scaled relative to an offset of +34,500 µm, with zero diopter set as the reference point, and the vertical axis has been adjusted to clearly represent this relative distance.

Figure 6.A chart of working distance variation with the focus adjustment of the electrically tunable lens (ETL).

Through this process, we determined that the system’s working distance changes by 0.009 diopters for every 1 µm of z-axis displacement in practice. This linearity ensures that the ETL can dynamically adjust the focal plane without altering the overall magnification of the system for precise height measurements during 3D scanning. Maintaining a reliable working distance-to-diopter relationship enables consistent z-axis measurements for accuracy in confocal imaging and 3D applications.

For 3D measurement, z-axis scanning was performed using the ETL. The system demonstrated a height resolution of approximately 1 µm, validated through calibration with the displacement sensor. Figure 7 shows the results of the height resolution test. The measured height values were recorded as the z-axis travel was adjusted in increments of 1 µm. These measurements closely align with the expected z-axis movement, indicating high precision across a 500 µm scanning range.

Figure 7.Chart of height resolution test results for z-axis scanning.

The maximum deviation observed was 0.1 µm (from 250 µm to 254.99 µm), demonstrating the accuracy and suitability of the system for high-precision z-axis scanning applications. Additionally, repeatability tests with a 10 µm repetitive movement showed a repeatability of 0.33 µm (3σ), further validating the reliability of the system in dynamic scanning environments.

The optical performance of the system was evaluated to meet the design specifications for 3D measurement. Using a 10× objective lens, the system achieved a height resolution of 1 µm and a lateral resolution of 0.55 µm. When a 20× objective lens is used, the lateral resolution is expected to improve to 0.275 µm. The reliability over the 500 µm z-axis scanning range and a repeatability of 0.33 µm during 10 µm repetitive movements confirm the system’s suitability for high-precision imaging tasks.

Figure 8 shows an image of pinholes focused on the optimal focus plane of the step height standard within the microscope’s FOV. In this experiment, a 20 µm step height standard from VLSI Standards was used to verify the performance of the DMD and ETL confocal microscope. The highlighted yellow indicates the region where intensity and 3D measurements were performed. The pinholes generated by the DMD are evenly distributed across the focal plane for precise alignment for intensity profiling. The camera captures this area specifically for accurate recording of pinhole intensities as the stage moves along the z-axis. This setup ensures that the maximum intensity of the light source is concentrated on the pinholes when they are in their most focused state to provide reliable data for 3D analysis.

Figure 8.Pinhole array on optimal focus plane with measurement area.

Figure 9 presents the results of the confocal microscope system with an analysis program. The left panel displays a 2D image in Fig. 9(a) that highlights the structural features of the step height standard. Due to contamination, some visible particles on the surface do not affect the overall measurement accuracy. The right panel shows a 3D measurement image in Fig. 9(b) that provides a detailed height profile of the same structure, with colors representing the topographical variations. The measured height was 20.96 µm, compared to the nominal height of 20 µm, resulting in a 0.96 µm difference. Although there is a slight difference in measurement, the system has sufficient accuracy for visual inspections and defect detection in industrial environments. The high-speed scanning capability of the system also offers significant advantages for industrial applications. This makes it particularly suitable for real-time quality control and surface inspection tasks where rapid data acquisition and efficient defect detection are needed.

Figure 9.Image analysis results using our 2D/3D measurement program: (a) 2D image (left) and (b) 3D measurement (right) derived from digital micromirror devices (DMD) confocal microscopy data.

Several factors could contribute to this measurement difference, including optical aberrations, errors with signal peak detection, and the need to optimize pinhole size, shape, and spacing. Additionally, full-field calibration, which is more complex than single-point calibration, could help address these issues and ensure consistent results across the entire field of view. Further study on these error factors could improve the accuracy of the system to expand its applicability to areas that demand more precision, such as microfabrication and biological studies.

Table 1 shows a comparative overview of various confocal microscopy techniques, including laser scanning, disc scanning, and the proposed DMD and ETL confocal system. The comparison covers essential features such as 2D and 3D scanning speed, lateral and z-axis resolution, and image quality. In summary, while the proposed system’s z-axis and lateral resolution may not reach the same levels as more traditional laser scanning systems, its overall performance in speed, efficiency, cost, and durability makes it highly advantageous for a broad range of applications [12, 32, 33].

TABLE 1 Comparison of confocal microscopy techniques

FeatureLaser Scanning ConfocalDisc Scanning ConfocalProposed DMD and ETL Confocal
2D Scanning Speed (fps)Slow (1–2)Moderate (10–100)Fast (up to 200)
3D Scanning SpeedSlow (requires mechanical z-scan)Moderate (requires mechanical z-scan)Fast (ETL-based, no mechanical z-scan)
Lateral Resolution (nm)High (200)Moderate (300)Moderate (275)
z-axis Scanning MethodMechanical (Piezo or Motor)Mechanical (Stage)Electronic (ETL)
z-axis Resolution (µm)High (0.5)Moderate (1)Moderate (1)
Image QualityExcellentGoodExcellent
Light EfficiencyLow (due to pinhole)ModerateHigh
ComplexityHigh (due to moving parts)ModerateLow (no mechanical parts)
CostHighModerateLow
DurabilityLow (moving parts prone to wear)ModerateHigh (no moving parts)
Application SuitabilityResearch and MedicalLimited Industry ApplicationsExcellent for Industry and Research

In this study, we developed and tested a novel confocal microscope system that uses DMD and ETL to eliminate the need for mechanical components commonly found in traditional confocal microscopes. The proposed system significantly improves the speed and durability of 3D imaging while maintaining high image quality and precision.

The system’s performance was thoroughly evaluated, and it achieved a lateral resolution of 0.55 µm at 10× magnification and 0.275 µm at 20× magnification, with a height resolution of approximately 1 µm. These results were validated through careful calibration using a displacement sensor. These results confirm the system’s ability to perform precise z-axis scanning without the complexity of mechanical stages. The DMD performed rapid lateral scanning and achieved 2D scanning speeds of up to 200 fps, making it suitable for high-throughput applications. The ETL-based z-axis scanning further enabled fast 3D imaging, eliminating the delays associated with traditional mechanical scanning systems.

Furthermore, the system’s optical design, including a telecentric tube lens and 4f relay lens optics, achieved consistent magnification and minimal image distortion during z-axis adjustments. This design ensures that the system can maintain high precision in various applications, from industrial quality control to biomedical research.

The impact of this technology is significant. By replacing traditional mechanical components with electronic elements such as the DMD and ETL, the system reduces wear and tear, enhances longevity, and lowers maintenance costs. Additionally, its fast and accurate imaging capabilities could revolutionize various fields. It enables rapid, non-contact quality inspections and defect detection in industrial settings, even in high-speed production environments. Biomedical research needs real-time 3D imaging for live cell imaging, tissue analysis, and other applications where speed and resolution are critical.

The DMD and ETL-based confocal microscope system provides a cost-effective, high-performance solution for real-time 3D imaging. Its high resolution, speed, and durability make it a versatile tool for various imaging applications. Embracing this technology could lead to broader advancements in precision imaging and reduce barriers related to cost and complexity in many research and industrial fields.

This research was funded by the Ministry of SMEs and Startups (MSS) as part of a technological innovation project. We want to thank all research team members for their valuable contributions to this project.

This research was supported by the Ministry of SMEs and Startups (MSS), Korea, under the Development of a Digital Optical Confocal 3D Sensing System project (Grant No. TRKO201700017131), which was supervised by the Korea Technology and Information Promotion Agency for SMEs (TIPA).

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

The data that support the findings of this study are available from the corresponding author upon reasonable request. Due to privacy or ethical restrictions, the data is not publicly available.

  1. M. R. Minsky and M. Minsky, “Memoir on inventing the confocal scanning microscope,” Scanning 10, 128-138 (1988).
    CrossRef
  2. M. Minsky, “Microscopy apparatus,” U.S. patent US3013467A (1961).
  3. T. Wilson, C. J. R. Shepparu, and K. Löschke, “Theory and practice of scanning optical microscopy. Academic Press, London 1984, 213 Seiten, 138 Abbildungen, Preis 39.50 ISBN 0-12-757760-2,” Cryst. Res. Technol. 20, 1608 (1985).
    CrossRef
  4. C. J. R. Sheppard and D. M. Shotton, "Confocal Laser Scanning Microscopy," in Royal Microscopy Society Handbooks, (BIOS Scientific Publishers, UK, 1997), Vol. 38, p. 106.
  5. G. J. Brakenhoff, “Imaging modes in confocal scanning light microscopy (CSLM),” J. Microsc. 117, 233-242 (1979).
    CrossRef
  6. M. Petrai, M. Hadravsk, M. D. Egger, and R. Galambos, “Tandem-Scanning Reflected-Light Microscope,” J. Opt. Soc. Am. 58, 661-664 (1968).
    CrossRef
  7. T. Tanaami, S. Otsuki, N. Tomosada, Y. Kosugi, M. Shimizu, and H. Ishida, “High-speed 1-frame/ms scanning confocal microscope with a microlens and Nipkow disks,” Appl. Opt. 41, 4704-4708 (2002).
    Pubmed CrossRef
  8. H.-J. Jeong, H. Yoo, and D. Gweon, “High-speed 3-D measurement with a large field of view based on direct-view confocal microscope with an electrically tunable lens,” Opt. Express 24, 3806-3816 (2016).
    Pubmed CrossRef
  9. D. Dan, M. Lei, B. Yao, W. Wang, M. Winterhalder, A. Zumbusch, Y. Qi, L. Xia, S. Yan, Y. Yang, P. Gao, T. Ye, and W. Zhao, “DMD-based LED-illumination super-resolution and optical sectioning microscopy,” Sci. Rep. 3, 1116 (2013).
    Pubmed KoreaMed CrossRef
  10. C. J. R. Sheppard, “Structured illumination microscopy and image scanning microscopy: A review and comparison of imaging properties,” Philos. Trans. R. Soc. A: Math. Phys. Eng. Sci. 379, 20200154 (2021).
    Pubmed CrossRef
  11. V. Richter, M. Piper, M. Wagner, and H. Schneckenburger, “Increasing resolution in live cell microscopy by structured illumination (SIM),” Appl. Sci. 9, 1188 (2019).
    CrossRef
  12. D. Wang, D. Wang, Y. Meng, Y. Meng, D. Chen, D. Chen, Y. Yam, Y. Yam, S. Chen, and S.-C. Chen, “High-speed 3D imaging based on structured illumination and electrically tunable lens,” Chin. Opt. Lett. 15, 090004 (2017).
    CrossRef
  13. G. J. Tearney, R. H. Webb, and B. E. Bouma, “Spectrally encoded confocal microscopy,” 23, 1152-1154 (1998).
    Pubmed CrossRef
  14. S. Kim, J. Hwang, J. Heo, S. Ryu, D. Lee, S.-H. Kim, S. J. Oh, and C. Joo, “Spectrally encoded slit confocal microscopy using a wavelength-swept laser,” J. Biomed. Opt. 20, 036016 (2015).
    Pubmed CrossRef
  15. L. Chen, M. Ghilardi, J. J. C. Busfield, and F. Carpi, “Electrically tunable lenses: A review,” Front. Robot. AI 8 (2021).
    Pubmed KoreaMed CrossRef
  16. C.-S. Kim, W. Kim, K. Lee, and H. Yoo, “High-speed color three-dimensional measurement based on parallel confocal detection with a focus tunable lens,” Opt. Express 27, 28466-28479 (2019).
    Pubmed CrossRef
  17. D. Rutkauskas and E. Auksorius, “Programmable high-speed confocal reflectance microscopy enabled by a digital micromirror device,” Opt. Lett. 49, 686-689 (2024).
    Pubmed CrossRef
  18. T. Kim, T. Kim, S. H. Kim, S. H. Kim, D. Do, D. Do, H. Yoo, H. Yoo, D. Gweon, and D.-G. Gweon, “Chromatic confocal microscopy with a novel wavelength detection method using transmittance,” Opt. Express 21, 6286-6294 (2013).
    Pubmed CrossRef
  19. M. Agour, C. Fallorf, and R. B. Bergmann, “Fast 3D form measurement using a tunable lens profiler based on imaging with LED illumination,” Opt. Express 29, 385-399 (2021).
    Pubmed CrossRef
  20. Y. Qu and Y. Hu, “Analysis of axial scanning range and magnification variation in wide-field microscope for measurement using an electrically tunable lens,” Microsc. Res. Tech. 82, 101-113 (2019).
    Pubmed CrossRef
  21. J. M. Jabbour, B. H. Malik, C. Olsovsky, R. Cuenca, S. Cheng, J. A. Jo, Y.-S. L. Cheng, J. M. Wright, and K. C. Maitland, “Optical axial scanning in confocal microscopy using an electrically tunable lens,” Biomed. Opt. Express 5, 645-652 (2014).
    Pubmed KoreaMed CrossRef
  22. Y. Zhang, A. Zeng, H. Huang, and W. Hou, “Large-area three-dimensional profilometer based on digital micromirror device,” J. Opt. Technol. 82, 102-107 (2015).
    CrossRef
  23. DLP5500, DLP® 0.55 XGA Series 450 DMD Datasheet, Rev. G, Texas Instruments, TX, USA (2019), Available: https://www.ti.com/product/DLP5500
  24. DLP6500, 0.65 1080p MVSP S600 DMD Datasheet, Rev. B, Texas Instruments, TX, USA (2016), Available: https://www.ti.com/product/DLP6500FYE
  25. V. Markandey, T. Clatanoff, and G. Pettitt, “Video processing for DLP display systems,” Proc. SPIE 2666, 21-32 (1996).
    CrossRef
  26. C. Falldorf, M. Agour, F. Thiemicke, and R. B. Bergmann, “Lens-based phase retrieval under spatially partially coherent illumination-Part I: Theory,” Opt. Lasers Eng. 139, 106507 (2021).
    CrossRef
  27. F. O. Fahrbach, F. F. Voigt, B. Schmid, F. Helmchen, and J. Huisken, “Rapid 3D light-sheet microscopy with a tunable lens,” Opt. Express 21, 21010-21026 (2013).
    Pubmed CrossRef
  28. Y. Cheng, M. Liu, Y. Ou, L. Liu, and Q. Hao, “Colorful 3D reconstruction and an extended depth of field for a monocular biological microscope using an electrically tunable lens,” Biomimetics 9, 49 (2024).
    Pubmed KoreaMed CrossRef
  29. J. Bai, X. Li, X. Wang, Q. Zhou, and K. Ni, “Chromatic confocal displacement sensor with optimized dispersion probe and modified centroid peak extraction algorithm,” Sensors 19, 3592 (2019).
    Pubmed KoreaMed CrossRef
  30. T. Delabie, J. De Schutter, and B. Vandenbussche, “An accurate and efficient Gaussian fit centroiding algorithm for star trackers,” J. Astronaut. Sci. 61, 60-84 (2014).
    CrossRef
  31. A. Patwardhan, “Subpixel position measurement using 1D, 2D and 3D centroid algorithms with emphasis on applications in confocal microscopy,” J. Microsc. 186, 246-257 (1997).
    CrossRef
  32. J. Fuseler, W. G. Jerome, and R. L. Price, "Types of confocal instruments: Basic principles and advantages and disadvantages,," in Basic Confocal Microscopy, 2 nd ed., W. G. Jerome and R. L. Price, Eds (Springer Cham, Swiss, 2018), pp. 187-213.
    CrossRef
  33. D. Toomre and J. B. Pawley, "Disk-scanning confocal microscopy,," in Handbook of Biological Confocal Microscopy, J. B. Pawley, Ed (Springer, USA, 2006), pp. 221-238.
    CrossRef

Article

Research Paper

Curr. Opt. Photon. 2024; 8(6): 632-640

Published online December 25, 2024 https://doi.org/10.3807/COPP.2024.8.6.632

Copyright © Optical Society of Korea.

A Fully Electronic Confocal Microscope Using a Digital Micromirror Device and an Electrically Tunable Lens for 3D Measurement

Sung Hyun Kim1, Min Sung Ko2, Jae Hun Shin3, Jin Sun Yu2, Cha-Hwan Oh1

1Department of Physics, Hanyang University, Seoul 04763, Korea
2Research and Development Department, Basler Korea Inc., Seoul 05836, Korea
3Siwon Optical Technology Co., Anyang 14117, Korea

Correspondence to:*choh@hanyang.ac.kr, ORCID 0000-0003-4772-6696
Current affiliation: ZBIT Co., Ltd., Yongin 16864, Korea

Received: September 30, 2024; Revised: November 18, 2024; Accepted: November 19, 2024

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

With its ability to provide high-resolution, 3D images, the confocal microscope has revolutionized various scientific fields. However, conventional confocal systems often rely on complex mechanical scanning mechanisms, making them less cost-effective, less durable, and limited in speed. We present a 3D confocal microscope that uses a digital micromirror device (DMD) and an electrically tunable lens (ETL) for a fully electronic, high-speed 3D measurement solution. By leveraging the parallel scanning capabilities of DMDs and the precise axial focusing of ETLs, our approach offers significant speed, accuracy, and simplicity advantages. This innovative technology could enable new applications in materials science, biomedical research, and industrial inspection.

Keywords: Confocal microscopy, Digital micromirror device, Electrically tunable lens, High-speed 3D measurement, Non-mechanical scanning

I. INTRODUCTION

Confocal microscopy has long been a cornerstone in high-resolution imaging by offering 3D visualization by selectively blocking out-of-focus light with pinholes. Marvin Minsky first conceptualized this technique in 1955 to improve image clarity by eliminating interference from out-of-focus light [1]. Although Minsky submitted a patent in 1957, the technology did not see widespread use until the development of laser scanning techniques in the 1980s [2]. The first commercial confocal microscopes were introduced in the early 1980s, with systems relying on stage scanning mechanisms to generate high-resolution images [3]. These early systems were innovative but lacked speed and usability due to their mechanical complexity. The advent of laser scanning confocal microscopy (LSCM) enabled more precise imaging by scanning a laser beam across the specimen and collecting light through a pinhole [4].

However, despite this advancement, LSCM systems still face challenges, such as slow imaging speeds due to the reliance on single-point scanning [5]. In the following decades, efforts to improve confocal microscopy focused on increasing speed and reducing system complexity. The Nipkow disk system, developed in 1967, introduced a rotating disk with multiple pinholes for faster image acquisition by scanning multiple points simultaneously. Despite its increased speed, this system suffered from reduced light efficiency [6, 7]. The direct-view confocal microscope (DVCM), based on this system, has continued to face challenges, including light inefficiency and a trade-off between signal intensity and optical sectioning ability [8].

Over the years, various techniques have been developed to enhance the speed and simplify the complexity of confocal microscopy. Structured illumination microscopy (SIM), introduced in the early 2000s, uses spatial light modulators (SLMs) such as digital micromirror devices (DMD) to project structured light patterns onto the sample, enhancing imaging speed and resolution while enabling effective optical sectioning. However, it requires complex pattern generation systems and may exhibit reduced performance under certain imaging conditions [912]. Another approach, spectrally encoded confocal microscopy (SECM), emerged in the late 1990s. It uses SLMs to encode spatial information into different wavelengths, eliminating the need for mechanical lateral scanning. While SECM enhances axial scanning speed with an electrically tunable lens (ETL) and offers unique advantages, it often suffers from reduced light efficiency due to the dispersive nature of its optical components [1317]. Further advancements, such as chromatic confocal microscopy [18], mitigated certain limitations, but the technology still relied on mechanical parts. Consequently, modern confocal microscopes continue to face issues with durability, speed, and cost due to the complexity of their mechanical scanning mechanisms.

This study addresses these limitations by introducing a fully electronic confocal microscope that uses DMD to create virtual pinholes for high-speed lateral scanning and an ETL for rapid axial adjustments. Our novel system can dynamically adjust the size and pitch of the virtual pinholes offering flexibility in optimizing optical sectioning strength and signal intensity. Furthermore, it ensures consistent magnification and minimizes optical distortion across the depth scanning range. To achieve proper pinhole scanning during axial movement, it is crucial to eliminate magnification changes along the optical axis. The proposed system removes mechanical movement and reduces distortion from the ETL optics. This allows for faster and more accurate 3D measurements while decreasing manufacturing costs. This innovation is particularly relevant for industrial and biomedical applications where high-speed and high-precision measurements are essential [19, 20].

II. EXPERIMENTAL METHOD

The confocal system in this study integrates dynamic lateral scanning using a DMD and z-axis scanning with an ETL to achieve fully electronic, high-speed 3D measurement. The system was designed to eliminate mechanical scanning components, thereby improving speed, durability, and cost-efficiency [12, 21].

In the configuration shown in Fig. 1, light from a high-power LED (90W, CBT-90; Luminus Devices Inc., CA, USA) passes through the DMD, which modulates the light beam for precise lateral scanning by adjusting the tilt positions of its micromirrors. This feature enables dynamic control over the illumination patterns, which is critical for high-speed imaging. The modulated light then travels through a pair of relay lenses. These lenses relay the light beam while preserving beam alignment and image quality without altering the magnification. The ETL between the relay lenses facilitates rapid focus adjustments along the z-axis. The ETL eliminates the need for mechanical z-axis scanning, which significantly improves the speed and precision of depth scanning for high-speed 3D imaging. The light subsequently passes through the tube lens and objectives, which focus it on the specimen. When the light interacts with the specimen, it is reflected through the same optical path and is retraced through the objective lens, tube lens, relay lens, ETL, and second relay lenses. At the final stage, the returning light reaches a beam splitter (BS), which directs a portion of the light to the camera. This light is then captured and processed to generate the final image.

Figure 1. Schematic of a digital micromirror devices (DMD)-based scanning confocal microscopy system integrated with an electrically tunable lens (ETL).

The primary focus of this research is the successful implementation and testing of this optical system, which leverages the unique capabilities of the DMD for lateral scanning and the ETL for dynamic z-axis control. This setup offers a robust solution for high-speed, high-precision 3D measurement, making it a valuable tool for industrial and research applications.

Figure 2(a) shows the initial test concept of a DMD-based confocal microscopy system. This setup was designed to evaluate the core functionality of the system before progressing to final integrations. This system employs a laser source that is expanded through a beam expander to provide uniform illumination.

Figure 2. Initial test setup for digital micromirror device (DMD)-based confocal system. (a) Schematic of an experimental setup for testing confocality of DMD’s pinhole array. (b) An experimental DMD-based confocal microscopy system is set up with a high-speed camera and optical components.

This configuration is shown in Fig. 2(b) was primarily used to test the basic functionality of the system by focusing on the DMD’s ability to control the illumination pattern and maintain optical alignment [22]. This setup initially evaluated image quality, resolution, and overall system performance before implementing more advanced features such as relay lenses, ETL, and further optical tuning.

Lateral scanning is performed using the DMD, which consists of an array of micro-mirrors that can be individually tilted to reflect light. The DMD was programmed to create pinhole arrays to scan the sample in the x- and y- directions [9]. Initially, a micromirror device evaluation kit (DLP5500; Texas Instruments, TX, USA) with a resolution of 1,024 × 768 pixels and a micromirror size of 10.8 µm was used [23]. The DLP6500 (Texas Instruments) with a 1,920 × 1,080 pixels resolution and a smaller micromirror size of 7.6 µm, was later integrated to improve spatial accuracy and resolution [24]. The DLP6500 also provided faster switching speeds, making it more suitable for high-precision 3D measurement applications [17].

This configuration used a green laser (λ = 532 nm, 40 mW, Z40M18B-F-532-pz; Z-laser GmbH, Freiburg im Breisgau, Germany) for illumination during lateral scanning. The reflected light from the sample was captured by a high-resolution camera (4M pixels, acA2040-180 km; Basler AG, Ahrensburg, Germany) to ensure accurate mapping of focus points across the lateral plane. The system processed images in real time by uploading pinhole array patterns (stored as 1-bit bitmap image files) to the memory of the main board. The captured images confirmed the system’s ability to adjust scanning parameters dynamically for flexible and high-speed 2D and 3D measurements [9, 25].

To verify the confocality of the system, a series of images were captured as the z-stage (Travel = 4 mm, LS6H-688; Misumi Co., Tokyo, Japan), previously calibrated using a displacement sensor, was displaced by ±3 µm around the focal plane, as shown in Fig. 3.

Figure 3. Captured images showing the effect of z-stage displacement on the DMD-generated pinhole array: (a) −3 µm movement, (b) optimal focus position, and (c) +3 µm movement.

At optimal focus (0 µm displacement), the pinhole array appeared sharp and well-defined, confirming that the system was correctly aligned with the focal plane. When the z-stage was displaced by −3 µm, blurring was observed in the top-right corner, while a displacement by +3 µm caused blurring in the bottom-left corner of the image. This demonstrated the system’s sensitivity to focal plane misalignment and further confirmed the accuracy of both the lateral and z-axis scanning capabilities.

The bright spots in the images represent the pinhole array generated by the DMD. Each pinhole was designed to be 2 pixels by 2 pixels for a well-defined image. The spacing between the pinholes was set to 50 pixels for a clear distinction between individual spots. The experiment revealed that the depth of focus of the pinhole is at least ±3 µm or less. This test considers these parameters to optimize confocality testing and demonstrate the ability of the system to maintain resolution and clarity in different z-axis positions.

As shown in Fig. 4, the confocal microscope system was designed using Zemax software to simulate light paths and optimize optical components for dynamic 3D scanning. The optical design consists primarily of two key sections: The tube lens assembly, which includes the objective lens, and a pair of relay lenses. This configuration allows the system to maintain high image quality and precise magnification control during z-axis scanning. The pair of relay lenses serves the crucial purpose of implementing the 4f system so that any changes in the focal length of the ETL do not affect the magnification of the system, which is essential to maintain image accuracy during z-axis scanning [20, 26, 27].

Figure 4. Optical simulation of a digital micromirror device (DMD)-based confocal system with constant magnification using an electrically tunable lens (ETL).

The system’s tube lens has a focal length of 200 mm, an entrance pupil diameter of 16 mm, and an overall length of 340 mm when combined with the objective lens. Its modulation transfer function (MTF) performance is 57.8% at 50-line pairs per millimeter (lp/mm), with a distortion of −0.33%. The lens comprises six elements optimized to maintain high image quality and minimize aberrations across the focal range.

The pair of relay lenses maintains a 1:1 optical magnification and ensures telecentricity in both object and image spaces. These lenses are part of the 4f system and are spaced four times their focal lengths apart to prevent magnification changes [19]. When the ETL’s focal length is fixed at 0 diopters, the optical path length, which includes two relay lenses and the ETL, measures 512 mm from the focal plane of the tube lens to the target detector. The MTF for the relay lenses is 53.6% at 50 lp/mm, and they exhibit a slight distortion of −0.51%. This optical setup guarantees accurate z-axis scanning and constant magnification, and preserves image quality throughout the total length of the system.

The detector and DMD are positioned along the same optical path and share the same focal plane via a beam splitter placed after mirror 2. Although the beam splitter is not shown in the optical simulation diagram, it was implemented during the fabrication to ensure proper alignment of the detector and DMD for accurate imaging. This optical design allows dynamic 3D scanning and height measurements with precise z-axis focus adjustments, achieved through the ETL and 4f relay lens system without magnification changes [20, 26, 28].

The system employed an ETL for z-axis scanning for rapid and precise axial adjustments. As shown in the middle of Fig. 4, The ETL used was the EL-16-40-TC-VIS-5D-C model (aperture 16 mm; Optotune Switzerland AG, Dietikon, Swiss), featuring a focal length adjustment range of −2 to +3 diopters. The ETL enabled fast z-axis sectioning without mechanical components, which significantly enhanced the speed and reliability of the system.

Figure 5 illustrates the variation in pinhole intensity as the stage moves along the z-axis. The intensity gradually increases and reaches a maximum y-value of 250 at x = 9.0. The centroid algorithm was used to find this peak. The intensity decreases symmetrically as the stage moves beyond the focal point. This figure visualizes how the pinhole intensity changes with z-stage movement, and the peak represents the optimal focus point.

Figure 5. Intensity distribution curve as a function of z-axis displacement.

The system incorporates the centroid algorithm to enhance z-axis scanning resolution, improving precision in height detection by calculating the weighted average of the intensity values across the focused light spots. This approach allows the system to determine the in-focus position during z-axis scanning accurately. The intensity distribution of light passing through the DMD-generated pinhole array was analyzed for each z-step. The centroid position of each light spot was computed using the formula in Eq. (1):

Zcentroid=Σ Ix, y ×  Zx, yΣ Ix, y,

where I(x, y) represents the intensity at pixel coordinates (x, y), and Z(x, y) is the corresponding height value at each pixel. By applying this algorithm, the system could detect even minor shifts in focal position for better z-axis measurements [2931].

III. RESULT AND DISCUSSION

Before evaluating the DMD and ETL confocal microscope, we set up and calibrated the z-axis motion stage to ensure accurate depth measurements. Calibration and height verification were conducted using a Minicom displacement sensor (Tokyo Seimitsu Co., Tokyo, Japan). This high-precision sensor offers a resolution of 0.1 µm for excellent adjustments to the z-stage, which was moved in 1 µm increments to calibrate the stage. We observed maximum measurement errors of +1 µm and −0.5 µm after moving the stage at 100-µm intervals over a 500 µm range following the stage calibration.

Figure 6 shows the linear relationship in the 10× objective working distance as the focal power of the ETL increases from −2 diopter to +4 diopter. As the ETL’s diopter increases, the working distance from the last surface of the objective lens to the specimen consistently decreases. The graph is scaled relative to an offset of +34,500 µm, with zero diopter set as the reference point, and the vertical axis has been adjusted to clearly represent this relative distance.

Figure 6. A chart of working distance variation with the focus adjustment of the electrically tunable lens (ETL).

Through this process, we determined that the system’s working distance changes by 0.009 diopters for every 1 µm of z-axis displacement in practice. This linearity ensures that the ETL can dynamically adjust the focal plane without altering the overall magnification of the system for precise height measurements during 3D scanning. Maintaining a reliable working distance-to-diopter relationship enables consistent z-axis measurements for accuracy in confocal imaging and 3D applications.

For 3D measurement, z-axis scanning was performed using the ETL. The system demonstrated a height resolution of approximately 1 µm, validated through calibration with the displacement sensor. Figure 7 shows the results of the height resolution test. The measured height values were recorded as the z-axis travel was adjusted in increments of 1 µm. These measurements closely align with the expected z-axis movement, indicating high precision across a 500 µm scanning range.

Figure 7. Chart of height resolution test results for z-axis scanning.

The maximum deviation observed was 0.1 µm (from 250 µm to 254.99 µm), demonstrating the accuracy and suitability of the system for high-precision z-axis scanning applications. Additionally, repeatability tests with a 10 µm repetitive movement showed a repeatability of 0.33 µm (3σ), further validating the reliability of the system in dynamic scanning environments.

The optical performance of the system was evaluated to meet the design specifications for 3D measurement. Using a 10× objective lens, the system achieved a height resolution of 1 µm and a lateral resolution of 0.55 µm. When a 20× objective lens is used, the lateral resolution is expected to improve to 0.275 µm. The reliability over the 500 µm z-axis scanning range and a repeatability of 0.33 µm during 10 µm repetitive movements confirm the system’s suitability for high-precision imaging tasks.

Figure 8 shows an image of pinholes focused on the optimal focus plane of the step height standard within the microscope’s FOV. In this experiment, a 20 µm step height standard from VLSI Standards was used to verify the performance of the DMD and ETL confocal microscope. The highlighted yellow indicates the region where intensity and 3D measurements were performed. The pinholes generated by the DMD are evenly distributed across the focal plane for precise alignment for intensity profiling. The camera captures this area specifically for accurate recording of pinhole intensities as the stage moves along the z-axis. This setup ensures that the maximum intensity of the light source is concentrated on the pinholes when they are in their most focused state to provide reliable data for 3D analysis.

Figure 8. Pinhole array on optimal focus plane with measurement area.

Figure 9 presents the results of the confocal microscope system with an analysis program. The left panel displays a 2D image in Fig. 9(a) that highlights the structural features of the step height standard. Due to contamination, some visible particles on the surface do not affect the overall measurement accuracy. The right panel shows a 3D measurement image in Fig. 9(b) that provides a detailed height profile of the same structure, with colors representing the topographical variations. The measured height was 20.96 µm, compared to the nominal height of 20 µm, resulting in a 0.96 µm difference. Although there is a slight difference in measurement, the system has sufficient accuracy for visual inspections and defect detection in industrial environments. The high-speed scanning capability of the system also offers significant advantages for industrial applications. This makes it particularly suitable for real-time quality control and surface inspection tasks where rapid data acquisition and efficient defect detection are needed.

Figure 9. Image analysis results using our 2D/3D measurement program: (a) 2D image (left) and (b) 3D measurement (right) derived from digital micromirror devices (DMD) confocal microscopy data.

Several factors could contribute to this measurement difference, including optical aberrations, errors with signal peak detection, and the need to optimize pinhole size, shape, and spacing. Additionally, full-field calibration, which is more complex than single-point calibration, could help address these issues and ensure consistent results across the entire field of view. Further study on these error factors could improve the accuracy of the system to expand its applicability to areas that demand more precision, such as microfabrication and biological studies.

Table 1 shows a comparative overview of various confocal microscopy techniques, including laser scanning, disc scanning, and the proposed DMD and ETL confocal system. The comparison covers essential features such as 2D and 3D scanning speed, lateral and z-axis resolution, and image quality. In summary, while the proposed system’s z-axis and lateral resolution may not reach the same levels as more traditional laser scanning systems, its overall performance in speed, efficiency, cost, and durability makes it highly advantageous for a broad range of applications [12, 32, 33].

TABLE 1. Comparison of confocal microscopy techniques.

FeatureLaser Scanning ConfocalDisc Scanning ConfocalProposed DMD and ETL Confocal
2D Scanning Speed (fps)Slow (1–2)Moderate (10–100)Fast (up to 200)
3D Scanning SpeedSlow (requires mechanical z-scan)Moderate (requires mechanical z-scan)Fast (ETL-based, no mechanical z-scan)
Lateral Resolution (nm)High (200)Moderate (300)Moderate (275)
z-axis Scanning MethodMechanical (Piezo or Motor)Mechanical (Stage)Electronic (ETL)
z-axis Resolution (µm)High (0.5)Moderate (1)Moderate (1)
Image QualityExcellentGoodExcellent
Light EfficiencyLow (due to pinhole)ModerateHigh
ComplexityHigh (due to moving parts)ModerateLow (no mechanical parts)
CostHighModerateLow
DurabilityLow (moving parts prone to wear)ModerateHigh (no moving parts)
Application SuitabilityResearch and MedicalLimited Industry ApplicationsExcellent for Industry and Research

IV. CONCLUSION

In this study, we developed and tested a novel confocal microscope system that uses DMD and ETL to eliminate the need for mechanical components commonly found in traditional confocal microscopes. The proposed system significantly improves the speed and durability of 3D imaging while maintaining high image quality and precision.

The system’s performance was thoroughly evaluated, and it achieved a lateral resolution of 0.55 µm at 10× magnification and 0.275 µm at 20× magnification, with a height resolution of approximately 1 µm. These results were validated through careful calibration using a displacement sensor. These results confirm the system’s ability to perform precise z-axis scanning without the complexity of mechanical stages. The DMD performed rapid lateral scanning and achieved 2D scanning speeds of up to 200 fps, making it suitable for high-throughput applications. The ETL-based z-axis scanning further enabled fast 3D imaging, eliminating the delays associated with traditional mechanical scanning systems.

Furthermore, the system’s optical design, including a telecentric tube lens and 4f relay lens optics, achieved consistent magnification and minimal image distortion during z-axis adjustments. This design ensures that the system can maintain high precision in various applications, from industrial quality control to biomedical research.

The impact of this technology is significant. By replacing traditional mechanical components with electronic elements such as the DMD and ETL, the system reduces wear and tear, enhances longevity, and lowers maintenance costs. Additionally, its fast and accurate imaging capabilities could revolutionize various fields. It enables rapid, non-contact quality inspections and defect detection in industrial settings, even in high-speed production environments. Biomedical research needs real-time 3D imaging for live cell imaging, tissue analysis, and other applications where speed and resolution are critical.

The DMD and ETL-based confocal microscope system provides a cost-effective, high-performance solution for real-time 3D imaging. Its high resolution, speed, and durability make it a versatile tool for various imaging applications. Embracing this technology could lead to broader advancements in precision imaging and reduce barriers related to cost and complexity in many research and industrial fields.

Acknowledgments

This research was funded by the Ministry of SMEs and Startups (MSS) as part of a technological innovation project. We want to thank all research team members for their valuable contributions to this project.

FUNDING

This research was supported by the Ministry of SMEs and Startups (MSS), Korea, under the Development of a Digital Optical Confocal 3D Sensing System project (Grant No. TRKO201700017131), which was supervised by the Korea Technology and Information Promotion Agency for SMEs (TIPA).

DISCLOSURES

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

DATA AVAILABILITY

The data that support the findings of this study are available from the corresponding author upon reasonable request. Due to privacy or ethical restrictions, the data is not publicly available.

Fig 1.

Figure 1.Schematic of a digital micromirror devices (DMD)-based scanning confocal microscopy system integrated with an electrically tunable lens (ETL).
Current Optics and Photonics 2024; 8: 632-640https://doi.org/10.3807/COPP.2024.8.6.632

Fig 2.

Figure 2.Initial test setup for digital micromirror device (DMD)-based confocal system. (a) Schematic of an experimental setup for testing confocality of DMD’s pinhole array. (b) An experimental DMD-based confocal microscopy system is set up with a high-speed camera and optical components.
Current Optics and Photonics 2024; 8: 632-640https://doi.org/10.3807/COPP.2024.8.6.632

Fig 3.

Figure 3.Captured images showing the effect of z-stage displacement on the DMD-generated pinhole array: (a) −3 µm movement, (b) optimal focus position, and (c) +3 µm movement.
Current Optics and Photonics 2024; 8: 632-640https://doi.org/10.3807/COPP.2024.8.6.632

Fig 4.

Figure 4.Optical simulation of a digital micromirror device (DMD)-based confocal system with constant magnification using an electrically tunable lens (ETL).
Current Optics and Photonics 2024; 8: 632-640https://doi.org/10.3807/COPP.2024.8.6.632

Fig 5.

Figure 5.Intensity distribution curve as a function of z-axis displacement.
Current Optics and Photonics 2024; 8: 632-640https://doi.org/10.3807/COPP.2024.8.6.632

Fig 6.

Figure 6.A chart of working distance variation with the focus adjustment of the electrically tunable lens (ETL).
Current Optics and Photonics 2024; 8: 632-640https://doi.org/10.3807/COPP.2024.8.6.632

Fig 7.

Figure 7.Chart of height resolution test results for z-axis scanning.
Current Optics and Photonics 2024; 8: 632-640https://doi.org/10.3807/COPP.2024.8.6.632

Fig 8.

Figure 8.Pinhole array on optimal focus plane with measurement area.
Current Optics and Photonics 2024; 8: 632-640https://doi.org/10.3807/COPP.2024.8.6.632

Fig 9.

Figure 9.Image analysis results using our 2D/3D measurement program: (a) 2D image (left) and (b) 3D measurement (right) derived from digital micromirror devices (DMD) confocal microscopy data.
Current Optics and Photonics 2024; 8: 632-640https://doi.org/10.3807/COPP.2024.8.6.632

TABLE 1 Comparison of confocal microscopy techniques

FeatureLaser Scanning ConfocalDisc Scanning ConfocalProposed DMD and ETL Confocal
2D Scanning Speed (fps)Slow (1–2)Moderate (10–100)Fast (up to 200)
3D Scanning SpeedSlow (requires mechanical z-scan)Moderate (requires mechanical z-scan)Fast (ETL-based, no mechanical z-scan)
Lateral Resolution (nm)High (200)Moderate (300)Moderate (275)
z-axis Scanning MethodMechanical (Piezo or Motor)Mechanical (Stage)Electronic (ETL)
z-axis Resolution (µm)High (0.5)Moderate (1)Moderate (1)
Image QualityExcellentGoodExcellent
Light EfficiencyLow (due to pinhole)ModerateHigh
ComplexityHigh (due to moving parts)ModerateLow (no mechanical parts)
CostHighModerateLow
DurabilityLow (moving parts prone to wear)ModerateHigh (no moving parts)
Application SuitabilityResearch and MedicalLimited Industry ApplicationsExcellent for Industry and Research

References

  1. M. R. Minsky and M. Minsky, “Memoir on inventing the confocal scanning microscope,” Scanning 10, 128-138 (1988).
    CrossRef
  2. M. Minsky, “Microscopy apparatus,” U.S. patent US3013467A (1961).
  3. T. Wilson, C. J. R. Shepparu, and K. Löschke, “Theory and practice of scanning optical microscopy. Academic Press, London 1984, 213 Seiten, 138 Abbildungen, Preis 39.50 ISBN 0-12-757760-2,” Cryst. Res. Technol. 20, 1608 (1985).
    CrossRef
  4. C. J. R. Sheppard and D. M. Shotton, "Confocal Laser Scanning Microscopy," in Royal Microscopy Society Handbooks, (BIOS Scientific Publishers, UK, 1997), Vol. 38, p. 106.
  5. G. J. Brakenhoff, “Imaging modes in confocal scanning light microscopy (CSLM),” J. Microsc. 117, 233-242 (1979).
    CrossRef
  6. M. Petrai, M. Hadravsk, M. D. Egger, and R. Galambos, “Tandem-Scanning Reflected-Light Microscope,” J. Opt. Soc. Am. 58, 661-664 (1968).
    CrossRef
  7. T. Tanaami, S. Otsuki, N. Tomosada, Y. Kosugi, M. Shimizu, and H. Ishida, “High-speed 1-frame/ms scanning confocal microscope with a microlens and Nipkow disks,” Appl. Opt. 41, 4704-4708 (2002).
    Pubmed CrossRef
  8. H.-J. Jeong, H. Yoo, and D. Gweon, “High-speed 3-D measurement with a large field of view based on direct-view confocal microscope with an electrically tunable lens,” Opt. Express 24, 3806-3816 (2016).
    Pubmed CrossRef
  9. D. Dan, M. Lei, B. Yao, W. Wang, M. Winterhalder, A. Zumbusch, Y. Qi, L. Xia, S. Yan, Y. Yang, P. Gao, T. Ye, and W. Zhao, “DMD-based LED-illumination super-resolution and optical sectioning microscopy,” Sci. Rep. 3, 1116 (2013).
    Pubmed KoreaMed CrossRef
  10. C. J. R. Sheppard, “Structured illumination microscopy and image scanning microscopy: A review and comparison of imaging properties,” Philos. Trans. R. Soc. A: Math. Phys. Eng. Sci. 379, 20200154 (2021).
    Pubmed CrossRef
  11. V. Richter, M. Piper, M. Wagner, and H. Schneckenburger, “Increasing resolution in live cell microscopy by structured illumination (SIM),” Appl. Sci. 9, 1188 (2019).
    CrossRef
  12. D. Wang, D. Wang, Y. Meng, Y. Meng, D. Chen, D. Chen, Y. Yam, Y. Yam, S. Chen, and S.-C. Chen, “High-speed 3D imaging based on structured illumination and electrically tunable lens,” Chin. Opt. Lett. 15, 090004 (2017).
    CrossRef
  13. G. J. Tearney, R. H. Webb, and B. E. Bouma, “Spectrally encoded confocal microscopy,” 23, 1152-1154 (1998).
    Pubmed CrossRef
  14. S. Kim, J. Hwang, J. Heo, S. Ryu, D. Lee, S.-H. Kim, S. J. Oh, and C. Joo, “Spectrally encoded slit confocal microscopy using a wavelength-swept laser,” J. Biomed. Opt. 20, 036016 (2015).
    Pubmed CrossRef
  15. L. Chen, M. Ghilardi, J. J. C. Busfield, and F. Carpi, “Electrically tunable lenses: A review,” Front. Robot. AI 8 (2021).
    Pubmed KoreaMed CrossRef
  16. C.-S. Kim, W. Kim, K. Lee, and H. Yoo, “High-speed color three-dimensional measurement based on parallel confocal detection with a focus tunable lens,” Opt. Express 27, 28466-28479 (2019).
    Pubmed CrossRef
  17. D. Rutkauskas and E. Auksorius, “Programmable high-speed confocal reflectance microscopy enabled by a digital micromirror device,” Opt. Lett. 49, 686-689 (2024).
    Pubmed CrossRef
  18. T. Kim, T. Kim, S. H. Kim, S. H. Kim, D. Do, D. Do, H. Yoo, H. Yoo, D. Gweon, and D.-G. Gweon, “Chromatic confocal microscopy with a novel wavelength detection method using transmittance,” Opt. Express 21, 6286-6294 (2013).
    Pubmed CrossRef
  19. M. Agour, C. Fallorf, and R. B. Bergmann, “Fast 3D form measurement using a tunable lens profiler based on imaging with LED illumination,” Opt. Express 29, 385-399 (2021).
    Pubmed CrossRef
  20. Y. Qu and Y. Hu, “Analysis of axial scanning range and magnification variation in wide-field microscope for measurement using an electrically tunable lens,” Microsc. Res. Tech. 82, 101-113 (2019).
    Pubmed CrossRef
  21. J. M. Jabbour, B. H. Malik, C. Olsovsky, R. Cuenca, S. Cheng, J. A. Jo, Y.-S. L. Cheng, J. M. Wright, and K. C. Maitland, “Optical axial scanning in confocal microscopy using an electrically tunable lens,” Biomed. Opt. Express 5, 645-652 (2014).
    Pubmed KoreaMed CrossRef
  22. Y. Zhang, A. Zeng, H. Huang, and W. Hou, “Large-area three-dimensional profilometer based on digital micromirror device,” J. Opt. Technol. 82, 102-107 (2015).
    CrossRef
  23. DLP5500, DLP® 0.55 XGA Series 450 DMD Datasheet, Rev. G, Texas Instruments, TX, USA (2019), Available: https://www.ti.com/product/DLP5500
  24. DLP6500, 0.65 1080p MVSP S600 DMD Datasheet, Rev. B, Texas Instruments, TX, USA (2016), Available: https://www.ti.com/product/DLP6500FYE
  25. V. Markandey, T. Clatanoff, and G. Pettitt, “Video processing for DLP display systems,” Proc. SPIE 2666, 21-32 (1996).
    CrossRef
  26. C. Falldorf, M. Agour, F. Thiemicke, and R. B. Bergmann, “Lens-based phase retrieval under spatially partially coherent illumination-Part I: Theory,” Opt. Lasers Eng. 139, 106507 (2021).
    CrossRef
  27. F. O. Fahrbach, F. F. Voigt, B. Schmid, F. Helmchen, and J. Huisken, “Rapid 3D light-sheet microscopy with a tunable lens,” Opt. Express 21, 21010-21026 (2013).
    Pubmed CrossRef
  28. Y. Cheng, M. Liu, Y. Ou, L. Liu, and Q. Hao, “Colorful 3D reconstruction and an extended depth of field for a monocular biological microscope using an electrically tunable lens,” Biomimetics 9, 49 (2024).
    Pubmed KoreaMed CrossRef
  29. J. Bai, X. Li, X. Wang, Q. Zhou, and K. Ni, “Chromatic confocal displacement sensor with optimized dispersion probe and modified centroid peak extraction algorithm,” Sensors 19, 3592 (2019).
    Pubmed KoreaMed CrossRef
  30. T. Delabie, J. De Schutter, and B. Vandenbussche, “An accurate and efficient Gaussian fit centroiding algorithm for star trackers,” J. Astronaut. Sci. 61, 60-84 (2014).
    CrossRef
  31. A. Patwardhan, “Subpixel position measurement using 1D, 2D and 3D centroid algorithms with emphasis on applications in confocal microscopy,” J. Microsc. 186, 246-257 (1997).
    CrossRef
  32. J. Fuseler, W. G. Jerome, and R. L. Price, "Types of confocal instruments: Basic principles and advantages and disadvantages,," in Basic Confocal Microscopy, 2 nd ed., W. G. Jerome and R. L. Price, Eds (Springer Cham, Swiss, 2018), pp. 187-213.
    CrossRef
  33. D. Toomre and J. B. Pawley, "Disk-scanning confocal microscopy,," in Handbook of Biological Confocal Microscopy, J. B. Pawley, Ed (Springer, USA, 2006), pp. 221-238.
    CrossRef