Ex) Article Title, Author, Keywords
Current Optics
and Photonics
Ex) Article Title, Author, Keywords
Curr. Opt. Photon. 2024; 8(1): 1-15
Published online February 25, 2024 https://doi.org/10.3807/COPP.2024.8.1.1
Copyright © Optical Society of Korea.
Youngrok Kim1, Ki-Hong Choi2, Chihyun In1, Keehoon Hong2, Sung-Wook Min1
Corresponding author: *mins@khu.ac.kr, ORCID 0000-0003-4794-356X
†These authors contributed equally to this paper.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
This paper presents a brief introduction to self-interference incoherent digital holography (SIDH). Holography conducted under incoherent light conditions has various advantages over digital holography performed with a conventional coherent light source. We categorize the methods for SIDH, which divides the incident light into two waves and modulates them differently. We also explore various optical concepts and techniques for the implementation and advancement of SIDH. This review presents the system design, performance analysis, and improvement of SIDH, as well as recent applications of SIDH, including optical sectioning and deep-learning-based SIDH.
Keywords: Digital holography, Incoherent holography, Self-interference
OCIS codes: (090.1995) Digital holography; (090.2880) Holographic interferometry
Since the basic principle of holography was invented by Gabor [1–3] as a three-dimensional (3D) information-recording and -reproduction technique, holography has developed rapidly. Interference is a phenomenon that depends on the wave nature of light, and requires the light waves to be coherent. The invention of the laser has enabled the construction of highly coherent and purified light sources, which has enabled the widespread study of holography in various fields, including holographic microscopy [4, 5], holographic displays [6–9], and computer-generated holograms [10, 11].
However, holography under incoherent-illumination conditions has also received attention. Laser illumination can significantly damage photosensitive materials and biological samples. Furthermore, applying fluorescence-imaging methods to in vivo, nondestructive measurements is difficult with conventional holographic techniques. More importantly, to apply holography in daily life we must overcome the limitations of light sources, and thus require holography based on general light source.
T. Young demonstrated the wavelike nature of light through interference experiments conducted in 1801 [12]. His famous interference experiment used sunlight passing through a pinhole, with the thin beam being split in half using a paper card. This improved the spatial coherence of the light, enabling interference even under incoherent illumination. In 1887, a Michelson-Morley experiment was conducted to detect the presence of the luminiferous ether [13]. Despite unsuccessful attempts to confirm the existence of the ether through changes in interference patterns, the related interferometer became the motivation for many later inventions.
Early incoherent holography was mostly based on the wavefront-dividing interferometers that Fresnel proposed [14–17]. Among his numerous accomplishments in wave optics, Fresnel proposed optical systems for capturing interference patterns before the concept of the laser was introduced. Michelson interferometers are powerful tools for incoherent holography. An orthogonal design comprising a beam splitter and mirrors can be exploited for a wavefront divider and the spatial modulation of light waves. Rotational-shearing interferometers form fringe patterns using a combination of tilted prisms, which induce optical-path differences based on the rotation angles.
The invention of the acousto-optic modulator was the primary turning point in the development of incoherent holography. Poon et al. [18] proposed optical scanning holography (OSH), which is based on optical heterodyning. This comprises a time-dependent Fresnel-zone plate and the scanning of a three-dimensional (3D) object in two dimensions. Various applications based on OSH have been proposed, including holographic microscopy and 3D displays [19].
Developments in liquid-crystal and semiconductor technologies have introduced a new era of displays, i.e. liquid-crystal displays. The polarization-selective properties of liquid crystals enable their use in a multitude of optical applications. Spatial light modulators (SLMs) enable wavefront modulation, which has resulted in significant advancements in holography. Fresnel incoherent correlation holography (FINCH) is a polarization-based incoherent digital holography technique that exploits phase-only SLMs (PSLMs) to divide and modulate wavefronts and to form self-interference fringe patterns [20]. Since its proposal by Rosen and Brooker [20] numerous methods have been suggested for self-interference incoherent holography.
This review is an introduction to self-interference incoherent digital holography (SIDH), from its basic principles to recent applications. According to the concept of self-interference, waves from the same point exhibit mutual coherence and interfere with each other. Thus, the main optical component of SIDH is a wavefront divider. In Section 2, we introduce the fundamentals of SIDH and various proposed optical configurations. Moreover, an analysis of the characteristics of SIDH is presented. Section 3 presents the system design, performance analysis, and improvement of SIDH. Section 4 focuses on recent applications of SIDH, and Section 5 offers concluding remarks.
The overlap of two mutually coherent wavefronts on the observation plane results in a distinctive pattern of dark and bright intensities. To ensure mutual coherence between two overlapping wavefronts, conventional interferometry requires a laser as the illumination source, and the beam-division module is configured immediately after the light source to guide the two paths for the object and reference beams. Conversely, self-interference uses a wavefront that has already entered the system. The input wavefront is divided into two parts to induce mutual interference. The parameters of the twin wavefronts originating from the same object point are subtly altered, using various methods. This ensures that the difference in the optical-path length between the two waves remains shorter than the coherence length of the light source used, thereby enabling their mutual interference. For example, if the wavefront division and modulation are adjusted to marginally alter the propagation vector of the input plane wave, the resulting interferogram resembles a grating pattern. Alternatively, if modulation is employed to produce spherical wavefronts with varying radii of curvature, the Fresnel-zone pattern (FZP) is obtained.
In practical self-interferometer systems, the division and modulation components can be either separate or integrated into a single component. The former approach draws largely from traditional interferometric systems, whereas the latter uses recently developed polarization-sensitive optical elements. Widely known interferometric structures, such as rotational shearing and Michelson interferometers, were adopted in the early stages of incoherent holograms as spatial-division methods.
Figure 1 illustrates the simplified optical structure of widely reported SIDH systems. The object to be recorded is regarded as a group of object points, for which the response functions of each point are independent of each other. To describe the wavefront-modulation procedure in the simplified SIDH system, we begin with a spherical wave diverging from an arbitrary single object point. The spherical wavefront of the object travels a distance zo to the front of the system, just in front of the field lens. When the initial amplitude and x-y coordinates of the object point source are C1(
After traveling a distance d from the field lens to the wavefront modulator, the field encounters the transmission function of the wavefront modulator. We assume that the wavefront modulator is a transmissive bifocal lens with focal lengths of f1 and f2. To employ the phase-shifting method to eliminate the bias and twin images, let us also assume that the bifocal lens can introduce a phase-shifting effect on one of the output wavefronts. Here the transmission function of the bifocal lens tbi is
where δ is the phase-shift value. After the bifocal lens, the field propagates further to the image sensor, which is located behind the bifocal lens at a distance zh. If the mutual coherence of the two wavefronts arriving at the image-sensor plane is maintained, interference occurs between the two spherical waves. Therefore, the intensity pattern of the interferogram, which contains the holographic information of the object point with the bias and twin-image information, is derived as follows:
In Eq. (3), the symbol ⊗ refers to the 2D convolution operation. Equation (3) can be simplified by noting that a spherical wave remains spherical after passing through an ideal lens [21]. When the original spherical wavefront is C(
where zrec is the reconstruction distance of the recorded hologram, M is the magnification factor of the system. C1,2,3,4 are coefficients for complex constant and C4′ means the conjugate number of C4. zrec and M are expressed as follows:
The parametrized Eqs. (5) and (6) are derived from the two-lens relaying system, where the first lens has a focal length of fo and the second is a bifocal lens with focal lengths of f1 and f2. The first imaging distance zi is calculated using the thin-lens equation, where the object distance is zo and the focal length of the lens is fo, which is
As described in Eq. (4), the desired hologram information exists with undesired bias and twin-image information. Several methods can be used to reduce or eliminate the bias and twin images, and the phase-shifting technique can in principle completely eliminate the bias and twin images. To apply the phase-shifting technique, we vary δ in Eq. (4) to obtain multiple images of Is. For the four-step phase-shifting technique with 90˚ intervals, the values of δ are [0, π/2, π, 3π/2]. The obtained Is images are as follows:
The complex hologram H without bias and twin-image noise is then obtained by combining Eq. (7) as follows:
As shown in Eq. (8), the obtained hologram H is the impulse-response function of the object point located at (xo, yo) at a distance of zo from the very front of the system. The obtained hologram can be reconstructed to that point again with a propagation distance of −zrec using a widely known numerical reconstruction algorithm, such as the angular-spectrum or Fresnel propagation method [2]. Because zrec depends on the axial location of the object zo, the axial location of each object point source is encoded by its own impulse-response function and can then be decoded again with the distinguished axial location.
Because the coherence of the light source used in this system is low compared to that of a laser-based interferometric system, there is hardly any mutual coherence of H from different point sources. Therefore, the intensity distribution at the sensor plane results from multiple point sources, and volumetric objects are incoherent summations of each impulse-response function. An arbitrary 3D object with a group of point sources is described as follows:
where Ao is the initial amplitude of each object points and δ is the phase-shifting angle.
In this section, we discuss research on various system designs, performance analyses, and improvements of SIDH. Through various studies, a number of wavefront-modulation techniques for splitting and modulating the incident wavefront, and a phase-shifting method for removing the bias and twin images, have been proposed. We introduce research that has increased the system magnification and resolution, or reduced the magnification but simplified the form factor. We also present research on full-color and real-time recording and introduce analyses of increasing the magnification or resolution by improving the system configuration.
The SIDH system configuration is divided into two main parts: A wavefront modulator and a phase shifter. Studies have proposed each of these parts as separate components or integrated them to simplify the system. Depending on the wavefront-division method, a conventional wavefront-separating interferometer can be used, or a common-path-type interferometer can be constructed. Phase shifters can be divided into three main categories: The physical moving of optical elements, introducing retardation between interfering wavefronts using birefringent media, and introducing a geometric phase using a combination of polarization elements and their rotation states. The description of the design is divided into three categories based on the method of wavefront modulation, and the specific phase-shifting techniques are described.
Figure 2 illustrates the various configurations of spatial-division-method-based SIDH. Early SIDH systems utilized a triangular interferometer [22–24]. Such systems can be considered to use the spatial-division method, because they spatially separate the incident wavefront so that it may propagate through the different arms of the triangular interferometer. After the incident wavefront has been split into two wavefronts by a beam splitter at the front of the system, the two arms are rotated in opposite directions and the fronts pass through lenses or mirrors installed in both arms. Because of this rotational-shearing effect, an interference pattern may be recorded as a Fresnel-zone-plate pattern on the recording plane.
Rotational-shearing interferometers were implemented in an early version of SIDH [17, 25–28] as shown in Fig. 2(a). This modified Michelson interferometer alternates mirrors on each arm to a roof prism or dove prism and rotates one of the prisms against the axis of the other. Fourier holograms based on rotational-shearing interferometers were also demonstrated [28–30]. Conventional Michelson-interferometer-based SIDH enables the adjustment of the mirror curvature in the arm or the phase difference with a piezo-actuator [22, 31–35]. Kim [32] captured a full-color 3D incoherent hologram using simple Michelson-interferometer SIDH.
A SIDH system using a Mach-Zehnder interferometer that can independently modulate spatially separated wavefronts was proposed. An incoherent spatial-filtering concept using a Mach-Zehnder interferometer was presented as an example of a system that reduces bias information through two-pupil synthesis [37]. A previous study restored 3D-hologram information by removing the bias and twin-image information using a phase-shifting technique that moved the mirror in one arm of a Mach-Zehnder interferometer [38]. In addition, another study acquired holograms with a color image sensor by moving the mirror for phase shifting to match the wavelength for full-color acquisition [39].
The traditional limitations of on-axis holography, such as twin images or bias, are also imposed on on-axis SIDHs. The most straightforward method to overcome such problems is to use an off-axis configuration, to spatially separate image terms. Off-axis SIDH systems can be created using tilted mirrors with a setup similar to that of conventional SIDH [40]. Off-axis incoherent Fourier holography can be implemented using a tilted mirror and Fourier-transform lens [41–43]. Kwon’s group [44] proposed a common-path off-axis SIDH system that acquired a Fourier hologram using a diffraction grating and lens set.
FINCH is the most important SIDH system; It splits the incident wavefront in a polarization-selective manner [20, 45–50]. Figure 3 illustrates the various configurations of polarization-division-method-based SIDH systems. A liquid-crystal-based SLM is the main component, which can induce phase retardation using the isotropy of liquid-crystal cells with a high pixel density. The conventional uses of the PSLM include holographic displays and beam shaping; However, fixing the input polarization state restricts the system parameters. In contrast, in FINCH self-interference occurs when combining light modulated by a PSLM with orthogonally polarized light that is not modulated by a PSLM. Since Rosen and Brooker [20] introduced the concept of FINCH, various follow-up studies have been proposed. The basic configuration for FINCH includes a PSLM and relayed polarizer. The first polarizer aligns the incident light’s polarization such that half the light is modulated, while the remaining light is unmodulated. Thus, the modulated light and transmitted light interfere on the image sensor after the analyzer polarizer. Subsequently various FINCH systems, including dual-lens configurations with a relayed lens or two SLMs, have been proposed [51–54]. The fundamental concepts of polarization-based wavefront division and digitized phase masks in FINCH have influenced polarization-based SIDH systems and computational phase-mask-based SIDH.
Just as the existing FINCH system uses the retardation of a liquid crystal to modulate the incident wavefront in a polarization-selective manner and induces a shift in the relative phase, a birefringent material can be introduced to implement a SIDH system in a similar manner. A SIDH system called conoscopic holography, using birefringent crystals, was developed decades ago [55–57]. Similarly, the differential-interference contrast-imaging technique, which uses a uniaxial crystal to shear a wavefront through a microscopic sample to acquire the interference fringe pattern, has been proposed as an example telemedicine application [58]. This system has the advantage of simply comprising the combination of a birefringence crystal and polarizer, if required, while maintaining the structure of on-axis Gabor holography, where optical components are arranged in parallel. However, limitations in manipulating the acquired fringe pattern exist when using only a crystal slab. To overcome this limitation, SIDH systems with birefringent lenses have been proposed [59, 60]. Seigel et al. [59] proposed high-magnification super-resolution microscopy by configuring a FINCH system using a birefringent lens, and they achieved single-shot-based super-resolution holographic imaging using a polarization image sensor [60]. Tahara et al. [61, 62] implemented single-shot multicolor holographic imaging using a birefringent-lens-based SIDH system. A monochromatic image sensor was utilized, in which a wavelength-dependent polarization-sensitive phase-modulation array was attached to each pixel. Fluorescent holographic imaging has also been performed using this system.
Incoherent hologram recording is possible, even when the amplitude and phase pattern of the wavefront modulator do not follow uniform amplitude and quadratic phase respectively. Therefore, the abovementioned systems may correspond to special cases of the broad concept of incoherent holography. Representative phase patterns of the wavefront modulator include a discretized Fresnel-zone aperture (FZA) [63–67], coded-phase aperture [47, 68–70], and random-phase aperture [71, 72]. When an incoherent-holography system is implemented with such a device, 3D holographic spatial light information can be acquired with a higher degree of freedom than that of a system with a quadratic-phase-based wavefront modulator, which can be applied in a variety of ways, such as improving image resolution [73] or annular-aperture imaging [74]. However, a hologram obtained in this manner cannot be retrieved through convolution with a standardized kernel, as in the angular-spectrum method. Furthermore, even when possible, significant noise is observed. Therefore, mitigating the noise of the reconstructed image is necessary via various image-processing techniques, such as using compressive sensing [64, 67], building a point-spread-function (PSF) library [47, 68], or training a deep neural network [65].
Shimano et al. [63] fabricated an FZA-patterned glass plate and demonstrated quasi-coherent-field 3D full-color imaging. To mitigate undesired noisy terms while achieving real-time recording capability, spatially divided FZA patterns (where the fringes are shifted by 90˚) are recorded on a glass plate. Cao and Barbastathis’ group [64] proposed a binary-type FZA-based incoherent-imaging system. A compressive-sensing algorithm with total-variation regularization was adopted to mitigate twin images and noise with single-shot image acquisition. A reconstruction technique using a deep-learning network based on the U-Net structure has also been proposed, to achieve the same effect but at a faster speed and for broadband imaging [65]. Chen et al. [66] derived an optical transfer function for an FZA-based lensless-imaging system by considering the diffraction effect. This study realized a faster reconstruction speed than that of the previous method and improved the quality of the reconstructed image, especially in its spatial resolution. Chen et al. [67] proposed an algorithm for confocal imaging using the FZA-based incoherent-imaging system. In this study a compressive-sensing algorithm was applied to eliminate twin images and other noisy elements. Furthermore, an autofocusing and hologram-segmentation method was introduced to reduce the effect of defocused images.
A quasi-random phase-mask-based coded aperture has been proposed to obtain more degrees of freedom in holographic recording performance [47, 68–70]. This technique is known as coded-aperture incoherent holography (COACH). This technique was originally developed to improve axial resolution, which is considered a drawback of FINCH [47]. COACH has demonstrated the superior lateral resolution of FINCH plus the axial resolution of a conventional imaging system [73]. The coded-aperture mask is optimized using the Gerchberg-Saxton algorithm, starting from a random phase pattern and feeding to the PSLM. The first system uses the principle of self-interference, where one of the wavefronts passes through the SLM as a window and the other becomes a chaotic wavefront modulated by the displayed coded-aperture mask [47]. Soon after the first report of COACH, a version of this system without interference was proposed [68]. Because this system does not require the mutual interference of modulated and unmodulated waves, it is robust to environmental vibrations, free from the limitation of optical-path-length differences less than the coherence length of the light source, and power-efficient. Unlike quadratic-phase-pattern-based self-interference holograms, COACH requires a library of PSFs recorded for each sampled longitudinal direction. The reconstructed image is then obtained by convolving the PSF of a given depth position with the obtained hologram.
For typical Gabor holograms or inline holograms, eliminating on-axis bias and twin-image noise is necessary. The most popular methods are the off-axis configuration setup and phase-shift method involving multiple exposures. Acquiring a series of phase-modulated interferograms has enabled the computation of a complex hologram without bias or twin-image noise [75]. Phase modulation can be induced using relayed wave plates to modulate the wavefront phase. Kim et al. [24] obtained real-time incoherent complex holograms, using four different image sensors and phase shifts by combining wave plates and linear polarizers.
FINCH utilizes a PSLM with multiple phase-mask patterns with induced periodic phase constraints [20, 49, 50, 76]. The phase modulation generated from the liquid-crystal cells of the SLM can be easily controlled using only three or four phase-constraint steps. Alternatively, piezo actuators can be used for phase modulation [31, 32, 77]. The movement of piezo-mounted mirrors induces different phases depending on wavelength.
Polarization-based phase modulation calculates the relative phase retardation based on the angular difference between the relayed polarizers. This approach enables a single-shot holography system. The invention of polarized image sensors has enabled the use of a parallel phase-shift method [78]. By employing pixelwise directed micro-polarizers, multiple exposures with different polarization states can be captured. Even though the resolution of the averaging pixels decreases, parallel-phase-shift-method-based SIDHs using various optical elements that enable video hologram recording have been proposed [79–83]. One polarization-based technique is to spatially divide polarization-state responses. A diffraction-grating pattern was designed to multiplex the plane carrier waves and separate the polarization states into multiple diffraction terms. The main advantage of this method is that it does not require the integration of a micro-polarizer into the image-sensor plane. Therefore, efficient single-shot incoherent hologram acquisition can be achieved [84–86].
The phase of the incident wavefront can be modulated using a planar arrangement of liquid crystals or nanostructures with birefringent properties. The concept of such elements has been proposed for decades; However, with significant advances in liquid-crystal technology and nanofabrication processes, we are now witnessing the commercialization of these methods [87–90]. These elements are typically called Pancharatnam-Berry-phase or geometric-phase (GP) optical devices [91, 92]. Among them, elements that achieve similar effects using nanostructures are called metasurfaces [93–95]. These elements exhibit local waveplate characteristics, and if the active axis of the waveplate follows the phase angle determined by the quadratic phase equation, the entire surface can serve as a lens [96, 97]. Because these GP lenses (or metalenses) have inherent selective wavefront-modulation characteristics depending on the incident polarization state, they are used as wavefront-modulation devices in incoherent digital holography.
Figure 4 illustrates the schematic of GP-based SIDH systems. The very first concept of GP-based SIDH utilizes its polarization-sensitive properties as shown in Fig. 4(a) [62, 98]. When a half-wave plate is located between two linear polarizers, a phase shift may occur, depending on the relative angle between the two linear polarizers [99]. Applying this concept, Choi et al. [100, 101] proposed a system that implemented both a wavefront modulator and phase shifter through the combination of a rotating polarizer, GP lens, and fixed polarizer. Subsequently, the parallel phase-shifting method was realized through a combination of a fixed polarizer, GP lens, and polarization image sensor. For this configuration, single-shot-hologram acquisition and holographic-video acquisition systems have been proposed [82, 102, 103]. Moreover, an incoherent digital-holography system capable of high-speed phase shifting without mechanical movement was proposed, using liquid-crystal cells that switch the linear-polarization angle of the output wavefront at high speed, and a geometric-phase lens. Through this, real-time hologram acquisition was demonstrated at over 30 fps using a nonpolarized image sensor. Tahara and Oi [104] proposed a holosensor that enabled hologram sensing by attaching a GP lens close to a polarization image sensor. An incoherent-holography system using a GP lens was attempted with a near-infrared light source, beyond the visible-light range, demonstrating that hologram acquisition is possible from a wider range of light sources [105].
Zhou et al. [106] introduced isotropic bifocal metalenses into incoherent hologram-recording systems. The twin-image, zero-order, and noise terms were successfully suppressed using the compressive reconstruction algorithm. Kim and his research group [107] proposed a metalens-based SIDH system and applied a compressive-sensing algorithm to a hologram obtained for section-wise holographic object reconstruction.
Various analyses and attempts have been made to determine the optimal SIDH configuration. For example, a spherical-wave-equation-based PSF analysis was conducted in [108–110]. Because SIDH formulates the Fresnel hologram as a superposition of the PSF, the resolution of the SIDH system can be determined using the Rayleigh criterion. However, under certain conditions of FINCH, the resolution of the holograms is affected by the curvature of the wavefront, distance between the wavefront divider and image sensor, and distance from the object. Thus, under perfect beam-overlap conditions, the resolution of the acquired hologram is significantly improved compared to conventional imaging techniques. However, the field of view of the system is restricted, and tradeoffs are introduced because lateral magnification is determined solely by the distance between the wavefront divider and imager [52, 111].
This indicates that SIDH can violate the Lagrange invariant [76, 112, 113]. The Lagrange (or Smith-Helmholtz) invariant is a fundamental property that characterizes the formation of images in optical-imaging systems. The Lagrange invariant is denoted by the height of the paraxial chief ray at the object and image planes and the angles of the marginal ray at the entrance and exit pupils, which are rewritten as the lateral and axial magnifications respectively. To derive the Lagrange invariant of a SIDH system, we must consider the resolution and magnification in terms of the reconstructed holograms. Unlike classical imaging systems, the resulting image is obtained by refocusing the Fresnel hologram patterns. The reconstruction distance is determined by several factors, including the relationship between the wavefront-divider-to-imager distance, curvature of the modulated wavefront, and numerical aperture. Hence the particular configuration of SIDH, the perfect overlap condition of FINCH, has improved the lateral-resolution results at the expense of the axial resolution.
This section introduces the recent progress and applications of SIDH. Figure 5 shows the experimental results of various topics in the SIDH application. SIDH shares common problems and limitations with digital holography and causes self-interference. The resolution and field of view are the primary concerns. Addressing the digital noise that inevitably occurs with the use of digital devices is also important. Furthermore, the utilization of SIDH extends to optical sectioning, a classical application of digital holography, and it can also be employed in 3D-display metrology by utilizing light-field measurements that exploit the characteristics of SIDH.
The resolution of the Fresnel holograms obtained through SIDH correlates with the numerical aperture of the capture system. The most straightforward approach for increasing the numerical aperture is using a synthetic aperture. This established technique captures multiple images from various locations using an aperture-limited system and assembles them into a single large image, which is equivalent to the result from a large-aperture system. To incorporate a synthetic aperture into a SIDH system, mechanical movement with synchronized diffractive patterns on the SLM is necessary [21, 114, 115]. By using a calculated phase mask that considers the field of view and viewpoint, multiple holograms can be acquired and arranged in a tiled manner to form a single large hologram. Rosen’s group [115] introduced a sparse synthetic-aperture method and an extrapolation algorithm that could achieve the same effects with fewer exposures. Another method for realizing a synthetic aperture is the modulation of the optical transfer function. Instead of utilizing the complete Fresnel-zone-plate pattern to modulate the wavefront, an optical-transfer-function-modulation method was proposed to induce the synthetic aperture effect by manipulating the phase mask [116, 117].
In contrast to synthetic apertures, structured illumination is an alternative method for achieving high resolution. This method employs patterned illumination to enhance the spatial-frequency spectrum of an object wave. Kashter et al. [118] reported a super-resolution FINCH system that utilizes structured illumination. Four-directional structured lights are implemented by displaying sinusoidal profiles along the intended direction on a PSLM, resulting in the duplication of the spectrum of the object. The resultant hologram is the sum of each sub-hologram, which compensates for high-order diffraction and the linear-phase function. Jeon et al. [111] proposed structured-illumination-based high-resolution FINCH with a DMD.
Noise is a fundamental problem in digital holography. Conventional digital image sensors construct an image-processing pipeline for noise suppression or color compensation owing to Poisson or electrical noise. SIDH is also subject to this noise, which affects the quality of the reconstructed hologram because it adds directly to the fringe pattern, thereby reducing the signal-to-noise ratio and making it difficult to resolve the high-frequency signal from the noise.
Temporal multiplexing is a simple solution for averaging random noise; It provides the appropriate number of exposures that can reduce and improve image quality [119]. Spatial averaging can also be used to reduce noise [120]. Grouping adjacent pixels and treating them as a single pixel result in resolution loss, but by optimizing the wavefront modulation of SIDH, a configuration that optimizes the resolution while reducing noise can be achieved.
Defocus blurring is a problem that must be addressed in digital-holography applications. In the numerical reconstruction process using diffraction formulas, the amount of defocus blur is determined by the numerical aperture of the hologram, which is an obstacle when extending the data to three dimensions. Even if the hologram is acquired under the assumption of incoherent illumination, the interpretation of the hologram is performed in a coherent manner; therefore, the defocus blur appears in the form of a coherent cutoff. Choi et al. [121] solved this problem optically by using a pinhole polarizer.
Optical aberration is another problem in holographic reconstruction. Aberration in classical optics can be modeled using Zernike polynomials, and the compensation of aberrations has been introduced numerically. Kim et al. [22, 122] proposed adaptive-optics approaches for aberration compensation in SIDH. Because a complex hologram can be acquired by multiple exposures with a phase shift, the aberration terms can be considered as multiplexes on the wavefront curvature. To alleviate this aberration, the modeled phase error is subtracted from the raw hologram in the Fourier domain. Muroi et al. [123] proposed a camera-model-based lens-distortion-compensation method. The relayed lens in the SIDH configuration can reduce image quality by distorting the incident light waves. To compensate for image distortion, a crosshatch-shaped calibration image is first captured. Subsequently a computer-vision-based camera model and the distortion coefficient are calculated. This transformation can be used to compensate barrel distortions at different depths.
Optical sectioning (or compressive sensing) is a method for recovering discrete depth planes and has been successfully applied to digital holography. To acquire accurate 3D tomography with holographic reconstruction, suppressing the defocus blur and obtaining an accurate reconstruction depth are necessary. However, SIDH has different resolution criteria compared to classic imaging, particularly worse axial resolution; Therefore, a solution that applies conventional optical-sectioning algorithms, such as the two-step iterative shrinkage-thresholding (TwIST) algorithm, to SIDH has been proposed [107, 124–127]. Rosen’s group [126] proposed multiple-view projection-based compressive sensing. In particular, this study demonstrated that compressive sensing based on incoherent holograms is possible with less hologram acquisition than that of conventional multiple-view projection. In addition, several studies have proposed the application of the TwIST algorithm to a single SIDH base. Weng et al. [127] analyzed the axial resolution of a Michelson-interferometer-based SIDH system and proposed a compressive-sensing technique to eliminate bias and defocus noise. Man et al. [128] and Lee et al. [107] proposed compressive sensing under FINCH conditions through an analysis of depth reconstruction.
The acquisition of holograms from incoherent illumination sources allows the measurement of the light field formed in a 3D display. Owing to this advantage, various studies have proposed the application of SIDH in the development of 3D displays. SLMs used to implement holographic 3D displays typically modulate the phase or amplitude, and the accuracy of the modulated light can be determined by analyzing the complex hologram acquired through SIDH [129–131]. Furthermore, light-field displays (which control the direction of light rays with a lens or pinhole) formulate voxels to represent 3D images and can be evaluated by capturing the incoherent hologram of the formulated voxel. Kim et al. [132, 133] proposed a quantitative voxel-measuring method for light-field displays using autofocus algorithms.
Deep-learning-based incoherent digital holography was inspired by the emergence of deep neural networks and artificial intelligence. Data-driven approaches in digital holography can be utilized to generate, reconstruct, and improve holograms, overcoming the bottlenecks observed in conventional digital holography such as computational load, speckle noise, and physical limitations. In particular, for SIDH deep learning has the potential to extend the use of holographic acquisition. Wu et al. [134] introduced a large-depth-of-field fluorescence FINCH microscope that utilizes a generative adversarial network (GAN). The network was trained to minimize the discrepancies between the reconstructed holograms and conventional wide-field microscopy datasets. The GAN model comprises a generator and discriminator to determine whether the generated data are real or fake. The demonstration was conducted using mouse brain cells and resolution targets, and wide-field reconstruction was achieved through deep-learning reconstruction. The random noise that inevitably results from SIDH can be mitigated using deep learning. Moon et al. [135] proposed a hologram-denoising model that utilizes Gaussian noise analysis. Tahara and Shimobaba [136] utilized a denoising model to execute a high-speed phase-shifting SIDH. Huang et al. [137] suggested a deep-learning-based phase-shift method. Conventional phase-shift methods require three or more interferograms to reduce bias and twin images, whereas this study employed a deep-learning network to yield a phase-shifted interferogram from a single interferogram. Yu et al. [138] introduced a deep-learning filter network that refined the resolution of intricate holograms and established a holographic streaming system using the system. In this study, the network distinguishes between 2D full-color images displayed on a typical panel and a restructured hologram acquired through SIDH. The network operates in real time and is presented on a holographic display to implement a holographic streaming system.
The historical background and modern topics of SIDH were reviewed. Over the past several decades, the demand for 3D imaging using incoherent light sources has increased. SIDH can be a powerful solution to this problem, and suggestions to overcome the major limitations of self-interference have been proposed. SIDH is categorized based on its configuration and principle of wavefront splitting. Furthermore, the lateral resolution, axial resolution, and field of view of SIDH are determined by the curvature of the wavefront modulation and the spacing relationship of each component. Specifically, the overlap of each wave can exceed the lateral resolution under some conditions. Various techniques have been proposed to enhance the performance of SIDH, and methods in the existing digital-holography domain have been utilized to improve SIDH. In addition, recent advancements in artificial-intelligence techniques, including deep learning, have been applied to SIDH. Since SIDH has the advantage of acquiring 3D information under incoherent illumination conditions, it enables image processing in a novel domain. It is anticipated that diverse attempts will apply cutting-edge computer vision techniques and data-driven approaches to the field of SIDH.
Institute for Information and Communications Technology Promotion (2019-0-00001).
The authors declare that they have no competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Data sharing is not applicable to this article as no new data were created or analyzed in this study.
Curr. Opt. Photon. 2024; 8(1): 1-15
Published online February 25, 2024 https://doi.org/10.3807/COPP.2024.8.1.1
Copyright © Optical Society of Korea.
Youngrok Kim1, Ki-Hong Choi2, Chihyun In1, Keehoon Hong2, Sung-Wook Min1
1Department of Information Display, Kyung Hee University, Seoul 02447, Korea
2Digital Holography Research Section, Electronics and Telecommunications Research Institution, Daejeon 34129, Korea
Correspondence to:*mins@khu.ac.kr, ORCID 0000-0003-4794-356X
†These authors contributed equally to this paper.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
This paper presents a brief introduction to self-interference incoherent digital holography (SIDH). Holography conducted under incoherent light conditions has various advantages over digital holography performed with a conventional coherent light source. We categorize the methods for SIDH, which divides the incident light into two waves and modulates them differently. We also explore various optical concepts and techniques for the implementation and advancement of SIDH. This review presents the system design, performance analysis, and improvement of SIDH, as well as recent applications of SIDH, including optical sectioning and deep-learning-based SIDH.
Keywords: Digital holography, Incoherent holography, Self-interference
Since the basic principle of holography was invented by Gabor [1–3] as a three-dimensional (3D) information-recording and -reproduction technique, holography has developed rapidly. Interference is a phenomenon that depends on the wave nature of light, and requires the light waves to be coherent. The invention of the laser has enabled the construction of highly coherent and purified light sources, which has enabled the widespread study of holography in various fields, including holographic microscopy [4, 5], holographic displays [6–9], and computer-generated holograms [10, 11].
However, holography under incoherent-illumination conditions has also received attention. Laser illumination can significantly damage photosensitive materials and biological samples. Furthermore, applying fluorescence-imaging methods to in vivo, nondestructive measurements is difficult with conventional holographic techniques. More importantly, to apply holography in daily life we must overcome the limitations of light sources, and thus require holography based on general light source.
T. Young demonstrated the wavelike nature of light through interference experiments conducted in 1801 [12]. His famous interference experiment used sunlight passing through a pinhole, with the thin beam being split in half using a paper card. This improved the spatial coherence of the light, enabling interference even under incoherent illumination. In 1887, a Michelson-Morley experiment was conducted to detect the presence of the luminiferous ether [13]. Despite unsuccessful attempts to confirm the existence of the ether through changes in interference patterns, the related interferometer became the motivation for many later inventions.
Early incoherent holography was mostly based on the wavefront-dividing interferometers that Fresnel proposed [14–17]. Among his numerous accomplishments in wave optics, Fresnel proposed optical systems for capturing interference patterns before the concept of the laser was introduced. Michelson interferometers are powerful tools for incoherent holography. An orthogonal design comprising a beam splitter and mirrors can be exploited for a wavefront divider and the spatial modulation of light waves. Rotational-shearing interferometers form fringe patterns using a combination of tilted prisms, which induce optical-path differences based on the rotation angles.
The invention of the acousto-optic modulator was the primary turning point in the development of incoherent holography. Poon et al. [18] proposed optical scanning holography (OSH), which is based on optical heterodyning. This comprises a time-dependent Fresnel-zone plate and the scanning of a three-dimensional (3D) object in two dimensions. Various applications based on OSH have been proposed, including holographic microscopy and 3D displays [19].
Developments in liquid-crystal and semiconductor technologies have introduced a new era of displays, i.e. liquid-crystal displays. The polarization-selective properties of liquid crystals enable their use in a multitude of optical applications. Spatial light modulators (SLMs) enable wavefront modulation, which has resulted in significant advancements in holography. Fresnel incoherent correlation holography (FINCH) is a polarization-based incoherent digital holography technique that exploits phase-only SLMs (PSLMs) to divide and modulate wavefronts and to form self-interference fringe patterns [20]. Since its proposal by Rosen and Brooker [20] numerous methods have been suggested for self-interference incoherent holography.
This review is an introduction to self-interference incoherent digital holography (SIDH), from its basic principles to recent applications. According to the concept of self-interference, waves from the same point exhibit mutual coherence and interfere with each other. Thus, the main optical component of SIDH is a wavefront divider. In Section 2, we introduce the fundamentals of SIDH and various proposed optical configurations. Moreover, an analysis of the characteristics of SIDH is presented. Section 3 presents the system design, performance analysis, and improvement of SIDH. Section 4 focuses on recent applications of SIDH, and Section 5 offers concluding remarks.
The overlap of two mutually coherent wavefronts on the observation plane results in a distinctive pattern of dark and bright intensities. To ensure mutual coherence between two overlapping wavefronts, conventional interferometry requires a laser as the illumination source, and the beam-division module is configured immediately after the light source to guide the two paths for the object and reference beams. Conversely, self-interference uses a wavefront that has already entered the system. The input wavefront is divided into two parts to induce mutual interference. The parameters of the twin wavefronts originating from the same object point are subtly altered, using various methods. This ensures that the difference in the optical-path length between the two waves remains shorter than the coherence length of the light source used, thereby enabling their mutual interference. For example, if the wavefront division and modulation are adjusted to marginally alter the propagation vector of the input plane wave, the resulting interferogram resembles a grating pattern. Alternatively, if modulation is employed to produce spherical wavefronts with varying radii of curvature, the Fresnel-zone pattern (FZP) is obtained.
In practical self-interferometer systems, the division and modulation components can be either separate or integrated into a single component. The former approach draws largely from traditional interferometric systems, whereas the latter uses recently developed polarization-sensitive optical elements. Widely known interferometric structures, such as rotational shearing and Michelson interferometers, were adopted in the early stages of incoherent holograms as spatial-division methods.
Figure 1 illustrates the simplified optical structure of widely reported SIDH systems. The object to be recorded is regarded as a group of object points, for which the response functions of each point are independent of each other. To describe the wavefront-modulation procedure in the simplified SIDH system, we begin with a spherical wave diverging from an arbitrary single object point. The spherical wavefront of the object travels a distance zo to the front of the system, just in front of the field lens. When the initial amplitude and x-y coordinates of the object point source are C1(
After traveling a distance d from the field lens to the wavefront modulator, the field encounters the transmission function of the wavefront modulator. We assume that the wavefront modulator is a transmissive bifocal lens with focal lengths of f1 and f2. To employ the phase-shifting method to eliminate the bias and twin images, let us also assume that the bifocal lens can introduce a phase-shifting effect on one of the output wavefronts. Here the transmission function of the bifocal lens tbi is
where δ is the phase-shift value. After the bifocal lens, the field propagates further to the image sensor, which is located behind the bifocal lens at a distance zh. If the mutual coherence of the two wavefronts arriving at the image-sensor plane is maintained, interference occurs between the two spherical waves. Therefore, the intensity pattern of the interferogram, which contains the holographic information of the object point with the bias and twin-image information, is derived as follows:
In Eq. (3), the symbol ⊗ refers to the 2D convolution operation. Equation (3) can be simplified by noting that a spherical wave remains spherical after passing through an ideal lens [21]. When the original spherical wavefront is C(
where zrec is the reconstruction distance of the recorded hologram, M is the magnification factor of the system. C1,2,3,4 are coefficients for complex constant and C4′ means the conjugate number of C4. zrec and M are expressed as follows:
The parametrized Eqs. (5) and (6) are derived from the two-lens relaying system, where the first lens has a focal length of fo and the second is a bifocal lens with focal lengths of f1 and f2. The first imaging distance zi is calculated using the thin-lens equation, where the object distance is zo and the focal length of the lens is fo, which is
As described in Eq. (4), the desired hologram information exists with undesired bias and twin-image information. Several methods can be used to reduce or eliminate the bias and twin images, and the phase-shifting technique can in principle completely eliminate the bias and twin images. To apply the phase-shifting technique, we vary δ in Eq. (4) to obtain multiple images of Is. For the four-step phase-shifting technique with 90˚ intervals, the values of δ are [0, π/2, π, 3π/2]. The obtained Is images are as follows:
The complex hologram H without bias and twin-image noise is then obtained by combining Eq. (7) as follows:
As shown in Eq. (8), the obtained hologram H is the impulse-response function of the object point located at (xo, yo) at a distance of zo from the very front of the system. The obtained hologram can be reconstructed to that point again with a propagation distance of −zrec using a widely known numerical reconstruction algorithm, such as the angular-spectrum or Fresnel propagation method [2]. Because zrec depends on the axial location of the object zo, the axial location of each object point source is encoded by its own impulse-response function and can then be decoded again with the distinguished axial location.
Because the coherence of the light source used in this system is low compared to that of a laser-based interferometric system, there is hardly any mutual coherence of H from different point sources. Therefore, the intensity distribution at the sensor plane results from multiple point sources, and volumetric objects are incoherent summations of each impulse-response function. An arbitrary 3D object with a group of point sources is described as follows:
where Ao is the initial amplitude of each object points and δ is the phase-shifting angle.
In this section, we discuss research on various system designs, performance analyses, and improvements of SIDH. Through various studies, a number of wavefront-modulation techniques for splitting and modulating the incident wavefront, and a phase-shifting method for removing the bias and twin images, have been proposed. We introduce research that has increased the system magnification and resolution, or reduced the magnification but simplified the form factor. We also present research on full-color and real-time recording and introduce analyses of increasing the magnification or resolution by improving the system configuration.
The SIDH system configuration is divided into two main parts: A wavefront modulator and a phase shifter. Studies have proposed each of these parts as separate components or integrated them to simplify the system. Depending on the wavefront-division method, a conventional wavefront-separating interferometer can be used, or a common-path-type interferometer can be constructed. Phase shifters can be divided into three main categories: The physical moving of optical elements, introducing retardation between interfering wavefronts using birefringent media, and introducing a geometric phase using a combination of polarization elements and their rotation states. The description of the design is divided into three categories based on the method of wavefront modulation, and the specific phase-shifting techniques are described.
Figure 2 illustrates the various configurations of spatial-division-method-based SIDH. Early SIDH systems utilized a triangular interferometer [22–24]. Such systems can be considered to use the spatial-division method, because they spatially separate the incident wavefront so that it may propagate through the different arms of the triangular interferometer. After the incident wavefront has been split into two wavefronts by a beam splitter at the front of the system, the two arms are rotated in opposite directions and the fronts pass through lenses or mirrors installed in both arms. Because of this rotational-shearing effect, an interference pattern may be recorded as a Fresnel-zone-plate pattern on the recording plane.
Rotational-shearing interferometers were implemented in an early version of SIDH [17, 25–28] as shown in Fig. 2(a). This modified Michelson interferometer alternates mirrors on each arm to a roof prism or dove prism and rotates one of the prisms against the axis of the other. Fourier holograms based on rotational-shearing interferometers were also demonstrated [28–30]. Conventional Michelson-interferometer-based SIDH enables the adjustment of the mirror curvature in the arm or the phase difference with a piezo-actuator [22, 31–35]. Kim [32] captured a full-color 3D incoherent hologram using simple Michelson-interferometer SIDH.
A SIDH system using a Mach-Zehnder interferometer that can independently modulate spatially separated wavefronts was proposed. An incoherent spatial-filtering concept using a Mach-Zehnder interferometer was presented as an example of a system that reduces bias information through two-pupil synthesis [37]. A previous study restored 3D-hologram information by removing the bias and twin-image information using a phase-shifting technique that moved the mirror in one arm of a Mach-Zehnder interferometer [38]. In addition, another study acquired holograms with a color image sensor by moving the mirror for phase shifting to match the wavelength for full-color acquisition [39].
The traditional limitations of on-axis holography, such as twin images or bias, are also imposed on on-axis SIDHs. The most straightforward method to overcome such problems is to use an off-axis configuration, to spatially separate image terms. Off-axis SIDH systems can be created using tilted mirrors with a setup similar to that of conventional SIDH [40]. Off-axis incoherent Fourier holography can be implemented using a tilted mirror and Fourier-transform lens [41–43]. Kwon’s group [44] proposed a common-path off-axis SIDH system that acquired a Fourier hologram using a diffraction grating and lens set.
FINCH is the most important SIDH system; It splits the incident wavefront in a polarization-selective manner [20, 45–50]. Figure 3 illustrates the various configurations of polarization-division-method-based SIDH systems. A liquid-crystal-based SLM is the main component, which can induce phase retardation using the isotropy of liquid-crystal cells with a high pixel density. The conventional uses of the PSLM include holographic displays and beam shaping; However, fixing the input polarization state restricts the system parameters. In contrast, in FINCH self-interference occurs when combining light modulated by a PSLM with orthogonally polarized light that is not modulated by a PSLM. Since Rosen and Brooker [20] introduced the concept of FINCH, various follow-up studies have been proposed. The basic configuration for FINCH includes a PSLM and relayed polarizer. The first polarizer aligns the incident light’s polarization such that half the light is modulated, while the remaining light is unmodulated. Thus, the modulated light and transmitted light interfere on the image sensor after the analyzer polarizer. Subsequently various FINCH systems, including dual-lens configurations with a relayed lens or two SLMs, have been proposed [51–54]. The fundamental concepts of polarization-based wavefront division and digitized phase masks in FINCH have influenced polarization-based SIDH systems and computational phase-mask-based SIDH.
Just as the existing FINCH system uses the retardation of a liquid crystal to modulate the incident wavefront in a polarization-selective manner and induces a shift in the relative phase, a birefringent material can be introduced to implement a SIDH system in a similar manner. A SIDH system called conoscopic holography, using birefringent crystals, was developed decades ago [55–57]. Similarly, the differential-interference contrast-imaging technique, which uses a uniaxial crystal to shear a wavefront through a microscopic sample to acquire the interference fringe pattern, has been proposed as an example telemedicine application [58]. This system has the advantage of simply comprising the combination of a birefringence crystal and polarizer, if required, while maintaining the structure of on-axis Gabor holography, where optical components are arranged in parallel. However, limitations in manipulating the acquired fringe pattern exist when using only a crystal slab. To overcome this limitation, SIDH systems with birefringent lenses have been proposed [59, 60]. Seigel et al. [59] proposed high-magnification super-resolution microscopy by configuring a FINCH system using a birefringent lens, and they achieved single-shot-based super-resolution holographic imaging using a polarization image sensor [60]. Tahara et al. [61, 62] implemented single-shot multicolor holographic imaging using a birefringent-lens-based SIDH system. A monochromatic image sensor was utilized, in which a wavelength-dependent polarization-sensitive phase-modulation array was attached to each pixel. Fluorescent holographic imaging has also been performed using this system.
Incoherent hologram recording is possible, even when the amplitude and phase pattern of the wavefront modulator do not follow uniform amplitude and quadratic phase respectively. Therefore, the abovementioned systems may correspond to special cases of the broad concept of incoherent holography. Representative phase patterns of the wavefront modulator include a discretized Fresnel-zone aperture (FZA) [63–67], coded-phase aperture [47, 68–70], and random-phase aperture [71, 72]. When an incoherent-holography system is implemented with such a device, 3D holographic spatial light information can be acquired with a higher degree of freedom than that of a system with a quadratic-phase-based wavefront modulator, which can be applied in a variety of ways, such as improving image resolution [73] or annular-aperture imaging [74]. However, a hologram obtained in this manner cannot be retrieved through convolution with a standardized kernel, as in the angular-spectrum method. Furthermore, even when possible, significant noise is observed. Therefore, mitigating the noise of the reconstructed image is necessary via various image-processing techniques, such as using compressive sensing [64, 67], building a point-spread-function (PSF) library [47, 68], or training a deep neural network [65].
Shimano et al. [63] fabricated an FZA-patterned glass plate and demonstrated quasi-coherent-field 3D full-color imaging. To mitigate undesired noisy terms while achieving real-time recording capability, spatially divided FZA patterns (where the fringes are shifted by 90˚) are recorded on a glass plate. Cao and Barbastathis’ group [64] proposed a binary-type FZA-based incoherent-imaging system. A compressive-sensing algorithm with total-variation regularization was adopted to mitigate twin images and noise with single-shot image acquisition. A reconstruction technique using a deep-learning network based on the U-Net structure has also been proposed, to achieve the same effect but at a faster speed and for broadband imaging [65]. Chen et al. [66] derived an optical transfer function for an FZA-based lensless-imaging system by considering the diffraction effect. This study realized a faster reconstruction speed than that of the previous method and improved the quality of the reconstructed image, especially in its spatial resolution. Chen et al. [67] proposed an algorithm for confocal imaging using the FZA-based incoherent-imaging system. In this study a compressive-sensing algorithm was applied to eliminate twin images and other noisy elements. Furthermore, an autofocusing and hologram-segmentation method was introduced to reduce the effect of defocused images.
A quasi-random phase-mask-based coded aperture has been proposed to obtain more degrees of freedom in holographic recording performance [47, 68–70]. This technique is known as coded-aperture incoherent holography (COACH). This technique was originally developed to improve axial resolution, which is considered a drawback of FINCH [47]. COACH has demonstrated the superior lateral resolution of FINCH plus the axial resolution of a conventional imaging system [73]. The coded-aperture mask is optimized using the Gerchberg-Saxton algorithm, starting from a random phase pattern and feeding to the PSLM. The first system uses the principle of self-interference, where one of the wavefronts passes through the SLM as a window and the other becomes a chaotic wavefront modulated by the displayed coded-aperture mask [47]. Soon after the first report of COACH, a version of this system without interference was proposed [68]. Because this system does not require the mutual interference of modulated and unmodulated waves, it is robust to environmental vibrations, free from the limitation of optical-path-length differences less than the coherence length of the light source, and power-efficient. Unlike quadratic-phase-pattern-based self-interference holograms, COACH requires a library of PSFs recorded for each sampled longitudinal direction. The reconstructed image is then obtained by convolving the PSF of a given depth position with the obtained hologram.
For typical Gabor holograms or inline holograms, eliminating on-axis bias and twin-image noise is necessary. The most popular methods are the off-axis configuration setup and phase-shift method involving multiple exposures. Acquiring a series of phase-modulated interferograms has enabled the computation of a complex hologram without bias or twin-image noise [75]. Phase modulation can be induced using relayed wave plates to modulate the wavefront phase. Kim et al. [24] obtained real-time incoherent complex holograms, using four different image sensors and phase shifts by combining wave plates and linear polarizers.
FINCH utilizes a PSLM with multiple phase-mask patterns with induced periodic phase constraints [20, 49, 50, 76]. The phase modulation generated from the liquid-crystal cells of the SLM can be easily controlled using only three or four phase-constraint steps. Alternatively, piezo actuators can be used for phase modulation [31, 32, 77]. The movement of piezo-mounted mirrors induces different phases depending on wavelength.
Polarization-based phase modulation calculates the relative phase retardation based on the angular difference between the relayed polarizers. This approach enables a single-shot holography system. The invention of polarized image sensors has enabled the use of a parallel phase-shift method [78]. By employing pixelwise directed micro-polarizers, multiple exposures with different polarization states can be captured. Even though the resolution of the averaging pixels decreases, parallel-phase-shift-method-based SIDHs using various optical elements that enable video hologram recording have been proposed [79–83]. One polarization-based technique is to spatially divide polarization-state responses. A diffraction-grating pattern was designed to multiplex the plane carrier waves and separate the polarization states into multiple diffraction terms. The main advantage of this method is that it does not require the integration of a micro-polarizer into the image-sensor plane. Therefore, efficient single-shot incoherent hologram acquisition can be achieved [84–86].
The phase of the incident wavefront can be modulated using a planar arrangement of liquid crystals or nanostructures with birefringent properties. The concept of such elements has been proposed for decades; However, with significant advances in liquid-crystal technology and nanofabrication processes, we are now witnessing the commercialization of these methods [87–90]. These elements are typically called Pancharatnam-Berry-phase or geometric-phase (GP) optical devices [91, 92]. Among them, elements that achieve similar effects using nanostructures are called metasurfaces [93–95]. These elements exhibit local waveplate characteristics, and if the active axis of the waveplate follows the phase angle determined by the quadratic phase equation, the entire surface can serve as a lens [96, 97]. Because these GP lenses (or metalenses) have inherent selective wavefront-modulation characteristics depending on the incident polarization state, they are used as wavefront-modulation devices in incoherent digital holography.
Figure 4 illustrates the schematic of GP-based SIDH systems. The very first concept of GP-based SIDH utilizes its polarization-sensitive properties as shown in Fig. 4(a) [62, 98]. When a half-wave plate is located between two linear polarizers, a phase shift may occur, depending on the relative angle between the two linear polarizers [99]. Applying this concept, Choi et al. [100, 101] proposed a system that implemented both a wavefront modulator and phase shifter through the combination of a rotating polarizer, GP lens, and fixed polarizer. Subsequently, the parallel phase-shifting method was realized through a combination of a fixed polarizer, GP lens, and polarization image sensor. For this configuration, single-shot-hologram acquisition and holographic-video acquisition systems have been proposed [82, 102, 103]. Moreover, an incoherent digital-holography system capable of high-speed phase shifting without mechanical movement was proposed, using liquid-crystal cells that switch the linear-polarization angle of the output wavefront at high speed, and a geometric-phase lens. Through this, real-time hologram acquisition was demonstrated at over 30 fps using a nonpolarized image sensor. Tahara and Oi [104] proposed a holosensor that enabled hologram sensing by attaching a GP lens close to a polarization image sensor. An incoherent-holography system using a GP lens was attempted with a near-infrared light source, beyond the visible-light range, demonstrating that hologram acquisition is possible from a wider range of light sources [105].
Zhou et al. [106] introduced isotropic bifocal metalenses into incoherent hologram-recording systems. The twin-image, zero-order, and noise terms were successfully suppressed using the compressive reconstruction algorithm. Kim and his research group [107] proposed a metalens-based SIDH system and applied a compressive-sensing algorithm to a hologram obtained for section-wise holographic object reconstruction.
Various analyses and attempts have been made to determine the optimal SIDH configuration. For example, a spherical-wave-equation-based PSF analysis was conducted in [108–110]. Because SIDH formulates the Fresnel hologram as a superposition of the PSF, the resolution of the SIDH system can be determined using the Rayleigh criterion. However, under certain conditions of FINCH, the resolution of the holograms is affected by the curvature of the wavefront, distance between the wavefront divider and image sensor, and distance from the object. Thus, under perfect beam-overlap conditions, the resolution of the acquired hologram is significantly improved compared to conventional imaging techniques. However, the field of view of the system is restricted, and tradeoffs are introduced because lateral magnification is determined solely by the distance between the wavefront divider and imager [52, 111].
This indicates that SIDH can violate the Lagrange invariant [76, 112, 113]. The Lagrange (or Smith-Helmholtz) invariant is a fundamental property that characterizes the formation of images in optical-imaging systems. The Lagrange invariant is denoted by the height of the paraxial chief ray at the object and image planes and the angles of the marginal ray at the entrance and exit pupils, which are rewritten as the lateral and axial magnifications respectively. To derive the Lagrange invariant of a SIDH system, we must consider the resolution and magnification in terms of the reconstructed holograms. Unlike classical imaging systems, the resulting image is obtained by refocusing the Fresnel hologram patterns. The reconstruction distance is determined by several factors, including the relationship between the wavefront-divider-to-imager distance, curvature of the modulated wavefront, and numerical aperture. Hence the particular configuration of SIDH, the perfect overlap condition of FINCH, has improved the lateral-resolution results at the expense of the axial resolution.
This section introduces the recent progress and applications of SIDH. Figure 5 shows the experimental results of various topics in the SIDH application. SIDH shares common problems and limitations with digital holography and causes self-interference. The resolution and field of view are the primary concerns. Addressing the digital noise that inevitably occurs with the use of digital devices is also important. Furthermore, the utilization of SIDH extends to optical sectioning, a classical application of digital holography, and it can also be employed in 3D-display metrology by utilizing light-field measurements that exploit the characteristics of SIDH.
The resolution of the Fresnel holograms obtained through SIDH correlates with the numerical aperture of the capture system. The most straightforward approach for increasing the numerical aperture is using a synthetic aperture. This established technique captures multiple images from various locations using an aperture-limited system and assembles them into a single large image, which is equivalent to the result from a large-aperture system. To incorporate a synthetic aperture into a SIDH system, mechanical movement with synchronized diffractive patterns on the SLM is necessary [21, 114, 115]. By using a calculated phase mask that considers the field of view and viewpoint, multiple holograms can be acquired and arranged in a tiled manner to form a single large hologram. Rosen’s group [115] introduced a sparse synthetic-aperture method and an extrapolation algorithm that could achieve the same effects with fewer exposures. Another method for realizing a synthetic aperture is the modulation of the optical transfer function. Instead of utilizing the complete Fresnel-zone-plate pattern to modulate the wavefront, an optical-transfer-function-modulation method was proposed to induce the synthetic aperture effect by manipulating the phase mask [116, 117].
In contrast to synthetic apertures, structured illumination is an alternative method for achieving high resolution. This method employs patterned illumination to enhance the spatial-frequency spectrum of an object wave. Kashter et al. [118] reported a super-resolution FINCH system that utilizes structured illumination. Four-directional structured lights are implemented by displaying sinusoidal profiles along the intended direction on a PSLM, resulting in the duplication of the spectrum of the object. The resultant hologram is the sum of each sub-hologram, which compensates for high-order diffraction and the linear-phase function. Jeon et al. [111] proposed structured-illumination-based high-resolution FINCH with a DMD.
Noise is a fundamental problem in digital holography. Conventional digital image sensors construct an image-processing pipeline for noise suppression or color compensation owing to Poisson or electrical noise. SIDH is also subject to this noise, which affects the quality of the reconstructed hologram because it adds directly to the fringe pattern, thereby reducing the signal-to-noise ratio and making it difficult to resolve the high-frequency signal from the noise.
Temporal multiplexing is a simple solution for averaging random noise; It provides the appropriate number of exposures that can reduce and improve image quality [119]. Spatial averaging can also be used to reduce noise [120]. Grouping adjacent pixels and treating them as a single pixel result in resolution loss, but by optimizing the wavefront modulation of SIDH, a configuration that optimizes the resolution while reducing noise can be achieved.
Defocus blurring is a problem that must be addressed in digital-holography applications. In the numerical reconstruction process using diffraction formulas, the amount of defocus blur is determined by the numerical aperture of the hologram, which is an obstacle when extending the data to three dimensions. Even if the hologram is acquired under the assumption of incoherent illumination, the interpretation of the hologram is performed in a coherent manner; therefore, the defocus blur appears in the form of a coherent cutoff. Choi et al. [121] solved this problem optically by using a pinhole polarizer.
Optical aberration is another problem in holographic reconstruction. Aberration in classical optics can be modeled using Zernike polynomials, and the compensation of aberrations has been introduced numerically. Kim et al. [22, 122] proposed adaptive-optics approaches for aberration compensation in SIDH. Because a complex hologram can be acquired by multiple exposures with a phase shift, the aberration terms can be considered as multiplexes on the wavefront curvature. To alleviate this aberration, the modeled phase error is subtracted from the raw hologram in the Fourier domain. Muroi et al. [123] proposed a camera-model-based lens-distortion-compensation method. The relayed lens in the SIDH configuration can reduce image quality by distorting the incident light waves. To compensate for image distortion, a crosshatch-shaped calibration image is first captured. Subsequently a computer-vision-based camera model and the distortion coefficient are calculated. This transformation can be used to compensate barrel distortions at different depths.
Optical sectioning (or compressive sensing) is a method for recovering discrete depth planes and has been successfully applied to digital holography. To acquire accurate 3D tomography with holographic reconstruction, suppressing the defocus blur and obtaining an accurate reconstruction depth are necessary. However, SIDH has different resolution criteria compared to classic imaging, particularly worse axial resolution; Therefore, a solution that applies conventional optical-sectioning algorithms, such as the two-step iterative shrinkage-thresholding (TwIST) algorithm, to SIDH has been proposed [107, 124–127]. Rosen’s group [126] proposed multiple-view projection-based compressive sensing. In particular, this study demonstrated that compressive sensing based on incoherent holograms is possible with less hologram acquisition than that of conventional multiple-view projection. In addition, several studies have proposed the application of the TwIST algorithm to a single SIDH base. Weng et al. [127] analyzed the axial resolution of a Michelson-interferometer-based SIDH system and proposed a compressive-sensing technique to eliminate bias and defocus noise. Man et al. [128] and Lee et al. [107] proposed compressive sensing under FINCH conditions through an analysis of depth reconstruction.
The acquisition of holograms from incoherent illumination sources allows the measurement of the light field formed in a 3D display. Owing to this advantage, various studies have proposed the application of SIDH in the development of 3D displays. SLMs used to implement holographic 3D displays typically modulate the phase or amplitude, and the accuracy of the modulated light can be determined by analyzing the complex hologram acquired through SIDH [129–131]. Furthermore, light-field displays (which control the direction of light rays with a lens or pinhole) formulate voxels to represent 3D images and can be evaluated by capturing the incoherent hologram of the formulated voxel. Kim et al. [132, 133] proposed a quantitative voxel-measuring method for light-field displays using autofocus algorithms.
Deep-learning-based incoherent digital holography was inspired by the emergence of deep neural networks and artificial intelligence. Data-driven approaches in digital holography can be utilized to generate, reconstruct, and improve holograms, overcoming the bottlenecks observed in conventional digital holography such as computational load, speckle noise, and physical limitations. In particular, for SIDH deep learning has the potential to extend the use of holographic acquisition. Wu et al. [134] introduced a large-depth-of-field fluorescence FINCH microscope that utilizes a generative adversarial network (GAN). The network was trained to minimize the discrepancies between the reconstructed holograms and conventional wide-field microscopy datasets. The GAN model comprises a generator and discriminator to determine whether the generated data are real or fake. The demonstration was conducted using mouse brain cells and resolution targets, and wide-field reconstruction was achieved through deep-learning reconstruction. The random noise that inevitably results from SIDH can be mitigated using deep learning. Moon et al. [135] proposed a hologram-denoising model that utilizes Gaussian noise analysis. Tahara and Shimobaba [136] utilized a denoising model to execute a high-speed phase-shifting SIDH. Huang et al. [137] suggested a deep-learning-based phase-shift method. Conventional phase-shift methods require three or more interferograms to reduce bias and twin images, whereas this study employed a deep-learning network to yield a phase-shifted interferogram from a single interferogram. Yu et al. [138] introduced a deep-learning filter network that refined the resolution of intricate holograms and established a holographic streaming system using the system. In this study, the network distinguishes between 2D full-color images displayed on a typical panel and a restructured hologram acquired through SIDH. The network operates in real time and is presented on a holographic display to implement a holographic streaming system.
The historical background and modern topics of SIDH were reviewed. Over the past several decades, the demand for 3D imaging using incoherent light sources has increased. SIDH can be a powerful solution to this problem, and suggestions to overcome the major limitations of self-interference have been proposed. SIDH is categorized based on its configuration and principle of wavefront splitting. Furthermore, the lateral resolution, axial resolution, and field of view of SIDH are determined by the curvature of the wavefront modulation and the spacing relationship of each component. Specifically, the overlap of each wave can exceed the lateral resolution under some conditions. Various techniques have been proposed to enhance the performance of SIDH, and methods in the existing digital-holography domain have been utilized to improve SIDH. In addition, recent advancements in artificial-intelligence techniques, including deep learning, have been applied to SIDH. Since SIDH has the advantage of acquiring 3D information under incoherent illumination conditions, it enables image processing in a novel domain. It is anticipated that diverse attempts will apply cutting-edge computer vision techniques and data-driven approaches to the field of SIDH.
Institute for Information and Communications Technology Promotion (2019-0-00001).
The authors declare that they have no competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Data sharing is not applicable to this article as no new data were created or analyzed in this study.