Ex) Article Title, Author, Keywords
Current Optics
and Photonics
Ex) Article Title, Author, Keywords
Curr. Opt. Photon. 2022; 6(1): 51-59
Published online February 25, 2022 https://doi.org/10.3807/COPP.2022.6.1.051
Copyright © Optical Society of Korea.
Corresponding author: ^{*}seungyeol@knu.ac.kr, ORCID 0000-0002-8987-9749
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Holographic display technology is one of the promising 3D display technologies. However, the large amount of computation time required to generate computer-generated holograms (CGH) is a major obstacle to the commercialization of digital hologram. In various systems such as multi-depth head-updisplays with hologram contents, it is important to transmit hologram data in real time. In this paper, we propose a rapid CGH computation method by applying an arraying of a down-scaled hologram with the multiplication of a shifted concave lens function array. Compared to conventional angular spectrum method (ASM) calculation, we achieved about 39 times faster calculation speed for 3840 × 2160 pixel CGH calculation. Through the numerical investigation and experiments, we verified the degradation of reconstructed hologram image quality made by the proposed method is not so much compared to conventional ASM.
Keywords: Computer-generated holograms generation algorithm, Digital holography, Image processing
OCIS codes: (090.1995) Digital holography; (090.2820) Heads-up displays; (100.2000) Digital image processing
Digital holography has been applied in various fields such as 3D displays, cultural asset exhibitions and performances, lithography, and microscope technologies [1_{–}7]. Since digital holograms can reconstruct actual 3D objects from planar display panels without causing dizziness to the observer, it is often referred to as the ultimate technology for creating 3D scenes. In recent years, various kinds of visual systems that directly apply holographic technology such as holographic head-up-displays (HUD) for vehicles have been studied and demonstrated [8_{–}12]. In such applications, one of the most important features is real-time computation of computer-generated holograms (CGH) to provide the current status of a vehicle and environmental information to passengers in real time. However, real-time calculation of CGH patterns is still a major bottleneck to fully commercializing holographic HUD systems due to the large amount of computation time and memory usage required to calculate CGH.
To reduce the calculation time of CGH, various studies have been conducted [13_{–}15]. For example, using polygon-based computational methods instead of point cloud [16], algorithms for the efficient numerical propagation of wavefields [17] were improved using wavelet transform-based calculation [18], and further studies have been conducted for fast calculation of CGH in many different routes [19_{,} 20]. Moreover, the use of multiple GPUs to calculate holograms in real time [21], fast CGH calculation from 3D objects consisting of multiple layers of line-drawn objects [22], or fast CGH algorithms based on pinhole-type look-up tables have been proposed [23].
However, applying the above-mentioned methods still seems to be not good enough for the practical use of CGH calculation for real-time application since the major bottleneck of plane-by-plane type CGH calculation time, caused by conducting 2D fast Fourier transform (FFT), is too large. For example, with MATLAB 2021a, computation time of 2D FFT of random phase data still consumes more than 0.3 s with a high-end specification GPU (NVIDIA GeForce RTX3090; NVIDIA, CA, USA) for 4096 × 4096 resolution, which required full data of CGH for a commercialized ultra-high definition (UHD) resolution spatial light modulator (SLM). Since the 2D FFT calculation in CGH generation is a necessary part, a method for calculating high-resolution CGH from low-resolution 2D FFT calculation may be needed. Conventional upscaling algorithms such as bicubic interpolation do not work well in CGH patterns, so a unique approach for upscaling CGH patterns may be needed.
In this work, we proposed to dramatically reduce the computational time of CGH using a concave lens array multiplied to the array of down-scaled resolution calculated CGH. Using this approach, we propose a method to generate a 4096 × 4096 resolution CGH in real time in a relatively low-performance computational GPU environment. Since the shape of the concave lens functions was fixed by the distance between CGH and object plane, computing CGHs of different objects with the same distance is much faster by pre-computing concave lens functions. We also compare the quality of reconstructed holograms from the CGHs computed by the conventional angular spectrum method (ASM) and proposed work through simulations and experiments.
Figure 1 shows a schematic of the proposed CGH resolution upscaling method using a concave function array compared to conventional ASM. For simplicity, we will assume that the pixel number (
Considering the reconstruction process of CGH, the reduced image is our “input” and the magnified original image is our “output” for the imaging equation. Therefore, the minus sign in
After preparing the resized image, the second step is a calculation of simple CGH with a reduced resolution condition. Since FFT calculations are conducted along
The final step of the proposed method is the multiplication of the concave lens array function into the array of reduced resolution CGH. The concave lens array function needs to be appropriately shifted to provide the reconstructed object image placed with the same size and position. In other words, after being magnified by the corresponding concave lens array function, each reduced-resolution CGH will generate an identical reconstructed image observed by a different viewing angle. To provide such a condition, the center location of each concave lens function
Here,
Therefore, each of the concave lens functions is relatively shifted with respect to the center of CGH blocks with the amount of
in order to provide the identical image location of the reconstructed hologram from each CGH block.
Since the proposed method is focused on a dramatic reduction of the calculation time of the CGH that is upscaled from a low-resolution CGH, there must be a trade-off relationship with fast calculation time. The loss of information compared to direct calculation of the full-resolution CGH might be observed as a degradation of image quality. Figure 2 shows the simulation results using the same object under conditions of wavelength at 627 nm and focal length (
Figure 2(a) shows CGH calculated by conventional ASM as a control group (3840 × 3840 resolution, 3.6 μm pixel pitch) and Fig. 2(b) shows CGH calculated by the proposed method of M = 5. Figure 2(d) shows a reconstructed hologram image of Fig. 2(a) and Fig. 2(e) shows a reconstructed hologram image of Fig. 2(b). When comparing Fig. 2(d) and Fig. 2(e) by enlarging the character parts, although speckle noise is increased by applying the proposed method, the reconstructed object is still observable. It is noteworthy that the CGH calculation time in Fig. 2(d) required 3879 ms in our system, whereas calculation of Fig. 2(e) only required 96 ms. On the other hand, Fig. 2(c) shows a single portion of CGH calculated by reduced image without
Since a discretized lens function needs to be multiplied to the array of reduced-size CGH pattern, we need to investigate the possible range of image reconstruction without unwanted diffracted images occurring. When the proposed method is applied, the unwanted diffracted images occur when the focal length is too close, whereas speckle noises become severe when the focal length is too far.
Table 1 shows a comparison of the reconstructed hologram image’s degradation effect with respect to focal length and magnification factor, and Table 2 shows a reconstructed hologram image of Table 1. In Table 2, The images in the first row shows the reference, which conducts the full ASM simulation of the 4k × 4k resolution CGH, and the images in the second to fourth rows are reconstructed images when
TABLE 1 Comparison of computer-generated holograms (CGH) noise and blurring according to focal length and magnification factor
Magnification Factor [ | Focal Length ( | ||||
---|---|---|---|---|---|
5 cm | 10 cm | 17 cm | 30 cm | 50 cm | |
Original | |||||
2× | |||||
4× | |||||
8× |
TABLE 2 Reconstructed hologram image of Table 1
Magnification Factor [ | Focal Length ( | ||||
---|---|---|---|---|---|
5 cm | 10 cm | 17 cm | 30 cm | 50 cm | |
Original | |||||
2× | |||||
4× | |||||
8× |
Except for the case of the first row (conventional ASM), the diffracted image can be seen at a focal length of 5 cm (first column). Although such diffracted images disappear in the 10 cm distance condition, weakly blurred images remain as background noise can be seen (second column). However, from about 17 cm, such noises disappear so that the image quality of the proposed method is reasonably good compared to conventional ASM. When the focal length of the image plane is far away, speckle noise increases much faster when
As shown in Table 2, diffracted images up to 17 cm and increased speckle noises are major problems caused by CGH generation using lens functions. The main cause of noise is the empty space of CGH that occurs when calculating CGH using reduced images according to given parameters (resolution, pixel pitch, wavelength, size of object and focal length), which can also be confirmed in Table 1. Because other parameters are difficult to modify in a fixed environment, it is recommended to design an appropriate focal length to avoid these problems. Since it only shows a qualitative analysis in Table 2, we also extracted the quantitative figure of merit parameter from each of the reconstructed images. We extract the structural similarity index measurement (SSIM) value that can be used for comparing the degree of distortion of structural information in images. This method can precisely determine how much the image of the experimental group differs from its reference group, which can be expressed as [24]
In Eq. (5), μ_{x}, μ_{y} indicate the
Based on these calculations, Fig. 3(a) shows the comparison results of SSIM graph according to the focal length and magnification factor
Table 3 shows the exact time consumption of the proposed method according to the variation of the value
TABLE 3 Comparison of computational time of 3840 × 3840 computergenerated holograms (CGH) according to magnification factor (computation time is obtained as the average of 100-time measurements)
Magnification Factor [ | 1 | 2 | 4 | 5 | 6 | 8 | 10 |
---|---|---|---|---|---|---|---|
Image Resizing (ms) | N/A | 43 | 34 | 30 | 35 | 33 | 32 |
Calculate ASM (ms) | 3879 | 520 | 140 | 96 | 68 | 43 | 38 |
Lens Function Preparation (ms) | N/A | 944 | 826 | 788 | 761 | 701 | 663 |
Arraying CGH and Multiply Lens Function (ms) | N/A | 45 | 32 | 29 | 31 | 31 | 29 |
Total Computation Time (ms) | 3879 | 1552 | 1032 | 943 | 895 | 808 | 762 |
Actual Calculation Time for Single Frame (ms) | 3879 | 608 | 206 | 155 | 134 | 107 | 99 |
To compare the image quality between each magnification factor condition, which is shown numerically in Table 2, we demonstrate a hologram generation experiment as shown in Fig. 4. Here, we used the Peony-62a SLM kit (May Display, Suwon, Korea), which has a pixel resolution of 3840 × 2160 and pixel pitch of 3.6 μm. Therefore, we could directly use the CGH pattern calculated in the previous section after simply cropping 840 rows from the upper and lower sides. Figure 4(a) shows a photograph of the optical setup. Since the SLM kit basically contains green and blue laser sources, we filtered them by using a red filter, and a convex lens was used for magnifying the reconstructed hologram image. After that, a neutral density (ND) filter and iris were used for controlling the brightness of the hologram and blocking the non-diffracted beam from the SLM, respectively. The CGH patterns have an off-axis angle of 2° to avoid the non-diffracted beam signal, and the reconstructed hologram magnified by the lens was observed by a complementary metal-oxide semiconductor (CMOS) digital camera. Figure 4(b) shows a reconstructed hologram image from CGH generated by the conventional ASM method, while Figs. 4(c)–4(h) are reconstructed hologram images of CGH using the proposed method with magnification factors of
We propose a method to generate CGH with rapid calculation time by using a pre-calculated concave lens function dependent on focal length and reduced-resolution CGH pattern. Compared to the conventional ASM method, the proposed method was able to generate approximately 6 to 39 times faster depending on the magnification factor. Therefore, the CGH pattern was able to generate at a rate of 10 fps in ×10 magnification factors in relatively low-performance computing devices. Experiments with the SLM show that the quality of reconstructed hologram images does not fall significantly even in high magnification factor, except for slight blurring of the reconstructed image and increase in noise in the background; therefore, the proposed method is expected to solve space and cost problems in various holographic systems. The rapid calculation time proportional to 1/
Institute of Information & communications Technology Planning & Evaluation (IITP) funded by MSIT (No. 2019-0-00001, Development of Holo-TV Core Technologies for Hologram Media Services); Technology Innovation Program funded by the MOTIE, Korea (P20010672).
Curr. Opt. Photon. 2022; 6(1): 51-59
Published online February 25, 2022 https://doi.org/10.3807/COPP.2022.6.1.051
Copyright © Optical Society of Korea.
School of Electronic and Electrical Engineering, College of IT Engineering, Kyungpook National University, Daegu 41566, Korea
Correspondence to:^{*}seungyeol@knu.ac.kr, ORCID 0000-0002-8987-9749
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Holographic display technology is one of the promising 3D display technologies. However, the large amount of computation time required to generate computer-generated holograms (CGH) is a major obstacle to the commercialization of digital hologram. In various systems such as multi-depth head-updisplays with hologram contents, it is important to transmit hologram data in real time. In this paper, we propose a rapid CGH computation method by applying an arraying of a down-scaled hologram with the multiplication of a shifted concave lens function array. Compared to conventional angular spectrum method (ASM) calculation, we achieved about 39 times faster calculation speed for 3840 × 2160 pixel CGH calculation. Through the numerical investigation and experiments, we verified the degradation of reconstructed hologram image quality made by the proposed method is not so much compared to conventional ASM.
Keywords: Computer-generated holograms generation algorithm, Digital holography, Image processing
Digital holography has been applied in various fields such as 3D displays, cultural asset exhibitions and performances, lithography, and microscope technologies [1_{–}7]. Since digital holograms can reconstruct actual 3D objects from planar display panels without causing dizziness to the observer, it is often referred to as the ultimate technology for creating 3D scenes. In recent years, various kinds of visual systems that directly apply holographic technology such as holographic head-up-displays (HUD) for vehicles have been studied and demonstrated [8_{–}12]. In such applications, one of the most important features is real-time computation of computer-generated holograms (CGH) to provide the current status of a vehicle and environmental information to passengers in real time. However, real-time calculation of CGH patterns is still a major bottleneck to fully commercializing holographic HUD systems due to the large amount of computation time and memory usage required to calculate CGH.
To reduce the calculation time of CGH, various studies have been conducted [13_{–}15]. For example, using polygon-based computational methods instead of point cloud [16], algorithms for the efficient numerical propagation of wavefields [17] were improved using wavelet transform-based calculation [18], and further studies have been conducted for fast calculation of CGH in many different routes [19_{,} 20]. Moreover, the use of multiple GPUs to calculate holograms in real time [21], fast CGH calculation from 3D objects consisting of multiple layers of line-drawn objects [22], or fast CGH algorithms based on pinhole-type look-up tables have been proposed [23].
However, applying the above-mentioned methods still seems to be not good enough for the practical use of CGH calculation for real-time application since the major bottleneck of plane-by-plane type CGH calculation time, caused by conducting 2D fast Fourier transform (FFT), is too large. For example, with MATLAB 2021a, computation time of 2D FFT of random phase data still consumes more than 0.3 s with a high-end specification GPU (NVIDIA GeForce RTX3090; NVIDIA, CA, USA) for 4096 × 4096 resolution, which required full data of CGH for a commercialized ultra-high definition (UHD) resolution spatial light modulator (SLM). Since the 2D FFT calculation in CGH generation is a necessary part, a method for calculating high-resolution CGH from low-resolution 2D FFT calculation may be needed. Conventional upscaling algorithms such as bicubic interpolation do not work well in CGH patterns, so a unique approach for upscaling CGH patterns may be needed.
In this work, we proposed to dramatically reduce the computational time of CGH using a concave lens array multiplied to the array of down-scaled resolution calculated CGH. Using this approach, we propose a method to generate a 4096 × 4096 resolution CGH in real time in a relatively low-performance computational GPU environment. Since the shape of the concave lens functions was fixed by the distance between CGH and object plane, computing CGHs of different objects with the same distance is much faster by pre-computing concave lens functions. We also compare the quality of reconstructed holograms from the CGHs computed by the conventional angular spectrum method (ASM) and proposed work through simulations and experiments.
Figure 1 shows a schematic of the proposed CGH resolution upscaling method using a concave function array compared to conventional ASM. For simplicity, we will assume that the pixel number (
Considering the reconstruction process of CGH, the reduced image is our “input” and the magnified original image is our “output” for the imaging equation. Therefore, the minus sign in
After preparing the resized image, the second step is a calculation of simple CGH with a reduced resolution condition. Since FFT calculations are conducted along
The final step of the proposed method is the multiplication of the concave lens array function into the array of reduced resolution CGH. The concave lens array function needs to be appropriately shifted to provide the reconstructed object image placed with the same size and position. In other words, after being magnified by the corresponding concave lens array function, each reduced-resolution CGH will generate an identical reconstructed image observed by a different viewing angle. To provide such a condition, the center location of each concave lens function
Here,
Therefore, each of the concave lens functions is relatively shifted with respect to the center of CGH blocks with the amount of
in order to provide the identical image location of the reconstructed hologram from each CGH block.
Since the proposed method is focused on a dramatic reduction of the calculation time of the CGH that is upscaled from a low-resolution CGH, there must be a trade-off relationship with fast calculation time. The loss of information compared to direct calculation of the full-resolution CGH might be observed as a degradation of image quality. Figure 2 shows the simulation results using the same object under conditions of wavelength at 627 nm and focal length (
Figure 2(a) shows CGH calculated by conventional ASM as a control group (3840 × 3840 resolution, 3.6 μm pixel pitch) and Fig. 2(b) shows CGH calculated by the proposed method of M = 5. Figure 2(d) shows a reconstructed hologram image of Fig. 2(a) and Fig. 2(e) shows a reconstructed hologram image of Fig. 2(b). When comparing Fig. 2(d) and Fig. 2(e) by enlarging the character parts, although speckle noise is increased by applying the proposed method, the reconstructed object is still observable. It is noteworthy that the CGH calculation time in Fig. 2(d) required 3879 ms in our system, whereas calculation of Fig. 2(e) only required 96 ms. On the other hand, Fig. 2(c) shows a single portion of CGH calculated by reduced image without
Since a discretized lens function needs to be multiplied to the array of reduced-size CGH pattern, we need to investigate the possible range of image reconstruction without unwanted diffracted images occurring. When the proposed method is applied, the unwanted diffracted images occur when the focal length is too close, whereas speckle noises become severe when the focal length is too far.
Table 1 shows a comparison of the reconstructed hologram image’s degradation effect with respect to focal length and magnification factor, and Table 2 shows a reconstructed hologram image of Table 1. In Table 2, The images in the first row shows the reference, which conducts the full ASM simulation of the 4k × 4k resolution CGH, and the images in the second to fourth rows are reconstructed images when
TABLE 1. Comparison of computer-generated holograms (CGH) noise and blurring according to focal length and magnification factor.
Magnification Factor [ | Focal Length ( | ||||
---|---|---|---|---|---|
5 cm | 10 cm | 17 cm | 30 cm | 50 cm | |
Original | |||||
2× | |||||
4× | |||||
8× |
TABLE 2. Reconstructed hologram image of Table 1.
Magnification Factor [ | Focal Length ( | ||||
---|---|---|---|---|---|
5 cm | 10 cm | 17 cm | 30 cm | 50 cm | |
Original | |||||
2× | |||||
4× | |||||
8× |
Except for the case of the first row (conventional ASM), the diffracted image can be seen at a focal length of 5 cm (first column). Although such diffracted images disappear in the 10 cm distance condition, weakly blurred images remain as background noise can be seen (second column). However, from about 17 cm, such noises disappear so that the image quality of the proposed method is reasonably good compared to conventional ASM. When the focal length of the image plane is far away, speckle noise increases much faster when
As shown in Table 2, diffracted images up to 17 cm and increased speckle noises are major problems caused by CGH generation using lens functions. The main cause of noise is the empty space of CGH that occurs when calculating CGH using reduced images according to given parameters (resolution, pixel pitch, wavelength, size of object and focal length), which can also be confirmed in Table 1. Because other parameters are difficult to modify in a fixed environment, it is recommended to design an appropriate focal length to avoid these problems. Since it only shows a qualitative analysis in Table 2, we also extracted the quantitative figure of merit parameter from each of the reconstructed images. We extract the structural similarity index measurement (SSIM) value that can be used for comparing the degree of distortion of structural information in images. This method can precisely determine how much the image of the experimental group differs from its reference group, which can be expressed as [24]
In Eq. (5), μ_{x}, μ_{y} indicate the
Based on these calculations, Fig. 3(a) shows the comparison results of SSIM graph according to the focal length and magnification factor
Table 3 shows the exact time consumption of the proposed method according to the variation of the value
TABLE 3. Comparison of computational time of 3840 × 3840 computergenerated holograms (CGH) according to magnification factor (computation time is obtained as the average of 100-time measurements).
Magnification Factor [ | 1 | 2 | 4 | 5 | 6 | 8 | 10 |
---|---|---|---|---|---|---|---|
Image Resizing (ms) | N/A | 43 | 34 | 30 | 35 | 33 | 32 |
Calculate ASM (ms) | 3879 | 520 | 140 | 96 | 68 | 43 | 38 |
Lens Function Preparation (ms) | N/A | 944 | 826 | 788 | 761 | 701 | 663 |
Arraying CGH and Multiply Lens Function (ms) | N/A | 45 | 32 | 29 | 31 | 31 | 29 |
Total Computation Time (ms) | 3879 | 1552 | 1032 | 943 | 895 | 808 | 762 |
Actual Calculation Time for Single Frame (ms) | 3879 | 608 | 206 | 155 | 134 | 107 | 99 |
To compare the image quality between each magnification factor condition, which is shown numerically in Table 2, we demonstrate a hologram generation experiment as shown in Fig. 4. Here, we used the Peony-62a SLM kit (May Display, Suwon, Korea), which has a pixel resolution of 3840 × 2160 and pixel pitch of 3.6 μm. Therefore, we could directly use the CGH pattern calculated in the previous section after simply cropping 840 rows from the upper and lower sides. Figure 4(a) shows a photograph of the optical setup. Since the SLM kit basically contains green and blue laser sources, we filtered them by using a red filter, and a convex lens was used for magnifying the reconstructed hologram image. After that, a neutral density (ND) filter and iris were used for controlling the brightness of the hologram and blocking the non-diffracted beam from the SLM, respectively. The CGH patterns have an off-axis angle of 2° to avoid the non-diffracted beam signal, and the reconstructed hologram magnified by the lens was observed by a complementary metal-oxide semiconductor (CMOS) digital camera. Figure 4(b) shows a reconstructed hologram image from CGH generated by the conventional ASM method, while Figs. 4(c)–4(h) are reconstructed hologram images of CGH using the proposed method with magnification factors of
We propose a method to generate CGH with rapid calculation time by using a pre-calculated concave lens function dependent on focal length and reduced-resolution CGH pattern. Compared to the conventional ASM method, the proposed method was able to generate approximately 6 to 39 times faster depending on the magnification factor. Therefore, the CGH pattern was able to generate at a rate of 10 fps in ×10 magnification factors in relatively low-performance computing devices. Experiments with the SLM show that the quality of reconstructed hologram images does not fall significantly even in high magnification factor, except for slight blurring of the reconstructed image and increase in noise in the background; therefore, the proposed method is expected to solve space and cost problems in various holographic systems. The rapid calculation time proportional to 1/
Institute of Information & communications Technology Planning & Evaluation (IITP) funded by MSIT (No. 2019-0-00001, Development of Holo-TV Core Technologies for Hologram Media Services); Technology Innovation Program funded by the MOTIE, Korea (P20010672).
TABLE 1 Comparison of computer-generated holograms (CGH) noise and blurring according to focal length and magnification factor
Magnification Factor [ | Focal Length ( | ||||
---|---|---|---|---|---|
5 cm | 10 cm | 17 cm | 30 cm | 50 cm | |
Original | |||||
2× | |||||
4× | |||||
8× |
TABLE 2 Reconstructed hologram image of Table 1
Magnification Factor [ | Focal Length ( | ||||
---|---|---|---|---|---|
5 cm | 10 cm | 17 cm | 30 cm | 50 cm | |
Original | |||||
2× | |||||
4× | |||||
8× |
TABLE 3 Comparison of computational time of 3840 × 3840 computergenerated holograms (CGH) according to magnification factor (computation time is obtained as the average of 100-time measurements)
Magnification Factor [ | 1 | 2 | 4 | 5 | 6 | 8 | 10 |
---|---|---|---|---|---|---|---|
Image Resizing (ms) | N/A | 43 | 34 | 30 | 35 | 33 | 32 |
Calculate ASM (ms) | 3879 | 520 | 140 | 96 | 68 | 43 | 38 |
Lens Function Preparation (ms) | N/A | 944 | 826 | 788 | 761 | 701 | 663 |
Arraying CGH and Multiply Lens Function (ms) | N/A | 45 | 32 | 29 | 31 | 31 | 29 |
Total Computation Time (ms) | 3879 | 1552 | 1032 | 943 | 895 | 808 | 762 |
Actual Calculation Time for Single Frame (ms) | 3879 | 608 | 206 | 155 | 134 | 107 | 99 |