We propose a novel method that synthesizes a color computer-generated-hologram (CGH) from light field data. CGH plays an important role in three-dimensional (3D) holographic displays [1, 2]. For the hologram synthesis, 3D objects are represented in various forms, and one of them is the light field. Light field data is expressed as a set of spatial and angular rays of light coming from 3D objects [3_–7]. The representation of these rays is equivalent to an array of views that look at the 3D object from various directions. Creating holograms from light field data has the advantage that the light field acquisition of real objects is easily achieved with a light field camera [4]. Another advantage of the light field CGH is that scene details such as occlusion and material reflection properties are already included in the light field data, and they can be reflected in the hologram with proper processing.

Synthesizing holograms from light field data has been researched for decades [5_–8]. Most of the conventional methods divide the holographic plane into small areas called hogels [9, 10]. Each hogel is processed with a corresponding view and the processed hogels are assembled together, completing the hologram. These hogel based methods, however, generally have a limitation on the number of hogels because of the tradeoff relationship between the number of hogels and the number of pixels in each hogel at a given hologram resolution. This hogel number limitation reduces the maximum spatial resolution of the reconstructed 3D images. Another limitation of the hogel based methods is the phase mismatch. Since reconstruction from each hogel has phase mismatch with the one from the neighboring hogel, continuous wavefront over the hologram plane cannot be reproduced.

Recently, a non-hogel-based CGH method has been introduced [11]. The non-hogel-based CGH method processes all the views in the light field data globally, solving the shortcomings of the hogel-based method. Applicability of arbitrary carrier wave or phase distribution on the 3D object surface is another advantage, enabling optimization of the hologram for each specific application. Although the requirement of a densely sampled large amounts of light field data was problematic in the initial proposal of the non-hogel-based CGH [11], a more efficient calculation scheme has been developed later, reducing the computation time significantly [12].

Despite the advantages, the non-hogel-based CGH has only been demonstrated for a single color. The non-hogel-based CGH method involves sampling and manipulation of the light field data in a spatial frequency domain. Different wavelength results in different spatial frequency for the same ray direction. As red, green, and blue color channels of each view in the light field data share the same field of view (FoV) and observing direction, each color channel has a different spatial frequency sampling grid whose efficient processing is not trivial and has not been developed yet.

In this paper, we propose a method for calculating color non-hogel based CGH. The proposed method interpolates the two-dimensional (2D) spatial frequency grid of green and blue color channels using the sampling interval of a red color channel. Meanwhile, the spatial frequency range is set by the blue channel and the red and green channels are zero-padded to match the range. The proposed spatial frequency grid manipulation using the red sampling interval and blue grid range ensures efficient processing without loss of information. The proposed method is verified by comparing the reconstructions of color CGHs synthesized with and without the proposed method.

The non-hogel-based CGH technique synthesizes complex fields from light field data of the 3D scenes. In a plane, the light field can be denoted by l(t_x, t_y, θ_x, θ_y), where (t_x, t_y) is the spatial position of the ray and (θ_x, θ_y) is the angular direction measured in radians. The light field l(t_x, t_y, θ_x, θ_y) is fully defined in geometric optics. However, to relate the light field with a hologram defined in wave optics, the non-hogel-based CGH technique interprets the ray direction (θ_x, θ_y) of the light field as the spatial frequency (u, v) of the plane wave having the same direction as the ray [11]. The spatial frequency – ray angle relation is given by

where λ is the wavelength. The light field is now represented by L(t_x, t_y, u, v) = l(t_x, t_y, θ_x = λu, θ_y=λv) with the spatial position and spatial frequency.

Hologram synthesis from the light field L(t_x, t_y, u, v) by the non-hogel-based CGH consists of two steps [11, 12]. In the first step, the light field L(t_x, t_y, u, v) is 2D Fourier transformed along the (u, v) axes, giving

where (τ_x, τ_y) represents axes after the 2D Fourier transform over (u, v) axes. The hologram H(x, y) is then synthesized by

where x_c and y_c are spatial positions in the hologram plane and W(x_c, y_c) represents a carrier wave which can be selected arbitrarily. Using τ_x = x − x_c, τ_y = y − y_c, t_x = x − τ_x / 2, t_y = y − τ_y / 2, Eq. (3) can also be rearranged to

Equation (4) indicates that each (τ_x, τ_y) slice of the L~ (t_x, t_y, τ_x, τ_y) is multiplied with W and accumulated in the hologram plane with a corresponding shift (±τ_x/2, ±τ_y/2), completing the hologram.

Implementation of the non-hogel-based CGH explained in the previous section involves discrete signals with adequate sampling. The wavelength dependency of the sampling requirement makes the color hologram synthesis non-trivial.

Suppose that a light field data l(t_x, t_y, θ_x, θ_y) is prepared with N_tx × N_ty × N_θx × N_θy samples with ∆t_x, ∆t_y, ∆θ_x, ∆θ_y sampling intervals. The angular range of the light field data, or FoV is given by FoV_x = N_θx∆θ_x, FoV_y = N_θy∆θ_y. From Eq. (1), the light field represented in spatial frequency L(t_x, t_y, u, v) has N_tx × N_ty × N_u(= N_θx) × N_v(= N_θy) samples with ∆t_x, ∆t_y, ∆u(=∆θ_x,/λ), ∆v(=∆θ_y/λ) sampling intervals. The hologram sampling number N_x × N_y and interval ∆x and ∆y are determined to cover the spatial and angular range of the light field, i.e. N_tx∆t_x × N_ty∆t_y in the spatial domain and FoV_x × FoV_y = N_θx∆θ_x × N_θy∆θ_y in the angular domain which gives a condition:

where Eq. (5) is deduced from the maximum diffraction angle of the hologram with a pixel pitch ∆x and ∆y for a wavelength λ. Equations (5) and (6) indicate that for a given FoV the required sampling pitch and the sampling number of the hologram are dependent on the wavelength.

Another sampling consideration is for the τ_x and τ_y of the light field. From Eq. (2), the sampling interval ∆τ_x and ∆τ_y are given by

where N_u = N_θx, N_v = N_θy, ∆u = ∆θ_x/λ, and ∆v = ∆θ_y/λ are used. Note that from Eqs. (5) and (7) the hologram sampling interval ∆x, ∆y and the light field sampling interval ∆τ_x, ∆τ_y can be the same and this is advantageous to implement the shift operation in Eq. (4).

Suppose a color light field data is given with the same FoV for all color channels. The red, green, and blue channels of the light field data l_R(t_x, t_y, θ_x, θ_y), l_G(t_x, t_y, θ_x, θ_y), l_B(t_x, t_y, θ_x, θ_y) have the same number of the sampling points N_tx × N_ty × N_θx(= N_u) × N_θy(= N_v) with the same sampling intervals ∆t_x, ∆t_y, ∆θ_x(= FoV_x / N_θx = FoV_x / N_u), and ∆θ_y(= FoV_y/N_θy = FoV_y / N_v) as illustrated in Fig. 1(a). Because of the wavelength dependency of the angle – spatial frequency relation in Eq. (1), the spatial frequency range u, v contained in the light field is different for each color channel at the same FoV as given by

Figure 1. Proposed method: (a) original color light field data with the same FoV for all color channels, (b) resampling by zero-padding, and (c) interpolation. Vertical axis is represented in ray angle θ_x.

where λ_R, λ_G, and λ_B are the wavelengths of the red, green, and blue color channel. The sampling interval of the spatial frequency can also be obtained by dividing the range in Eq. (8) by the number of samples N_θx = N_u, N_θy = N_v:

Equations (8) and (9) indicate that the spatial frequency range and the sampling interval in L_R(t_x, t_y, u, v), L_G(t_x, t_y, u, v), and L_B(t_x, t_y, u, v) of the same FoV are inversely proportional to the wavelength, giving largest values for the blue color channel as illustrated in Fig. 2(a). Note that the sampling interval ∆τ_x, ∆τ_y is also different for different colors, implementation of the shift operation in Eq. (4) is complicated.

Figure 2. Proposed method: (a) original color light field data with the same FoV for all color channels, (b) resampling by zero-padding, and (c) interpolation. Vertical axis is represented in spatial frequency u.

To synthesize a color hologram without information loss of the light field, the proposed method sets a single common hologram pixel pitch and performs 2D interpolation along the u, v axes of the light field with proper zero-padding operation. First, the hologram pixel pitch is determined for the diffraction angle of each color channel to cover the FoV. From Eq. (5), the pixel pitch requirements of three-color channels are given by

To satisfy three conditions of Eq. (10) together, the hologram pixel pitch is set to be

following the blue condition of the shortened wavelength.

Once the pixel pitch condition is determined, the FoV of each color channel is extended with zero-padded in the proposed method. As mentioned earlier, it is advantageous that the sampling interval ∆τ_x is equal to the hologram sampling interval ∆x in implementing the shift and accumulation operation of Eq. (4). Considering the hologram pixel pitch condition given in Eq. (11), the FoVs of red and green color channels are manipulated according to Eq. (7) by

as illustrated in Fig. 1(b). Note that the manipulated FoVs, i.e., FoV_x,R and FoV_x,G, are always larger than the original FoV_x.

Finally, each color channel of the light field data L_R(t_x, t_y, u, v), L_G(t_x, t_y, u, v), L_B(t_x, t_y, u, v) is interpolated along the u, v axes with the original sampling interval of the red channel ∆u_R and ∆v_R given by Eq. (9). Note that the u, v sampling interval common to all color channels is selected to be red channel one ∆u_R and ∆v_R because it is the smallest according to Eq. (9) and prevents possible information loss due to the increase of the sampling interval. Because the FoV increases in the case of red and green channels, i.e., from FoV_x to FoV_x,R or FoV_x,G and the sampling interval decreases in the case of green and blue channels, i.e., from ∆u_G or ∆u_B to ∆u_R, the number of sampling points along the u, v axes increases in all color channels to

where Eqs. (8), (9), and (12) are used. In Fig. 2 (a), (b), illustrate the extended FoV with zero-padding and the new interpolation of the proposed method, respectively. Note that by the proposed method all three color channels have the same sampling points N_u′ × N_v′ with the same sampling interval ∆u_R and ∆v_R along u, v axes, which correspond to the same u, v range but different FoVs.

The proposed method is verified by synthesizing color holograms from light field data of various 3D scenes. In all calculations, the red, green, and blue wavelengths are set to be λ_R = 633 nm, λ_G = 532 nm, and λ_B = 488 nm. Figure 3 shows light field data of cross target objects. The scene consists of 4 cross targets which are in the same z = −0.377 mm plane, but have different colors i.e., red, green, blue, and white. The light field data has N_u × N_v = 64 × 64 orthographic views with FoV_x = FoV_y = 7.5°. Each orthographic view has N_tx × N_ty = 300 × 300 pixels with ∆t_x = ∆t_y = 3.73 µm pixel pitch. Using this light field data, color holograms are synthesized with and without the proposed method. When the proposed method is not applied, all red, green, and blue channels of the hologram are synthesized using the blue wavelength λ_B = 488 nm to make the hologram pixel pitch the same for all color channels. In the reconstruction, the red, green, and blue wavelengths λ_R = 633 nm, λ_G = 532 nm, and λ_B = 488 nm are used for the corresponding color channel of the hologram. Figure 4 shows the amplitude and phase of the synthesized holograms. Figure 5 gives their reconstruction results. As can be seen in the top row of Fig. 5, when the proposed method is not applied, color aberration occurs as expected. Although the blue cross target is focused at the original distance z = −0.377 mm, the green and red cross targets are focused at aberrated distances z = −0.346 mm and z = −0.291 mm, respectively, which correspond to λ_B/λ_G and λ_B/λ_R times the original distance z = −0.377 mm. This is because the sampling condition determination and the synthesis of all color channels of the hologram are conducted using the blue wavelength to generate a color hologram with a single pixel pitch shared for all color channels. To the contrary, when the proposed method is applied, the color aberration is removed and all red, green, blue, and white cross targets are focused at the correct distance z = −0.377 mm as shown in the bottom row of Fig. 5. Therefore, the correct color hologram synthesis with the same pixel pitch for all color channels by the proposed method is confirmed for the case of a single depth scene.

Figure 3. Light field data of cross target array. (a) Single orthographic view, and (b) collection of 64 × 64 orthographic views.

Figure 4. Amplitude and phase of hologram without (top) and with (bottom) proposed method.

Figure 5. Numerical reconstructions of holograms synthesized without (top), and with (bottom) proposed method.

Figure 6 shows the light field data of a 3D scene with multiple depths. The scene consists of 4 × 4 resolution targets located at different depths from z = −14.13 mm to z = +14.13 mm with 1.767 mm spacing except z = 0 mm. The light field data has N_u = N_v = 64 orthographic views each of which has N_tx × N_ty = 1563 × 1563 pixels with ∆t_x = ∆t_y = 3.67 µm pixel pitch and FoV_x = FoV_y = 7.62°. Figure 7 shows the amplitudes and the phase of the synthesized hologram without and with the proposed method. Figure 8 shows the numerical reconstructions at various depths, and focused resolution target images corresponding to three depths z = −3.53 mm, −8.83 mm, and −14.13 mm are highlighted. Figure 8(a) shows that color aberration occurs when the proposed method is not applied. It is also observed that the color aberration becomes more severe as the depth increases as the aberrated depths for green and red channels are given by λ_B/λ_G and λ_B/λ_R times the original depth and thus proportional to the original depth. On the other hand, when the proposed method is applied, the resolution targets are focused at their original depths without any color aberration as shown in Fig. 8(b).

Figure 6. Light field data of resolution target array. (a) Single orthographic view, and (b) collection of 64 × 64 orthographic views.

Figure 7. Amplitude and phase of hologram without (top) and with (bottom) proposed method.

Figure 8. Numerical reconstructions of holograms synthesized (a) without, and (b) with proposed method.

Finally, the proposed method is tested with a natural scene with continuous depth 3D objects. A 3D scene model is loaded to a software Blender and orthographic images are rendered as shown in Fig. 9. Each orthographic view has 1563 × 1563 pixels with ∆t_x = ∆t_y = 3.73 µm pixel pitch. N_u × N_v = 64 × 64 orthographic views are rendered within FoV_x = FoV_y = 7.5°. The central depths of objects are z₁ = −0.438 mm (grapes), z₂ = −0.568 mm (tangerine, apple), z₃ = −0.767 mm (pumpkin), z₄ = −0.996 mm (orange), and z₅ = −1.315 mm (basket). Figure 10 shows the amplitude and the phase of the synthesized holograms with and without proposed method and Fig. 11 shows their reconstruction results. In Fig. 11, red and blue boxes highlight focused object parts. It is clearly confirmed from Fig. 11 that the reconstructions are blurred (marked A), dispersed (marked B), or false-colored (marked C) all because of the color aberration when the proposed method is not applied. The proposed method solves these problems, generating clear focused reconstructions as shown in the lower part of Fig. 11.

Figure 9. Light field data of natural scene with continuous depth 3D objects. (a) Single orthographic view, and (b) collection of 64 × 64 orthographic views.

Figure 10. Amplitude and phase of hologram without (top) and with (bottom) proposed method.

Figure 11. Numerical reconstructions of holograms synthesized without (top), and with (bottom) proposed method.

In this paper, we propose a method to calculate a full-color non-hogel-based CGH from light field data of 3D scenes. All color channels of the light field data share the same FoV. But the wavelength dependency of the angle equivalent spatial frequency makes their sampling condition different, making a synthesis of color hologram with a shared pixel pitch complicated. The proposed method resamples the light field data along the spatial frequency axes such that all red, green, and blue color channels have the same sampling grid. The sampling range and interval along the spatial frequency axes are set by the ones of blue and red color channels, respectively, which ensure no loss of information. The proposed method is verified using various 3D scenes including single depth plane test target, multiple plane test targets at different depths, and natural continuous 3D objects distributed in space. The reconstructions of the color holograms synthesized by the proposed method show clear focus of the 3D objects at their depths without any color aberrations, successfully confirming the feasibility of the proposed method.

This work was supported by INHA UNIVERSITY Research Grant.