A transparent display is a technology to pass the ambient light from real objects behind a display panel such as a liquid crystal display (LCD) or an organic light emitting diode (OLED) display [1-4]. Though the concept of a transparent display was proposed several decades ago through various science fiction comics and movies, display devices of the past generation were not appropriate to realize the vision at that time. Recently, the technological developments in flat panel displays have made the realization of commercial two-dimensional (2D) transparent displays become possible for a wide range of augmented reality (AR) applications such as smart glasses or smart windows.

However, there still remain some issues, such as a convergence mismatch, which disturb a natural perception when the observer is using a 2D transparent display. A convergence mismatch is a problem that causes the convergence point of a 2D transparent image to be separated from the real object as shown in Fig. 1(a). Since the convergence mismatch can cause a double vision or duplicated perception when the binocular fusion of the observer is locked to the real object or the 2D transparent image, it is necessary to resolve the problem and let the observer perceive a 3D transparent image at the location of the real object as shown in Fig. 1(b). For that purpose, several three-dimensional (3D) technologies such as super multi-view (SMV), holographic optical elements (HOE), and integral imaging can be good solutions to realize a 3D transparent display [5-14]. However, those methods still have a technological gap to be bridged before being implemented in a commercial product.

Figure 1. The convergence mismatch between a real object and a display panel in case of T2D display (a) and T3D display (b).

A parallax barrier (PB) is a well-known method to realize an autostereoscopic 3D display without special glasses [15-16]. Since a PB is already verified by having been used in various commercial autostereoscopic 3D displays, it can be a practical solution to realize a 3D transparent display. The idea of a 3D transparent display using a PB is shown in Fig. 2. As shown in Fig. 2, each eye of the observer can see different pixels to show the left-eye or the right-eye images separately. Therefore, if a combination of left-eye and right-eye images is shown on an LCD panel with proper disparity, the observer can see the 3D transparent image without the convergence mismatch problem as shown in Fig. 2.

However, since the conventional PB 3D display is not designed for implementing a 3D transparent display, there exists an issue due to the low transmittance through the 3D transparent images. As shown in Fig. 2, the light ray from a real object can be blocked by the pixels composing the 3D transparent image and this phenomenon can cause a severe problem when the 3D image becomes darker. Therefore, we propose a method to enhance the transmittance and the resolution of the 3D transparent display by using an active PB and an LCD panel with see-through areas.

Figure 2. The principle and structure of a 3D transparent display using a conventional PB.

In this chapter, a principle to realize a 3D transparent display by using an active PB and an LCD panel with see-through areas is described. At first, the see-through areas are realized by fully white pixels to pass ambient light rays. Then, the transmittance of the 3D transparent display can be enhanced by distributing those see-through areas in the left-eye and right-eye images. If the see-through areas are uniformly distributed, a set of pixel array to be composed of four kinds of pixels (LV pixel for left-eye view, RV pixel for right-eye view, LS pixel for left see-through area, and RS pixel for right see-through area) will be formed as shown in Fig. 3. Therefore, the see-through area can guarantee the transmittance of the LCD panel regardless of the grayscale of the displayed 3D image, even though the 3D image is composed of black pixels only as shown in the example of Fig. 3.

Figure 3. Method to generate displayed image on T3D display by using fixed PB with the see-through area.

However, the adaptation of the see-through areas makes the 3D image consist of fewer pixels and the perceived 3D resolution can therefore be degraded. Thus, it is necessary to overcome the trade-off relation between the 3D image quality and the transmittance of the 3D transparent display. Though the implementation of a 3D transparent image using a display device with higher resolution can be the simplest solution to improve the picture quality, there still exists a problem that the transmittance can be reduced. For that point of view, a time-multiplexing scheme using a display panel with higher refresh rate can be a better candidate to enhance the resolution of the 3D display than to increase the pixel density [17-20].

For that purpose, we propose an advanced method replacing the conventional PB with an active PB with a time-multiplexing scheme. The operation of an active PB can be divided into two sequences. In odd frames, half of the even pixels from the left-eye images and half of the odd pixels from the right-eye images are displayed at the LCD panel with the see-through areas at the other halves while the active PBs are at the odd lines as shown in Fig. 4(a). On the other hand, in the even frames, half of the odd pixels from the left-eye images and half of the even pixels from the right-eye images are displayed on the LCD panel while the actives PBs are moved to the even lines as shown in Fig. 4(b). Thus, with the synchronized operation between the above two sequential phases, the information to be displayed by the active PB method with see-through areas can be doubled compared with the 3D transparent display with fixed PB.

Figure 4. Principle of the time-multiplexing 3D transparent display with active PB and see-through areas: (a) odd frame; (b) even frame.

Based on the principles above, we realized a 6.9-inch 3D transparent display using an active PB and a display screen with see-through areas by using two different LCD panels. As real objects, two different university symbols are located at 30 mm and 100 mm from the LCD. Then, the system provides 3D transparent images of ‘N’ at 30 mm and ‘F’ at 100 mm, which means near and far respectively. Since the locations of 3D transparent images are the same as those of real objects, it is expected that they have same perspectives at the left and the right views as shown in Fig. 5. A word ‘Sejong’ was also located at 30 mm for the comparison of the image resolution between the conventional PB and active PB with more detail.

Figure 5. Experimental setup and conditions.

In considering the experimental setups such as the viewing distance and the space between the left-eye and the right-eye view, the basic geometry shown in Fig. 6 was used to eliminate the crosstalk between the views. At first, in order to prevent the mixture of views within a single slit of the PB in Fig. 6(a), the angle of separation θ₁ between the views has the following relation with the pixel pitch p, the gap g between the display panel and the PB, and the space e between the views.

Figure 6. Basic geometry of a transparent 3D display using a PB array.

Secondly, the separation angle θ₁ also needs to have a relation with the resolution R which means the number of the pixels to compose the displayed image. In order to make the left-eye and the right-eye images be separated from each other for the entire display panel area as shown in Fig. 6(b), the following relation should be satisfied.

Then, from Eq. (1) and (2), the equation below can be derived.

The viewing distance v also has a relation with the gap g, the space e, and the pixel pitch p from the geometry shown in Fig. 6(a).

Therefore, it is possible to derive the following equation from Eq. (3) and (4) that the space e between the views is a function of the pixel pitch p and the resolution R.

In the preliminary experiment, the pixel pitch p of the LCD panel is 0.248 mm and the resolution R of the displayed image is 500 pixels. Thus, the space e between the views is set to be 123 mm with viewing distance v of 747 mm. If the system can be upgraded by adopting an LCD panel with finer pixel pitch less than 0.13 mm, the space e between the views can be similar to the interpupillary distance (IPD) of 65 mm.

The experimental results of the 3D transparent display with the conventional PB and the active PB are shown in Fig. 7. Both methods can provide the 3D images of ‘N’ and ‘F’ with the same parallax with the near and the far objects. This means that each image is located at the same location as the corresponding object. Though the 3D images are composed of black pixels, the real objects can be seen through the 3D images because of the see-through areas, as expected. The difference of the resolution from the principle of the active PB is shown at the magnified pictures. In area 1, it can be easily recognized that the proposed active PB method can provide higher 3D resolution since the shapes of letters composing a word ‘Sejong’ are easier to perceive than the conventional ones. The magnified picture of area 2 also shows that the time-sequential operation also enhances the visibility of the real object though the transmittance is the same. Therefore, it can be concluded that the experimental results prove the effect of the proposed method to improve the picture quality of the 3D images and the visibility of the real object.

Figure 7. The comparison of T3D display with between fixed PB and active PB.

In this paper, a method to realize a 3D transparent display with enhanced transmittance and resolution using an active PB with see-through areas is proposed. The see-through areas can increase the transmittance of the proposed system by delivering the information of the real objects behind the display panel regardless of the grayscale of the displayed 3D image. The time-sequential active PB also enhances the picture quality of the 3D image and the visibility of the real objects. Therefore, we expect that the proposed principle can contribute to realize a practical 3D transparent display for various AR applications.