Wide FOV (Field-Of-View) imaging systems are able to acquire significantly more scene information compared to traditional imaging systems. Wdie FOV imaging systems have a wide range of applications which include surveillance, teleconferencing, and autonomous navigation in mobile robots, as well as the AVM (Around View Monitoring) technology which is adopted in the latest vehicle technology to improve safety during driving and parking [1-5]. There are two primary methods to acquire wide FOV images using a single camera: the first involves using a fisheye lens, and the other uses a curved mirror. The curved mirror approach is referred to as the catadioptric method. This type of imaging system is relatively simple and efficient to design, and the acquired images have lower chromatic aberration compared to the image acquired using a fisheye lens imaging system [6-8]. In [9], a catadioptric imaging system using a conical mirror was proposed to acquire 360° images, and to extract omnidirectional distances from the image for autonomous navigation of a mobile robot. An omnidirectional ranging sensor was developed using a catadioptric hyperbolic mirror with line lasers in [10-12]. A catadioptric imaging approach is applicable also to stereo image acquisition. In [13-16], omnidirectional stereo image acquisition methods were proposed using two cameras with hyperbolic mirrors, and a single camera with two hyperbolic mirrors. A similar single camera approach using a hyperbolic mirror with an additional concave lens was presented for omnidirectional stereo imaging in [17].

The catadioptric imaging method acquires a reflected image from a curved mirror using a conventional camera placed in front of the mirror. There are several kinds of bowl-shaped curved mirrors available to achieve this objective such as conic, hyperbolic, parabolic, and elliptic mirrors [18, 19]. In [20], a combined hyperbolic mirror was proposed that consists of a cylindrical hyperbolic upper part and an omnidirectional hyperbolic lower part.

Placing a camera at a specified distance in front of a curved mirror is cumbersome in practice because the camera module requires additional equipment and space. This causes unnecessary occlusions in the image. In order to solve this problem, a secondary planar mirror is used in this study; a small-sized planar mirror replaces the camera, and the camera is shifted to the rear concave side of a curved mirror. The camera acquires an image reflected from the primary curved mirror followed by the secondary planar mirror through a hole at the apex of the curved mirror. A compact imaging module containing a curved mirror and a camera is designed in this study to alleviate the cumbersome camera placement at the front side of the curved mirror. It is easier to deploy the secondary planar mirror instead of a camera module because of its smaller size and proximity to the primary curved mirror in general. The size of the secondary planar mirror depends on the position with respect to the primary curved mirror. This study addresses the position of the planar mirror in consideration of the central imaging condition of a curved mirror. The image acquisition model is also discussed based on principles of geometric optics and the image restoration algorithm.

The organization of this study is as follows: In Sec. II and Sec. III, the imaging model of the catadioptric system using a curved mirror with a secondary planar mirror is explained. Sec. IV addresses the size and the placement of the planar mirror to satisfy the central imaging condition. The experimental results and the concluding remarks are presented in Sec. V and Sec. VI.

In this study, the combined hyperbolic mirror in [20] is adopted as a curved mirror. The hyperbolic mirror is designed to achieve 213° horizontal and 126° vertical FOV and can be used for the vehicle AVM. The original design of the combined hyperbolic mirror is modified to compactly attach a camera at the rear concave side of the mirror, as shown in Fig. 1. Figure 2 shows the concept of the imaging system using the combined hyperbolic mirror with a secondary planar mirror as implemented in this study. The idea of this study is applicable to every kind of curved mirror without loss of generality.

Figure 1. Design of the combined hyperbolic mirror.

Figure 2. Imaging system using the combined hyperbolic mirror and secondary planar mirror.

The upper part of the curved mirror is described by a cylindrical hyperbolic function as

where a and b are the parameters of the hyperbolic function. The hyperbolic function has a focal point as follows:

The lower part of the curved mirror is described by a half of the omnidirectional hyperbolic function as

The upper and the lower parts are smoothly combined without any seam lines because the hyperbolic functions of the two parts have the same parameters.

The image acquisition model represents the relationship between an object point, P_o = [ x_o y_o z_o]^t in 3D space in 3D space and an image point P_i = [x_i z_i]^t on a 2D image plane. The image acquisition model in this study is similar to that in [20] except the part of the secondary planar mirror. It is well-known that the planar mirror induces a virtual effective camera in its back. The image acquisition model is briefly explained in this section.

3.1. The Upper Part of the Mirror

Figure 3 shows the light reflection in the horizontal plane of the upper part of the imaging system. A light ray (Line I) from an object point, P_o to the focal point, −F of the hyperbolic mirror, is reflected on the mirror surface and directed toward the symmetrical focal point, F as Line II. The light ray (Line II) is reflected in the secondary planar mirror at y = m a second time and produces an image on the camera sensor through a hole at the apex of the hyperbolic mirror. Imaging from a planar mirror is the same as imaging on an effective image plane through an effective pinhole as shown in Fig. 3. The positions of the effective pinhole and the effective image plane are symmetrical with those of the real camera with respect to the planar mirror. It is noted that the planar mirror should be placed at the middle of the actual pinhole position, P and the focal point of the symmetrical hyperbolic function, F as described by Eq. (4) in order to locate the effective pinhole at F for the central imaging condition.

Figure 3. Image acquisition model on horizontal plane.

The image acquisition model between P_o and P_i is summarized as follows:

where λ denotes the focal length of the camera lens. In Eq. (5), x_c and y_c denote the reflection point of the incidence ray (Line I) and are obtained by a solution of Eq. (6).

The parameters, t_i, t_p, and t_o for z_i in Eq. (5b) are given by

Please refer to [20] for detailed derivation of the image acquisition model.

3.2. The Lower Part of the Mirror

Figure 4 shows the cross-section in the x − y(r − y) plane of the lower part of the curved mirror. Because of the rotational symmetry about the y-axis, x is substituted with r for representing the cylindrical coordinates.

Figure 4. Image acquisition model for lower part of the mirror: (a) Cross-section in x − y(r − y) plane, (b) Front view.

The relationship between P_o and P_i is represented by

where r_o is as shown in Fig. 4(b). Because Fig. 4 is the same as Fig. 3, r_i is obtained from Eqs. (5a) and (6) by changing x with r.

f5

This section addresses the position and the size of the secondary mirror, which has an influence on the central imaging condition and on the occluded area of the resultant image in the proposed catadioptric imaging system.

4.1. Size and Position of the Secondary Planar Mirror

The effective pinhole is the symmetric transformation of P about the secondary planar mirror and should be located at F for the central imaging condition of the hyperbolic mirror; this provides the condition for the position of the planar mirror as described in Eq. (4). The size of the planar mirror is determined according to its position, as depicted in Fig. 5. From the simple proposition, it is easy to determine the right size of the planar mirror, M, as Eq. (9).

Figure 5. Size and position of secondary planar mirror (top view).

The size of the mirror M depends on the pinhole position of the camera, P. If the size of the mirror is less than M, the planar mirror does not cover the entire reflected field of light from the curved mirror. On the contrary, if the size is bigger than M, the occluded area by the planar mirror becomes too large as shown in Fig. 6.

In Fig. 6, the occlusion angle is obtained as follows:

Figure 6. Occlusion due to secondary planar mirror.

In order to reduce this angle ρ, the size of the planar mirror M, should be small enough, which requires that the pinhole position of the camera P, should be as close as possible to the apex of the hyperbolic mirror in the backside.

4.2. Hole Size at the Apex of the Primary Curved Mirror

Figure 7 shows the hole size of the hyperbolic mirror.

Figure 7. Hole size of primary curved mirror.

The appropriate hole size depends on the size of the planar mirror. The light ray passing through the edge of the planar mirror P_m, and the focal point −F, are represented by the line equation, Eq. (11) using Eq. (4).

The diameter of the hole is determined by the solution of the hyperbolic function Eq. (1) and the line equation Eq. (12) as follows:

The diameter of the hole φ is given by 2x_h.

Experiments were performed to evaluate the imaging system presented in this study. The experimental setup is shown in Fig. 8 where the parameters of the hyperbolic mirror are summarized in Table 1. The parameters are defined in Fig. 1.

Figure 8. Imaging system using a combined hyperbolic mirror and secondary planar mirror: (a) Front and rear view of combined hyperbolic mirror with a camera, (b) Experimental setup with the secondary planar mirror.

TABLE 1.. Parameters of the combined hyperbolic mirror (mm).

According to the parameters, the size of the planar mirror in Eq. (9), the occlusion angle in Eq. (10), and the radius of the hole in Eq. (12) are given as follows when P is closely located around the apex of the hyperbolic mirror:

It is well-known that in a catadioptric imaging system with a hyperbolic mirror, the central imaging condition is of importance for image restoration to a natural image. In the experiment, the secondary planar mirror is placed at the position described by Eq. (4) by manual adjustment while taking into account the central imaging condition. It is possible to confirm the central imaging condition in Fig. 9(a); the parallel lines of the grid pattern in 3D space have circular images with common vanishing point on the image plane, and a line connecting the vanishing points passes through the optical center of the image [21]. On the contrary, the circular images of the parallel lines do not have a common vanishing point in the non-central image in Fig. 9(b).

Figure 9. Placement of the secondary planar mirror for central imaging: (a) Central image, (b) Non-central image.

Figure 10(a) shows another image acquired by the imaging system. The upper part and the lower parts of the image are restored to show a panoramic image in Fig. 10(b) and a bird’s eye view in Fig. 10(c), based on the image acquisition model in Sec. III.

Figure 10. Experimental result: (a) Original image, (b) Restored image: mercator image, (c) Restored image: bird’s eye view.

Wide FOV imaging systems are able to acquire rich visual information and have attracted increasing attention for many industrial applications such as video surveillance, autonomous navigation in mobile robots, and vehicle AVM, etc. Conventional catadioptric imaging systems with a curved mirror for wide FOV is hindered by the cumbersome camera deployment and the image occlusion. This study presented a new catadioptric imaging system consisting of a compact combination of a curved mirror and camera module, and a secondary planar mirror. In general, it is easier to install a secondary planar mirror in front of a curved mirror, compared to a bulky camera module. This study presents an approach for determining both the size of the planar mirror and the placement of the mirror to satisfy the central imaging condition. The image acquisition model was addressed for the catadioptric imaging system with the secondary planar mirror based on geometrical optics. The image acquisition model is a forward mapping between a scene point in 3D space and an image point on a 2D image plane. As a backward mapping, the acquired image is restored to a distortion-free image and the central imaging condition was verified in the experiments depending on the placement of the secondary planar mirror.