G-0K8J8ZR168
검색
검색 팝업 닫기

Ex) Article Title, Author, Keywords

Article

Split Viewer

Article

Current Optics and Photonics 2019; 3(4): 329-335

Published online August 25, 2019 https://doi.org/10.3807/COPP.2019.3.4.329

Copyright © Optical Society of Korea.

Compact Catadioptric Wide Imaging with Secondary Planar Mirror

Young-Jun Ko, and Soo-Yeong Yi*

Department of Electrical and Information Engineering, Seoul National University of Science and Technology, Seoul 01811, Korea

Corresponding author: suylee@seoultech.ac.kr

Received: March 2, 2019; Revised: May 28, 2019; Accepted: May 31, 2019

Wide FOV imaging systems are important for acquiring rich visual information. A conventional catadioptric imaging system deploys a camera in front of a curved mirror to acquire a wide FOV image. This is a cumbersome setup and causes unnecessary occlusions in the acquired image. In order to reduce both the burden of the camera deployment and the occlusions in the images, this study uses a secondary planar mirror in the catadioptric imaging system. A compact design of the catadioptric imaging system and a condition for the position of the secondary planar mirror to satisfy the central imaging are presented. The image acquisition model of the catadioptric imaging system with a secondary planar mirror is discussed based on the principles of geometric optics in this study. As a backward mapping, the acquired image is restored to a distortion-free image in the experiments.

Keywords: Catadioptric system, Wide FOV image, Geometric optics, Hyperbolic mirror, Central imaging

Wide FOV (Field-Of-View) imaging systems are able to acquire significantly more scene information compared to traditional imaging systems. Wdie FOV imaging systems have a wide range of applications which include surveillance, teleconferencing, and autonomous navigation in mobile robots, as well as the AVM (Around View Monitoring) technology which is adopted in the latest vehicle technology to improve safety during driving and parking [1-5]. There are two primary methods to acquire wide FOV images using a single camera: the first involves using a fisheye lens, and the other uses a curved mirror. The curved mirror approach is referred to as the catadioptric method. This type of imaging system is relatively simple and efficient to design, and the acquired images have lower chromatic aberration compared to the image acquired using a fisheye lens imaging system [6-8]. In [9], a catadioptric imaging system using a conical mirror was proposed to acquire 360° images, and to extract omnidirectional distances from the image for autonomous navigation of a mobile robot. An omnidirectional ranging sensor was developed using a catadioptric hyperbolic mirror with line lasers in [10-12]. A catadioptric imaging approach is applicable also to stereo image acquisition. In [13-16], omnidirectional stereo image acquisition methods were proposed using two cameras with hyperbolic mirrors, and a single camera with two hyperbolic mirrors. A similar single camera approach using a hyperbolic mirror with an additional concave lens was presented for omnidirectional stereo imaging in [17].

The catadioptric imaging method acquires a reflected image from a curved mirror using a conventional camera placed in front of the mirror. There are several kinds of bowl-shaped curved mirrors available to achieve this objective such as conic, hyperbolic, parabolic, and elliptic mirrors [18, 19]. In [20], a combined hyperbolic mirror was proposed that consists of a cylindrical hyperbolic upper part and an omnidirectional hyperbolic lower part.

Placing a camera at a specified distance in front of a curved mirror is cumbersome in practice because the camera module requires additional equipment and space. This causes unnecessary occlusions in the image. In order to solve this problem, a secondary planar mirror is used in this study; a small-sized planar mirror replaces the camera, and the camera is shifted to the rear concave side of a curved mirror. The camera acquires an image reflected from the primary curved mirror followed by the secondary planar mirror through a hole at the apex of the curved mirror. A compact imaging module containing a curved mirror and a camera is designed in this study to alleviate the cumbersome camera placement at the front side of the curved mirror. It is easier to deploy the secondary planar mirror instead of a camera module because of its smaller size and proximity to the primary curved mirror in general. The size of the secondary planar mirror depends on the position with respect to the primary curved mirror. This study addresses the position of the planar mirror in consideration of the central imaging condition of a curved mirror. The image acquisition model is also discussed based on principles of geometric optics and the image restoration algorithm.

The organization of this study is as follows: In Sec. II and Sec. III, the imaging model of the catadioptric system using a curved mirror with a secondary planar mirror is explained. Sec. IV addresses the size and the placement of the planar mirror to satisfy the central imaging condition. The experimental results and the concluding remarks are presented in Sec. V and Sec. VI.

In this study, the combined hyperbolic mirror in [20] is adopted as a curved mirror. The hyperbolic mirror is designed to achieve 213° horizontal and 126° vertical FOV and can be used for the vehicle AVM. The original design of the combined hyperbolic mirror is modified to compactly attach a camera at the rear concave side of the mirror, as shown in Fig. 1. Figure 2 shows the concept of the imaging system using the combined hyperbolic mirror with a secondary planar mirror as implemented in this study. The idea of this study is applicable to every kind of curved mirror without loss of generality.

Figure 1.Design of the combined hyperbolic mirror.
Figure 2.Imaging system using the combined hyperbolic mirror and secondary planar mirror.

The upper part of the curved mirror is described by a cylindrical hyperbolic function as

where a and b are the parameters of the hyperbolic function. The hyperbolic function has a focal point as follows:

The lower part of the curved mirror is described by a half of the omnidirectional hyperbolic function as

The upper and the lower parts are smoothly combined without any seam lines because the hyperbolic functions of the two parts have the same parameters.

The image acquisition model represents the relationship between an object point, Po = [ xo yo zo]t in 3D space in 3D space and an image point Pi = [xi zi]t on a 2D image plane. The image acquisition model in this study is similar to that in [20] except the part of the secondary planar mirror. It is well-known that the planar mirror induces a virtual effective camera in its back. The image acquisition model is briefly explained in this section.

3.1. The Upper Part of the Mirror

Figure 3 shows the light reflection in the horizontal plane of the upper part of the imaging system. A light ray (Line I) from an object point, Po to the focal point, −F of the hyperbolic mirror, is reflected on the mirror surface and directed toward the symmetrical focal point, F as Line II. The light ray (Line II) is reflected in the secondary planar mirror at y = m a second time and produces an image on the camera sensor through a hole at the apex of the hyperbolic mirror. Imaging from a planar mirror is the same as imaging on an effective image plane through an effective pinhole as shown in Fig. 3. The positions of the effective pinhole and the effective image plane are symmetrical with those of the real camera with respect to the planar mirror. It is noted that the planar mirror should be placed at the middle of the actual pinhole position, P and the focal point of the symmetrical hyperbolic function, F as described by Eq. (4) in order to locate the effective pinhole at F for the central imaging condition.

Figure 3.Image acquisition model on horizontal plane.

The image acquisition model between Po and Pi is summarized as follows:

where λ denotes the focal length of the camera lens. In Eq. (5), xc and yc denote the reflection point of the incidence ray (Line I) and are obtained by a solution of Eq. (6).

The parameters, ti, tp, and to for zi in Eq. (5b) are given by

Please refer to [20] for detailed derivation of the image acquisition model.

3.2. The Lower Part of the Mirror

Figure 4 shows the cross-section in the xy(ry) plane of the lower part of the curved mirror. Because of the rotational symmetry about the y-axis, x is substituted with r for representing the cylindrical coordinates.

Figure 4.Image acquisition model for lower part of the mirror: (a) Cross-section in xy(ry) plane, (b) Front view.

The relationship between Po and Pi is represented by

where ro is as shown in Fig. 4(b). Because Fig. 4 is the same as Fig. 3, ri is obtained from Eqs. (5a) and (6) by changing x with r.

f5

This section addresses the position and the size of the secondary mirror, which has an influence on the central imaging condition and on the occluded area of the resultant image in the proposed catadioptric imaging system.

4.1. Size and Position of the Secondary Planar Mirror

The effective pinhole is the symmetric transformation of P about the secondary planar mirror and should be located at F for the central imaging condition of the hyperbolic mirror; this provides the condition for the position of the planar mirror as described in Eq. (4). The size of the planar mirror is determined according to its position, as depicted in Fig. 5. From the simple proposition, it is easy to determine the right size of the planar mirror, M, as Eq. (9).

Figure 5.Size and position of secondary planar mirror (top view).

The size of the mirror M depends on the pinhole position of the camera, P. If the size of the mirror is less than M, the planar mirror does not cover the entire reflected field of light from the curved mirror. On the contrary, if the size is bigger than M, the occluded area by the planar mirror becomes too large as shown in Fig. 6.

In Fig. 6, the occlusion angle is obtained as follows:

Figure 6.Occlusion due to secondary planar mirror.

In order to reduce this angle ρ, the size of the planar mirror M, should be small enough, which requires that the pinhole position of the camera P, should be as close as possible to the apex of the hyperbolic mirror in the backside.

4.2. Hole Size at the Apex of the Primary Curved Mirror

Figure 7 shows the hole size of the hyperbolic mirror.

Figure 7.Hole size of primary curved mirror.

The appropriate hole size depends on the size of the planar mirror. The light ray passing through the edge of the planar mirror Pm, and the focal point −F, are represented by the line equation, Eq. (11) using Eq. (4).

The diameter of the hole is determined by the solution of the hyperbolic function Eq. (1) and the line equation Eq. (12) as follows:

The diameter of the hole φ is given by 2xh.

Experiments were performed to evaluate the imaging system presented in this study. The experimental setup is shown in Fig. 8 where the parameters of the hyperbolic mirror are summarized in Table 1. The parameters are defined in Fig. 1.

Figure 8.Imaging system using a combined hyperbolic mirror and secondary planar mirror: (a) Front and rear view of combined hyperbolic mirror with a camera, (b) Experimental setup with the secondary planar mirror.

TABLE 1. Parameters of the combined hyperbolic mirror (mm)


According to the parameters, the size of the planar mirror in Eq. (9), the occlusion angle in Eq. (10), and the radius of the hole in Eq. (12) are given as follows when P is closely located around the apex of the hyperbolic mirror:

It is well-known that in a catadioptric imaging system with a hyperbolic mirror, the central imaging condition is of importance for image restoration to a natural image. In the experiment, the secondary planar mirror is placed at the position described by Eq. (4) by manual adjustment while taking into account the central imaging condition. It is possible to confirm the central imaging condition in Fig. 9(a); the parallel lines of the grid pattern in 3D space have circular images with common vanishing point on the image plane, and a line connecting the vanishing points passes through the optical center of the image [21]. On the contrary, the circular images of the parallel lines do not have a common vanishing point in the non-central image in Fig. 9(b).

Figure 9.Placement of the secondary planar mirror for central imaging: (a) Central image, (b) Non-central image.

Figure 10(a) shows another image acquired by the imaging system. The upper part and the lower parts of the image are restored to show a panoramic image in Fig. 10(b) and a bird’s eye view in Fig. 10(c), based on the image acquisition model in Sec. III.

Figure 10.Experimental result: (a) Original image, (b) Restored image: mercator image, (c) Restored image: bird’s eye view.

Wide FOV imaging systems are able to acquire rich visual information and have attracted increasing attention for many industrial applications such as video surveillance, autonomous navigation in mobile robots, and vehicle AVM, etc. Conventional catadioptric imaging systems with a curved mirror for wide FOV is hindered by the cumbersome camera deployment and the image occlusion. This study presented a new catadioptric imaging system consisting of a compact combination of a curved mirror and camera module, and a secondary planar mirror. In general, it is easier to install a secondary planar mirror in front of a curved mirror, compared to a bulky camera module. This study presents an approach for determining both the size of the planar mirror and the placement of the mirror to satisfy the central imaging condition. The image acquisition model was addressed for the catadioptric imaging system with the secondary planar mirror based on geometrical optics. The image acquisition model is a forward mapping between a scene point in 3D space and an image point on a 2D image plane. As a backward mapping, the acquired image is restored to a distortion-free image and the central imaging condition was verified in the experiments depending on the placement of the secondary planar mirror.

  1. S. K. Nayar, "Catadioptric omnidirectional camera," in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (USA, Jun. 1997), pp. 482-488.
  2. B. Micusik, and T. Pajdla, "Structure from motion with wide circular field of view cameras," IEEE Trans. Pattern Anal. Mach. Intell. 28, 1135-1149 (2006).
    CrossRef
  3. C.-C. Lin, and M.-S. Wang, "A vision based top-view transformation model for a vehicle parking assistant," Sensors (Basel) 12, 4431-4446 (2012).
    CrossRef
  4. B. Zhang, V. Appia, I. Pekkucuksen, Y. Liu, A. U. Batur, P. Shastry, S. Liu, S. Sivasankaran, and K. Chitnis, "A surround view camera solution for embedded systems," in Proc. IEEE Conference on Computer Vision and Pattern Recognition Workshops (USA, Jun. 2014), pp. 676-681.
  5. Y.-Y. Chen, Y.-Y. Tu, C.-H. Chiu, and Y.-S. Chen, "An embedded system for vehicle surrounding monitoring," in Proc. 2nd International Conference on Power Electronics and Intelligent Transportation System (PEITS) (China, Dec. 2009), pp. 92-95.
  6. J. Kannala, and S. S. Brandt, "A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses," IEEE Trans. Pattern Anal. Mach. Intell. 28, 1335-1340 (2006).
    CrossRef
  7. S. Shestha, A. Overvig, and N. Yu, "Broadband achromatic metasurface lenses," in Proc. Conference on Lasers and Electro-Optics, OSA Technical Digest, paper FM1H.3 (Optical Society of America 2017).
  8. T. E. Boult, and G. Wolberg, "Correcting chromatic aberrations using image warping," in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (USA, Jun. 1992), pp. 684-687.
  9. Y. Yagi, and S. Kawato, "Panorama scene analysis with conic projection," in Proc. IEEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications (Japan, Jul. 1990), pp. 181-187.
  10. R. Orghidan, J. Salvi, and E. M. Mouaddib, "Calibration of a structured light-based stereo catadioptric sensor," in Proc. Conference on Computer Vision and Pattern Recognition Workshop (USA, Jun. 2003), pp. 70.
  11. R. Orghidan, E. M. Mouaddib, and J. Salvi, "A computer vision sensor for panoramic depth perception," in Proc. 2nd Iberian Conference on Pattern Recognition and Image Analysis (Portugal, Jun. 2005), pp. 153-160.
  12. S. Yi, B. Choi, and N. Ahuja, "Real-time omni-directional distance measurement with active panoramic vision," Int. J. Control Autom. Syst. 5, 184-191 (2007).
  13. J. Gluckman, and S. K. Nayar, "Catadioptric stereo using planar mirrors," Int. J. Computer Vision 44, 65-79 (2001).
    CrossRef
  14. J. Gluckman, S. K. Nayar, and K. J. Thoresz, "Real-time omnidirectional and panoramic stereo," in Proc. DARPA Image Understanding Workshop (USA, Nov. 1998), pp. 299-303.
  15. G. Jang, S. Kim, and I. Kweon, "Single camera catadioptric stereo system," in Proc. of Workshop on Omnidirectional Vision, Camera Networks and Nonclassical cameras (OMNIVIS2005) (China, Oct. 2005), pp. 1-8.
  16. T. Svoboda, and T. Pajdlar, "Epipolar geometry for central catadioptric cameras," Int. J. Computer Vision 49, 23-37 (2002).
    CrossRef
  17. S. Yi, and N. Ahuja, "An omnidirectional stereo vision system using a single camera," in Proc. 18th International Conference on Pattern Recognition (ICPR’06) (China, Aug. 2006), pp. 861-865.
  18. S. Baker, and S. K. Nayar, "A theory of single-viewpoint catadioptric image formation," Int. J. Computer Vision 35, 175-196 (1999).
    CrossRef
  19. A. M. Bruckstein, and T. J. Richardson, "Omniview cameras with curved surface mirrors," in Proc. IEEE Workshop on Omnidirectional Vision, Cat. No.PR00704 (USA, Jun. 2000), pp. 79-84.
  20. S. Yi, and Y. Ko, "Wide field-of-view imaging using a combined hyperbolic mirror," Curr. Opt. Photon. 1, 336-343 (2017).
  21. C. Geyer, and K. Daniilidis, "A unifying theory for central panoramic systems and practical implications," in Proc. European Conference on Computer Vision (ECCV2000) (Ireland, Jun. 2000), pp. 445-461.

Article

Article

Current Optics and Photonics 2019; 3(4): 329-335

Published online August 25, 2019 https://doi.org/10.3807/COPP.2019.3.4.329

Copyright © Optical Society of Korea.

Compact Catadioptric Wide Imaging with Secondary Planar Mirror

Young-Jun Ko, and Soo-Yeong Yi*

Department of Electrical and Information Engineering, Seoul National University of Science and Technology, Seoul 01811, Korea

Correspondence to:suylee@seoultech.ac.kr

Received: March 2, 2019; Revised: May 28, 2019; Accepted: May 31, 2019

Abstract

Wide FOV imaging systems are important for acquiring rich visual information. A conventional catadioptric imaging system deploys a camera in front of a curved mirror to acquire a wide FOV image. This is a cumbersome setup and causes unnecessary occlusions in the acquired image. In order to reduce both the burden of the camera deployment and the occlusions in the images, this study uses a secondary planar mirror in the catadioptric imaging system. A compact design of the catadioptric imaging system and a condition for the position of the secondary planar mirror to satisfy the central imaging are presented. The image acquisition model of the catadioptric imaging system with a secondary planar mirror is discussed based on the principles of geometric optics in this study. As a backward mapping, the acquired image is restored to a distortion-free image in the experiments.

Keywords: Catadioptric system, Wide FOV image, Geometric optics, Hyperbolic mirror, Central imaging

I. INTRODUCTION

Wide FOV (Field-Of-View) imaging systems are able to acquire significantly more scene information compared to traditional imaging systems. Wdie FOV imaging systems have a wide range of applications which include surveillance, teleconferencing, and autonomous navigation in mobile robots, as well as the AVM (Around View Monitoring) technology which is adopted in the latest vehicle technology to improve safety during driving and parking [1-5]. There are two primary methods to acquire wide FOV images using a single camera: the first involves using a fisheye lens, and the other uses a curved mirror. The curved mirror approach is referred to as the catadioptric method. This type of imaging system is relatively simple and efficient to design, and the acquired images have lower chromatic aberration compared to the image acquired using a fisheye lens imaging system [6-8]. In [9], a catadioptric imaging system using a conical mirror was proposed to acquire 360° images, and to extract omnidirectional distances from the image for autonomous navigation of a mobile robot. An omnidirectional ranging sensor was developed using a catadioptric hyperbolic mirror with line lasers in [10-12]. A catadioptric imaging approach is applicable also to stereo image acquisition. In [13-16], omnidirectional stereo image acquisition methods were proposed using two cameras with hyperbolic mirrors, and a single camera with two hyperbolic mirrors. A similar single camera approach using a hyperbolic mirror with an additional concave lens was presented for omnidirectional stereo imaging in [17].

The catadioptric imaging method acquires a reflected image from a curved mirror using a conventional camera placed in front of the mirror. There are several kinds of bowl-shaped curved mirrors available to achieve this objective such as conic, hyperbolic, parabolic, and elliptic mirrors [18, 19]. In [20], a combined hyperbolic mirror was proposed that consists of a cylindrical hyperbolic upper part and an omnidirectional hyperbolic lower part.

Placing a camera at a specified distance in front of a curved mirror is cumbersome in practice because the camera module requires additional equipment and space. This causes unnecessary occlusions in the image. In order to solve this problem, a secondary planar mirror is used in this study; a small-sized planar mirror replaces the camera, and the camera is shifted to the rear concave side of a curved mirror. The camera acquires an image reflected from the primary curved mirror followed by the secondary planar mirror through a hole at the apex of the curved mirror. A compact imaging module containing a curved mirror and a camera is designed in this study to alleviate the cumbersome camera placement at the front side of the curved mirror. It is easier to deploy the secondary planar mirror instead of a camera module because of its smaller size and proximity to the primary curved mirror in general. The size of the secondary planar mirror depends on the position with respect to the primary curved mirror. This study addresses the position of the planar mirror in consideration of the central imaging condition of a curved mirror. The image acquisition model is also discussed based on principles of geometric optics and the image restoration algorithm.

The organization of this study is as follows: In Sec. II and Sec. III, the imaging model of the catadioptric system using a curved mirror with a secondary planar mirror is explained. Sec. IV addresses the size and the placement of the planar mirror to satisfy the central imaging condition. The experimental results and the concluding remarks are presented in Sec. V and Sec. VI.

II. IMAGING SYSTEM USING HYPERBOLIC MIRROR

In this study, the combined hyperbolic mirror in [20] is adopted as a curved mirror. The hyperbolic mirror is designed to achieve 213° horizontal and 126° vertical FOV and can be used for the vehicle AVM. The original design of the combined hyperbolic mirror is modified to compactly attach a camera at the rear concave side of the mirror, as shown in Fig. 1. Figure 2 shows the concept of the imaging system using the combined hyperbolic mirror with a secondary planar mirror as implemented in this study. The idea of this study is applicable to every kind of curved mirror without loss of generality.

Figure 1. Design of the combined hyperbolic mirror.
Figure 2. Imaging system using the combined hyperbolic mirror and secondary planar mirror.

The upper part of the curved mirror is described by a cylindrical hyperbolic function as

where a and b are the parameters of the hyperbolic function. The hyperbolic function has a focal point as follows:

The lower part of the curved mirror is described by a half of the omnidirectional hyperbolic function as

The upper and the lower parts are smoothly combined without any seam lines because the hyperbolic functions of the two parts have the same parameters.

III. IMAGE ACQUISITION MODEL

The image acquisition model represents the relationship between an object point, Po = [ xo yo zo]t in 3D space in 3D space and an image point Pi = [xi zi]t on a 2D image plane. The image acquisition model in this study is similar to that in [20] except the part of the secondary planar mirror. It is well-known that the planar mirror induces a virtual effective camera in its back. The image acquisition model is briefly explained in this section.

3.1. The Upper Part of the Mirror

Figure 3 shows the light reflection in the horizontal plane of the upper part of the imaging system. A light ray (Line I) from an object point, Po to the focal point, −F of the hyperbolic mirror, is reflected on the mirror surface and directed toward the symmetrical focal point, F as Line II. The light ray (Line II) is reflected in the secondary planar mirror at y = m a second time and produces an image on the camera sensor through a hole at the apex of the hyperbolic mirror. Imaging from a planar mirror is the same as imaging on an effective image plane through an effective pinhole as shown in Fig. 3. The positions of the effective pinhole and the effective image plane are symmetrical with those of the real camera with respect to the planar mirror. It is noted that the planar mirror should be placed at the middle of the actual pinhole position, P and the focal point of the symmetrical hyperbolic function, F as described by Eq. (4) in order to locate the effective pinhole at F for the central imaging condition.

Figure 3. Image acquisition model on horizontal plane.

The image acquisition model between Po and Pi is summarized as follows:

where λ denotes the focal length of the camera lens. In Eq. (5), xc and yc denote the reflection point of the incidence ray (Line I) and are obtained by a solution of Eq. (6).

The parameters, ti, tp, and to for zi in Eq. (5b) are given by

Please refer to [20] for detailed derivation of the image acquisition model.

3.2. The Lower Part of the Mirror

Figure 4 shows the cross-section in the xy(ry) plane of the lower part of the curved mirror. Because of the rotational symmetry about the y-axis, x is substituted with r for representing the cylindrical coordinates.

Figure 4. Image acquisition model for lower part of the mirror: (a) Cross-section in xy(ry) plane, (b) Front view.

The relationship between Po and Pi is represented by

where ro is as shown in Fig. 4(b). Because Fig. 4 is the same as Fig. 3, ri is obtained from Eqs. (5a) and (6) by changing x with r.

f5

IV. THE SECONDARY PLANAR MIRROR

This section addresses the position and the size of the secondary mirror, which has an influence on the central imaging condition and on the occluded area of the resultant image in the proposed catadioptric imaging system.

4.1. Size and Position of the Secondary Planar Mirror

The effective pinhole is the symmetric transformation of P about the secondary planar mirror and should be located at F for the central imaging condition of the hyperbolic mirror; this provides the condition for the position of the planar mirror as described in Eq. (4). The size of the planar mirror is determined according to its position, as depicted in Fig. 5. From the simple proposition, it is easy to determine the right size of the planar mirror, M, as Eq. (9).

Figure 5. Size and position of secondary planar mirror (top view).

The size of the mirror M depends on the pinhole position of the camera, P. If the size of the mirror is less than M, the planar mirror does not cover the entire reflected field of light from the curved mirror. On the contrary, if the size is bigger than M, the occluded area by the planar mirror becomes too large as shown in Fig. 6.

In Fig. 6, the occlusion angle is obtained as follows:

Figure 6. Occlusion due to secondary planar mirror.

In order to reduce this angle ρ, the size of the planar mirror M, should be small enough, which requires that the pinhole position of the camera P, should be as close as possible to the apex of the hyperbolic mirror in the backside.

4.2. Hole Size at the Apex of the Primary Curved Mirror

Figure 7 shows the hole size of the hyperbolic mirror.

Figure 7. Hole size of primary curved mirror.

The appropriate hole size depends on the size of the planar mirror. The light ray passing through the edge of the planar mirror Pm, and the focal point −F, are represented by the line equation, Eq. (11) using Eq. (4).

The diameter of the hole is determined by the solution of the hyperbolic function Eq. (1) and the line equation Eq. (12) as follows:

The diameter of the hole φ is given by 2xh.

V. EXPERIMENTAL RESULTS

Experiments were performed to evaluate the imaging system presented in this study. The experimental setup is shown in Fig. 8 where the parameters of the hyperbolic mirror are summarized in Table 1. The parameters are defined in Fig. 1.

Figure 8. Imaging system using a combined hyperbolic mirror and secondary planar mirror: (a) Front and rear view of combined hyperbolic mirror with a camera, (b) Experimental setup with the secondary planar mirror.

TABLE 1.. Parameters of the combined hyperbolic mirror (mm).


According to the parameters, the size of the planar mirror in Eq. (9), the occlusion angle in Eq. (10), and the radius of the hole in Eq. (12) are given as follows when P is closely located around the apex of the hyperbolic mirror:

It is well-known that in a catadioptric imaging system with a hyperbolic mirror, the central imaging condition is of importance for image restoration to a natural image. In the experiment, the secondary planar mirror is placed at the position described by Eq. (4) by manual adjustment while taking into account the central imaging condition. It is possible to confirm the central imaging condition in Fig. 9(a); the parallel lines of the grid pattern in 3D space have circular images with common vanishing point on the image plane, and a line connecting the vanishing points passes through the optical center of the image [21]. On the contrary, the circular images of the parallel lines do not have a common vanishing point in the non-central image in Fig. 9(b).

Figure 9. Placement of the secondary planar mirror for central imaging: (a) Central image, (b) Non-central image.

Figure 10(a) shows another image acquired by the imaging system. The upper part and the lower parts of the image are restored to show a panoramic image in Fig. 10(b) and a bird’s eye view in Fig. 10(c), based on the image acquisition model in Sec. III.

Figure 10. Experimental result: (a) Original image, (b) Restored image: mercator image, (c) Restored image: bird’s eye view.

VI. CONCLUSION

Wide FOV imaging systems are able to acquire rich visual information and have attracted increasing attention for many industrial applications such as video surveillance, autonomous navigation in mobile robots, and vehicle AVM, etc. Conventional catadioptric imaging systems with a curved mirror for wide FOV is hindered by the cumbersome camera deployment and the image occlusion. This study presented a new catadioptric imaging system consisting of a compact combination of a curved mirror and camera module, and a secondary planar mirror. In general, it is easier to install a secondary planar mirror in front of a curved mirror, compared to a bulky camera module. This study presents an approach for determining both the size of the planar mirror and the placement of the mirror to satisfy the central imaging condition. The image acquisition model was addressed for the catadioptric imaging system with the secondary planar mirror based on geometrical optics. The image acquisition model is a forward mapping between a scene point in 3D space and an image point on a 2D image plane. As a backward mapping, the acquired image is restored to a distortion-free image and the central imaging condition was verified in the experiments depending on the placement of the secondary planar mirror.

Fig 1.

Figure 1.Design of the combined hyperbolic mirror.
Current Optics and Photonics 2019; 3: 329-335https://doi.org/10.3807/COPP.2019.3.4.329

Fig 2.

Figure 2.Imaging system using the combined hyperbolic mirror and secondary planar mirror.
Current Optics and Photonics 2019; 3: 329-335https://doi.org/10.3807/COPP.2019.3.4.329

Fig 3.

Figure 3.Image acquisition model on horizontal plane.
Current Optics and Photonics 2019; 3: 329-335https://doi.org/10.3807/COPP.2019.3.4.329

Fig 4.

Figure 4.Image acquisition model for lower part of the mirror: (a) Cross-section in xy(ry) plane, (b) Front view.
Current Optics and Photonics 2019; 3: 329-335https://doi.org/10.3807/COPP.2019.3.4.329

Fig 5.

Figure 5.Size and position of secondary planar mirror (top view).
Current Optics and Photonics 2019; 3: 329-335https://doi.org/10.3807/COPP.2019.3.4.329

Fig 6.

Figure 6.Occlusion due to secondary planar mirror.
Current Optics and Photonics 2019; 3: 329-335https://doi.org/10.3807/COPP.2019.3.4.329

Fig 7.

Figure 7.Hole size of primary curved mirror.
Current Optics and Photonics 2019; 3: 329-335https://doi.org/10.3807/COPP.2019.3.4.329

Fig 8.

Figure 8.Imaging system using a combined hyperbolic mirror and secondary planar mirror: (a) Front and rear view of combined hyperbolic mirror with a camera, (b) Experimental setup with the secondary planar mirror.
Current Optics and Photonics 2019; 3: 329-335https://doi.org/10.3807/COPP.2019.3.4.329

Fig 9.

Figure 9.Placement of the secondary planar mirror for central imaging: (a) Central image, (b) Non-central image.
Current Optics and Photonics 2019; 3: 329-335https://doi.org/10.3807/COPP.2019.3.4.329

Fig 10.

Figure 10.Experimental result: (a) Original image, (b) Restored image: mercator image, (c) Restored image: bird’s eye view.
Current Optics and Photonics 2019; 3: 329-335https://doi.org/10.3807/COPP.2019.3.4.329

TABLE 1. Parameters of the combined hyperbolic mirror (mm)


References

  1. S. K. Nayar, "Catadioptric omnidirectional camera," in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (USA, Jun. 1997), pp. 482-488.
  2. B. Micusik, and T. Pajdla, "Structure from motion with wide circular field of view cameras," IEEE Trans. Pattern Anal. Mach. Intell. 28, 1135-1149 (2006).
    CrossRef
  3. C.-C. Lin, and M.-S. Wang, "A vision based top-view transformation model for a vehicle parking assistant," Sensors (Basel) 12, 4431-4446 (2012).
    CrossRef
  4. B. Zhang, V. Appia, I. Pekkucuksen, Y. Liu, A. U. Batur, P. Shastry, S. Liu, S. Sivasankaran, and K. Chitnis, "A surround view camera solution for embedded systems," in Proc. IEEE Conference on Computer Vision and Pattern Recognition Workshops (USA, Jun. 2014), pp. 676-681.
  5. Y.-Y. Chen, Y.-Y. Tu, C.-H. Chiu, and Y.-S. Chen, "An embedded system for vehicle surrounding monitoring," in Proc. 2nd International Conference on Power Electronics and Intelligent Transportation System (PEITS) (China, Dec. 2009), pp. 92-95.
  6. J. Kannala, and S. S. Brandt, "A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses," IEEE Trans. Pattern Anal. Mach. Intell. 28, 1335-1340 (2006).
    CrossRef
  7. S. Shestha, A. Overvig, and N. Yu, "Broadband achromatic metasurface lenses," in Proc. Conference on Lasers and Electro-Optics, OSA Technical Digest, paper FM1H.3 (Optical Society of America 2017).
  8. T. E. Boult, and G. Wolberg, "Correcting chromatic aberrations using image warping," in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (USA, Jun. 1992), pp. 684-687.
  9. Y. Yagi, and S. Kawato, "Panorama scene analysis with conic projection," in Proc. IEEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications (Japan, Jul. 1990), pp. 181-187.
  10. R. Orghidan, J. Salvi, and E. M. Mouaddib, "Calibration of a structured light-based stereo catadioptric sensor," in Proc. Conference on Computer Vision and Pattern Recognition Workshop (USA, Jun. 2003), pp. 70.
  11. R. Orghidan, E. M. Mouaddib, and J. Salvi, "A computer vision sensor for panoramic depth perception," in Proc. 2nd Iberian Conference on Pattern Recognition and Image Analysis (Portugal, Jun. 2005), pp. 153-160.
  12. S. Yi, B. Choi, and N. Ahuja, "Real-time omni-directional distance measurement with active panoramic vision," Int. J. Control Autom. Syst. 5, 184-191 (2007).
  13. J. Gluckman, and S. K. Nayar, "Catadioptric stereo using planar mirrors," Int. J. Computer Vision 44, 65-79 (2001).
    CrossRef
  14. J. Gluckman, S. K. Nayar, and K. J. Thoresz, "Real-time omnidirectional and panoramic stereo," in Proc. DARPA Image Understanding Workshop (USA, Nov. 1998), pp. 299-303.
  15. G. Jang, S. Kim, and I. Kweon, "Single camera catadioptric stereo system," in Proc. of Workshop on Omnidirectional Vision, Camera Networks and Nonclassical cameras (OMNIVIS2005) (China, Oct. 2005), pp. 1-8.
  16. T. Svoboda, and T. Pajdlar, "Epipolar geometry for central catadioptric cameras," Int. J. Computer Vision 49, 23-37 (2002).
    CrossRef
  17. S. Yi, and N. Ahuja, "An omnidirectional stereo vision system using a single camera," in Proc. 18th International Conference on Pattern Recognition (ICPR’06) (China, Aug. 2006), pp. 861-865.
  18. S. Baker, and S. K. Nayar, "A theory of single-viewpoint catadioptric image formation," Int. J. Computer Vision 35, 175-196 (1999).
    CrossRef
  19. A. M. Bruckstein, and T. J. Richardson, "Omniview cameras with curved surface mirrors," in Proc. IEEE Workshop on Omnidirectional Vision, Cat. No.PR00704 (USA, Jun. 2000), pp. 79-84.
  20. S. Yi, and Y. Ko, "Wide field-of-view imaging using a combined hyperbolic mirror," Curr. Opt. Photon. 1, 336-343 (2017).
  21. C. Geyer, and K. Daniilidis, "A unifying theory for central panoramic systems and practical implications," in Proc. European Conference on Computer Vision (ECCV2000) (Ireland, Jun. 2000), pp. 445-461.