G-0K8J8ZR168
검색
검색 팝업 닫기

Ex) Article Title, Author, Keywords

## Article

Curr. Opt. Photon. 2021; 5(6): 606-616

Published online December 25, 2021 https://doi.org/10.3807/COPP.2021.5.6.606

## Spatial Frequency Coverage and Image Reconstruction for Photonic Integrated Interferometric Imaging System

Wang Zhang , Hongliu Ma, Kang Huang

School of Mechanical and Aerospace Engineering, Jilin University, Changchun 130025, China

Corresponding author: *wangzhang@jlu.edu.cn, ORCID 0000-0001-9029-1320

Received: June 25, 2021; Revised: September 2, 2021; Accepted: September 7, 2021

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

A photonic integrated interferometric imaging system possesses the characteristics of small-scale, low weight, low power consumption, and better image quality. It has potential application for replacing conventional large space telescopes. In this paper, the principle of photonic integrated interferometric imaging is investigated. A novel lenslet array arrangement and lenslet pairing approach are proposed, which are helpful in improving spatial frequency coverage. For the novel lenslet array arrangement, two short interference arms were evenly distributed between two adjacent long interference arms. Each lens let in the array would be paired twice through the novel lenslet pairing approach. Moreover, the image reconstruction model for optical interferometric imaging based on compressed sensing was established. Image simulation results show that the peak signal to noise ratio (PSNR) of the reconstructed image based on compressive sensing is about 10 dB higher than that of the direct restored image. Meanwhile, the normalized mean square error (NMSE) of the direct restored image is approximately 0.38 higher than that of the reconstructed image. Structural similarity index measure (SSIM) of the reconstructed image based on compressed sensing is about 0.33 higher than that of the direct restored image. The increased spatial frequency coverage and image reconstruction approach jointly contribute to better image quality of the photonic integrated interferometric imaging system.

Keywords: Compressive sensing, Image reconstruction, Optical imaging, Optical interferometry, Photonic integrated circuits

OCIS codes: (040.5160) Photodetectors; (110.3010) Image reconstruction techniques; (110.3175) Interferometric imaging

### I. INTRODUCTION

In order to improve spatial resolution, conventional optical telescopes were usually designed with large apertures, leading to large volume, heavy weight and high power consumption [1, 2]. With the rapid development of photonic integrated circuit (PIC) technology and interferometric imaging technology, the concept known as the segmented planar imaging detector for electro-optical reconnaissance (SPIDER) has been proposed, which is of great importance for its potential applications in astronomy and space science [3]. For photonic integrated interferometric imaging (PIII), the large optical components, devices and mounting structures in conventional system are replaced by a densely packed interferometer array realized in an extremely thin layer of lenslets and PIC components, which offers substantial reductions in system size, weight, and cost. Using on-chip components, the high quality image is reconstructed by collecting and processing the interferometric information of Fourier domain at the output end [48]. At present, PIII technology is at the primary research stage and has not been transformed into production. Relevant research institutions only carried out some demonstrative experiments and theoretical investigations [9]. Key technologies concerning PIII need to be addressed, which include the principle of optical interferometric imaging and signal transmission mechanism, spatial frequency coverage and image reconstruction of the scene.

This paper focuses on spatial frequency coverage and image reconstruction algorithm of PIII. The former aspect determines the direct restored image, and the latter one influences the reconstructed image quality. Section 2 discusses the basic theory and concept of PIII. Section 3 describes a novel lenslet array arrangement and lenslet pairing approach that effectively improve spatial frequency coverage of the scene. Meanwhile, it introduces an image reconstruction approach based on compressive sensing (CS). Section 4 shows image reconstruction from a simulated PIII observation. Section 5 has the discussion. The conclusion is given in Section 6.

### II. THEORY AND CONCEPT

PIII system is to put a linear array of lenslets onto a PIC card, and the PIC cards are mounted as radial spokes on a disc. PIII system consists of three working modules: signal receiving module, signal transmission module and signal processing module. The signal receiving module mainly includes the lenslet array and its components. The signal transmission module consists of optical waveguides, arrayed waveguide gratings (AWGs), phase shifters, and multi-mode interferometers (MMIs). The signal processing module contains balanced detectors and digital signal processing. PICs transform the target intensity distribution into Fourier domain information. By scanning the phase delay on one of the arms in the MMI, the balanced detectors receive interferometric information, the phase and the amplitude of the complex visibility will be calculated. Then the image is restored by processing Fourier transform of the complex visibility information [10, 11]. Figure 1 shows the traditional structure of system and the working principle of a PIC.

Figure 1.The traditional structure of system and the working principle of a photonic integrated circuit (PIC) [12,13].

PIII is a new implementation approach of optical interferometric technology. According to the Van Cittert–Zernike theory [14], the measured complex visibility y between the paired lenslets (x1, y1) and (x2, y2) can be expressed as

y(u,v)= x(ξ,η)ρ ξ,ηexp j2π(uξ+vη)dξdη,

where ρ is the lenslet coupling efficiency projected onto the object plane. (ξ, η) is the coordinate of image intensity x. (u, v) is the spatial frequency corresponding to the distance vector between two lenslets (x1, y1) and (x2, y2):

u=Δxλz=x2x1λz,

u=Δyλz=y2y1λz,

where Δx, Δy are the distances of the chosen pair of lenslets in the x and y directions. λ is the current working wavelength of the system, and z is the scene distance.

For the PIII system, the paired beam passing through the phase shifters would interfere in MMI, and the interferometric information would be measured by the balanced detectors. As shown in Fig. 1, the optical signals input into the 90° optical hybrids can be expressed as ES and ER respectively, and the output signals of the 90° optical hybrids can be expressed as [15]

Eout1=12ES+ER,

Eout2=12ESER,

Eout3=12ES+jER,

Eout4=12ESjER.

The output electrical signals I and Q of balanced detectors can be derived as follows:

I=I1I2=Eout1Eout1*Eout2Eout2*=Re(ESER*),

Q=Q1Q2=Eout3Eout3*Eout4Eout4*=Im(ESER*),

where E* is the conjugate of E. From equations above, the phase φ and amplitude E of the measured complex visibility can be expressed as

E=I2+Q212,

φ=arctanQ1.

The measured complex visibility y can be obtained by using the output information from the system as follows:

y(u,v)=I+jQ=Eexp(jφ).

According to the Van Cittert-Zernike theorem, the complex visibility can be found by a two-dimensional Fourier transformation of the intensity distribution across the source [14]. Therefore, the restored image can be expressed as

x(ξ,η)=F1yu,v.

### 3.1. Structure and Spatial Frequency Coverage

We designed a novel arrangement of lenslet array as shown in Fig. 2. This lenslet array arrangement is composed of thirty-seven long interference arms. Between two adjacent long interference arms, pairs of short interference arms are evenly distributed. Each long interference arm consists of twenty-nine lenslets (red color) and short interference arm contains fifteen lenslets (yellow color). The interval between two adjacent lenslets is equal to the diameter d of the lenslet. Considering the system is composed of P1 long interference arms, the angle between adjacent long interference arms is α1 = 2π / P1. Considering the system is composed of P2 short interference arms, the angle between adjacent short interference arms is α2 = 2π/(P1 + P2).

Figure 2.The schematic diagram of the new system: (a) the new structure of system and (b) the novel arrangement of lenslet array.

The spatial frequency coverage is influenced by the lenslet array arrangement and pairing approach of the lenslets in each radial spoke. As shown in Fig. 3, 1 × 2 beam splitters are designed to follow each lenslet except the middle and last lenslets. Two 1 × 3 beam splitters are designed to follow the middle and last lenslets respectively. Erbium-doped fibre amplifiers (EDFAs) are designed to amplify the signal intensity output from each 1 × 3 beam splitter to maintain equal signal intensity corresponding to the 1 × 2 beam splitter. The PIII system takes advantage of AWGs to divide broadband light into multiple narrow spectral channels to improve spatial frequency coverage. The paired narrow spectral beam passing through the phase shifters would be interfered in MMI, and the interferometric information would be measured by the balanced detectors. By setting up 1 × 2 and 1 × 3 beam splitters following the lenslets, all the lenslets can be paired twice in each radial spoke. For example, assuming each radial spoke contains 5 lenslets, the pairing approach is (1, 5), (2, 4), (3, 3), (1, 4), (2, 3), (5, 5).

Figure 3.The schematic diagram of the optimized photonic integrated device. (a) The working principle of a PIC, (b) the 1 × 2 beam splitter, (c) the 1 × 3 beam splitter, and (d) 5 lenslets pairing.

Based on setting up 1 × 2 and 1 × 3 beam splitters following the lenslets in Fig. 3, the novel pairing details of lenslets can be described as follows. Assuming each interference arm contains N lenslets, the lenslets were paired firstly following (1, N − 1), (2, N − 2), (3, N − 3)... ((N − 1) / 2, (N + 1) / 2), (N, N) as shown in Fig. 4(a). After that, the lenslets were secondly matched as (1, N), (2, N − 1), (3, N − 2)... ((N − 1) / 2, (N + 3) / 2), ((N + 1) / 2, (N + 1) / 2), as shown in Fig. 4(b). Zero spatial frequency sampling with the image average brightness information is obtained by pairing lenslets N and (N + 1) / 2 with itself [16]. Through this novel lenslet pairing approach, the length of the baseline is 0, d, 2d, 3d, …, (N − 1)d. The spatial frequency coverage is approximately twice than that of traditional lenslet pairing [17]. Therefore, this novel lenslet pairing approach can effectively improve the spatial frequency coverage of the scene. Consequently, low frequency and high frequency information of the scene can be achieved simultaneously, which is beneficial for the image restoration of both outline and detail of the scene.

Figure 4.The Schematic diagram of the novel lenslet pairing method: (a) the first pairing and (b) the second pairing.

### 3.2. Compressive Sensing Theory

CS is a novel signal sampling theory based on the sparsity and compression of an observed signal. CS is a robust framework for signal reconstruction. The theoretical framework of CS motivates sparse regularisation for solving inverse problems, through which, the quality of image reconstruction can be improved effectively [18].

### 3.2.1. Measurement Model

According to Van Cittert-Zernike theorem, the measurement equation takes the form of a Fourier transform. By inverse Fourier transform of the complex visibility measured by the system, intensity distribution of the scene will be achieved. Due to the imaging character of the PIII system, it is not possible to sample the entire (u, v) domain, leading to an ill-posed inverse problem of reconstructing image x from measured values y. The measurement equation for the PIII system can be represented as

y=Ax+n,

where x ϵ RT×1 is intensity of image. T is the total number of image pixels. y ϵ CM×1 is the measured complex visibility. M represents the number of measurements. A ϵ CM×T is the linear transformation matrix from the spatial domain to the frequency domain. n represents independently and identically distributed noise. Noise is not considered in this manuscript. A is the linear transformation matrix from the spatial domain to the frequency domain:

A=SF,

where S ϵ RM×T is the binary matrix for spatial frequency sampling of the object. F ϵ CT×T is the discrete Fourier transform. Then, we have the following linear model:

y=SFx+n,

where y can be calculated from the equation (12). S ϵ RM×T can be obtained through the spatial frequency coverage of the scene. F ϵ CT×T can be gained through the discrete Fourier transform basis.

### 3.2.2. Reconstruction Method

Fourier frequencies are not measured adequately through the PIII system, fitting the data alone does not uniquely define the image sought. Therefore, the image is reconstructed by using the total variation augmented Lagrangian alternating direction algorithm. The basic framework for image reconstruction is shown in Fig. 5.

Figure 5.The basic framework for image reconstruction.

Sparse regularization contributes to sparse solutions in terms of solving ill-posed inverse problems. To be specific, the ill-posed inverse problem can be solved by sparse regularization through imposing a priori constraints to select a unique image among all those that are consistent with the data. Moreover, this inverse problem of reconstructing image x from measured values y can be solved by building a total variation regularization mode, which can be expressed as

minx i D ixsubjecttoAx=y,

where Dix is gradient of x at pixel i. The slack variable wi is introduced. The model of image reconstruction problem can be expressed as

minwi,x i wisubjecttoAx=y,Dix=wiforalli.

The constrained model (18) can be converted to an unconstrained objective function by the augmented Lagrangian method, which can be expressed as

l˜A(wi,x)= iwiviTDixwi+βi2 Dixwi22kT(Axy)+μ2 Axy22,

where νi, κ are Lagrangian multipliers. βi, μ are penalty parameters. As shown in Fig. 5, the image reconstruction problem can be decomposed into two simple sub-problems by the alternating direction algorithm, the sub-problem wi and the sub-problem x. Through the iterative method, the sub-problem wi is solved firstly, then the sub-problem x is done. The sub-problem is to minimize the augmented Lagrangian function, which can be represented as

minwi,xl˜Awi,x= i wiviT Dixwi+ βi2 D ix w i22kT(Axy)+μ2 Axy22.

Supposing xk denotes the minimizer of the sub-problem x at the k-th iteration, wik+1 can be obtained by minimizing the objective function with respect to wi on each edge as follows:

minwi i wiviT Dixkwi+ βi2 Dixkwi22.

Having wik+1 at hand, xk+1 can be achieved by minimizing the objective function with respect to x as follows:

minxl˜Awik+1,x= i wik+1viT Dixwik+1+ βi2 Dixwik+122kT(Axy)+μ2 Axy22.

Lagrangian multipliers are updated through the well known formulas:

vik+1=vikβik(Dixk+1wik+1),

κk+1=κkμk(Axk+1y).

New penalty parameters μ and βi are chosen. The most effective way to choose new penalty parameters is to try different values and compare the corresponding image quality under different parameter values as shown in Fig. 6. The selected penalty parameters have better image quality.

Figure 6.Image quality under different parameter values: (a) image quality under different μ parameter values and (b) image quality under different βi parameter values.

Variables wi and x are solved alternatively through updating Lagrangian multipliers and choosing penalty parameters to ensure that the constraint condition is met in the final solution [19].

### IV. RESULTS

The PIII system parameters used for the simulations are listed in Table 1. The sampling pattern in the uv-plane is shown in Fig. 7. Since the acquisition of spatial frequency information is an under-sampling process, this results in the loss of the partial image content. Therefore, the PSNR, the NMSE and SSIM are used as the evaluation parameters of image reconstruction. For a test image Y and its original image X, the PSNR, NMSE and SSIM metrics are defined as [20]

TABLE 1 System parameters used for the simulations

ParameterSymbol of ParametersParameter Values
Spectral Coverageλ380–760 nm
Lenslet Diameterd1 mm
Longest BaselineBmax28 mm
Number of Long PIC SpokesP137
Number of Lenslets per Long PIC SpokesN129
Number of Short PIC SpokesP274
Number of Lenslets per Short PIC SpokesN215
Scene Distancez100 km
Number of Spectral Binsm10

Figure 7.The sampling pattern of PIII system in the uv-plane.

PSNR=10log10MAX2MSE,

NMSE=i=1Wj=1H X i,jY i,j2i=1Wj=1H X i,j2,

SSIM=(2μXμY+C1)(2σXY+C2)(μX2+μY2+C1)(σX2+σY2+C2),

where MAX is the maximum possible pixel value of the image. MSE is the mean square error of the image. μX and μY are the averages of X and Y. σ2X and σ2Y are the variances of X and Y. σXY is the covariance of X and Y. C1 and C2 are two variables used to stabilize division involving a weak denominator. The larger PSNR and SSIM are, the better image quality is acquired. On the contrary, the smaller NMSE is, the better image quality is achieved.

Traditional lenslet array arrangement (as shown in Fig. 1) consists of radial spokes with equal length on a disc. The traditional lenslet pairing follows symmetrical scheme: (1, N), (2, N−1), (3, N−2)…(N/2, N/2+1). For comparison, the imaging simulation experiments were carried out for the PIII system in traditional and novel lenslet array arrangements and lenslet pairing method. All the images acquired through different approaches are shown in Figs. 810. The graphs of spatial frequency coverage are shown in Fig. 11. The calculated PSNR, NMSE and SSIM are listed in Table 2.

TABLE 2 The calculated PSNR, NMSE and SSIM

The ImagePSNR(/dB)NMSESSIM
USAF1951(b)10.370.450.47
USAF1951(c)10.810.410.53
USAF1951(d)19.640.050.96
USAF1951(e)24.090.010.98
The Saturn(b)24.530.400.67
The Saturn(c)24.670.390.68
The Saturn(d)33.450.050.96
The Saturn(e)34.420.040.97
The Nebula(b)30.550.500.68
The Nebula(c)30.740.480.69
The Nebula(d)37.840.090.94
The Nebula(e)38.060.080.95

Figure 8.USAF1951: (a) the original image, (b) direct restored image with traditional lenslet array arrangement and symmetrical lenslet pairing method, (c) direct restored image with novel lenslet array arrangement and lenslet pairing method presented in this paper, (d) compressive sensing (CS) reconstruction image with traditional lenslet array arrangement and symmetrical lenslet pairing method, and (e) CS reconstruction image with novel lenslet array arrangement and lenslet pairing method.

Figure 9.The Saturn: (a) the original image, (b) direct restored image with traditional lenslet array arrangement and symmetrical lenslet pairing method, (c) direct restored image with novel lenslet array arrangement and lenslet pairing method presented in this paper, (d) compressive sensing (CS) reconstruction image with traditional lenslet array arrangement and symmetrical lenslet pairing method, and (e) CS reconstruction image with novel lenslet array arrangement and lenslet pairing method.

Figure 10.The Nebula: (a) the original image, (b) direct restored image with traditional lenslet array arrangement and symmetrical lenslet pairing method, (c) direct restored image with novel lenslet array arrangement and lenslet pairing method presented in this paper, (d) compressive sensing (CS) reconstruction image with traditional lenslet array arrangement and symmetrical lenslet pairing method, and (e) CS reconstruction image with novel lenslet array arrangement and lenslet pairing method.

Figure 11.The graphs of spatial frequency coverage: (a) the graph of spatial frequency coverage constructed by a conventional system, (b) the graph of spatial coverage constructed by our system, and (c) the graph of increased spatial frequency coverage.

As listed in Table 2, the PSNR of direct restored image using the novel lenslet array arrangement and pairing method increases approximately 0.25 dB than that of the direct restored image using the traditional lenslet array arrangement and symmetrical pairing method. The NMSE of the direct restored image using the novel lenslet array arrangement and pairing method proposed by this paper decreases approximately 0.02 more than that of the direct restored image using the traditional lenslet array arrangement and pairing method. The SSIM of the direct restored image using the novel lenslet array arrangement and pairing method increases approximately 0.02 than that of direct restored image by using traditional lenslet array arrangement and symmetrical pairing method.

Image reconstruction based on CS theory is carried out and uses the novel lenslet array arrangement and pairing method. The PSNR of CS reconstructed image increases by approximately 10 dB from that of the direct restored image. Meanwhile, the NMSE calculated from the direct restored image is approximately 0.38 higher than that of the reconstructed image based on CS. The SSIM of the CS reconstructed image increases approximately 0.33 than that of the direct restored image. The PSNR of the CS reconstructed image using the novel lenslet array arrangement and pairing method increases approximately 1.88 dB from that of CS reconstructed image using the traditional lenslet array arrangement and symmetrical pairing method. Meanwhile, the NMSE calculated from the CS reconstructed image using traditional lenslet array arrangement and symmetrical pairing method increases approximately 0.02 than that of CS reconstructed image using the novel lenslet array arrangement and the pairing method. The SSIM of the CS reconstructed image using the novel lenslet array arrangement and pairing method increases approximately 0.01 than that of the CS reconstructed image using traditional lenslet array arrangement and symmetrical pairing method.

Figures 810 are the images acquired through different approaches. According to Figs. 810, the image restoration and reconstruction simulation results show that the novel lenslet array arrangement and pairing method presented in this paper could effectively improve the spatial frequency coverage and have influence on direct restored image quality. Meanwhile, image reconstruction based on CS theory could improve the image quality of interferometric imaging.

The novel lenslet array arrangement adds lenslet arrays in more directions, which are different from the directions of the existing radial lenslets spokes. The novel lenslet pairing approach adds more spatial frequency sampling points. As shown in Fig. 11, the traditional lenslet array arrangement and lenslet pairing method can collect interferometric information in 37 directions (radial spokes) and obtains 5180 measured points, while this novel design can collect interferometric information in 111 directions and obtains 20720 measured points. The obtained spatial frequency points based on the novel lenslet array arrangement and lenslet pairing approach are approximately four times than that of traditional design. Therefore, this novel design improves spatial frequency coverage of the system. The novel lenslet array arrangement and lenslet pairing method designed in this paper can improve the image quality because spatial frequency coverage has the direct influence on image quality.

### V. DISCUSSION

This paper proposed the novel approach for improving spatial frequency coverage and image reconstruction. As shown in Fig. 1, the traditional lenslet array arrangement is composed of radial spokes with equal length on a disc. However, the novel lenslet array arrangement consists of different length spokes on a disc, as shown in Fig. 2. Compared with the traditional lenslet array arrangement, the obtained spatial frequency points based on the novel lenslet array arrangement are approximately twice than that of traditional design, which can effectively improve the spatial frequency coverage of the scene.

The traditional lenslet pairing follows symmetrical scheme: (1, N), (2, N−1), (3, N−2)…(N/2, N/2+1), the length of the baseline obtained through the traditional lenslet pairing approach is d, 3d, 5d, …, (N−1)d. Nevertheless, the length of the baseline obtained by the novel lenslet pairing approach is 0, d, 2d, 3d, …, (N−1)d, and the number of the baseline is approximately twice than that of traditional lenslet pairing. Therefore, the novel lenslet pairing approach can acquire the more spatial frequency information, which efficiently improve the image quality of the PIII system.

The traditional reconstruction algorithms mainly include CLEAN algorithm and maximum entropy method (MEM) algorithm. CLEAN algorithm can be seen as a greedy pursuit algorithm with an over-complete dictionary. Therefore, CLEAN algorithm seems unstable. The core idea of MEM algorithm is to obtain the least informative image that is consistent with the data [21, 22]. CS algorithm is applied to the PIII system in this paper, and it has significant impacts in other applications such as medical imaging, synthetic aperture radar imaging, remote sensing imaging, face recognition, speech recognition. Compared with traditional imaging methods, CS provides a framework for recovering image from the fewer measurements and has great advantages in flexibility [23, 24].

### VI. CONCLUSION

In this paper, a spatial frequency coverage method and image reconstruction algorithm for the PIII system were proposed. The novel lenslet array arrangement and lenslet pairing method positively influence the spatial frequency coverage. The acquired dense spatial frequencies efficiently improve the quality of the direct restored image. Based on CS theory, the image reconstruction model for the PIII system was established. By using this image reconstruction model, interferometric image with better quality could be acquired. The research content of this paper can provide technological support for optical interferometric imaging.

Although the image quality of the PIII system is improved based on the research content of this paper, there is still potential space for improving the quality of the image. In the future, the arrangement of lenslet array and the approach of lenslet pairing, which can increase the spatial frequency coverage, will still continue to be researched.

### References

1. R. Athale, D. M. Healy, D. J. Brady and M. A. Neifeld, “Reinventing the camera,” Opt. Photonics News 19, 32-37 (2008).
2. J. E. Baldwin, M. G. Beckett, R. C. Boysen, D. Burns and T. R. Scott, “The first images from an optical aperture synthesis array: mapping of Capella with COAST at two epochs,” Astron. Astrophys. 306, L13-L16 (1996).
3. R. P. Scott, T. H. Su, C. Ogden, S. T. Thurman, R. L. Kendrick, A. Duncan, R. X. Yu and S. J. B. Yoo, “Demonstration of a photonic integrated circuit for multi-baseline interferometric imaging,,” in in Proc. IEEE Photonics Conference, (San Diego, CA, USA, 2015).
4. Q. H. Chu, Y. J. Shen, M. Yuan and M. L. Gong, “Numerical simulation and optimal design of segmented planar imaging detector for electro-optical reconnaissance,” Opt. Commun. 405, 288-296 (2017).
5. G. Liu, T. Su, S. Li, J. Chun, W. Lai, M. Prost, C. Ogden, S. T. Thurman, R. L. Kendrick, A. Duncan and S. J. B. Yoo, “Si3N4 photonic integrated circuit for multi-baseline interferometric imaging,,” in in Proc. IEEE Photonics Conference-IPC, (Orlando, FL, USA, 2017). pp. 51-52.
6. W. P. Gao, X. R. Wang, L. Ma, Y. Yuan and D. F. Guo, “Quantitative analysis of segmented planar imaging quality based on hierarchical multistage sampling lens array,” Opt. Express 27, 7955-7967 (2019).
7. R. L. Kendrick, A. Duncan, C. Ogden, J. Wilm and S. T. Thurman, “Segmented planar imaging detector for EO reconnaissance,” in Imaging and Applied Optics, paper CM4C.1 (2013).
8. S. T. Thurman, R. L. Kendrick, A. Duncan, D. Wuchenich and C. Ogden, “System design for a SPIDER imager,” in Frontiers in Optics, paper FM3E.3 (2015).
9. T. Su, G. Liu, K. E. Badham, S. T. Thurman, R. L. Kendrick, A. Duncan, D. Wuchenich, C. Ogden, G. Chriqui, S. Feng, J. Chun, W. Lai and S. J. B. Yoo, “Interferometric imaging using Si3N4 photonic integrated circuits for a SPIDER imager,” Opt. Express 26, 12801-12812 (2018).
10. N. K. Fontaine, R. P. Scott, J. P. Heritage and S. J. B. Yoo, “Near quantum-limited, single-shot coherent arbitrary optical waveform measurements,” Opt. Express 17, 12332-12344 (2009).
11. D. J. Geisler, N. K. Fontaine, R. P. Scott, J. P. Heritage, K. Okamoto and S. J. B. Yoo, “360-Gb/s optical transmitter with arbitrary modulation format and dispersion precompensation,” IEEE Photonics Technol. Lett. 21, 489-491 (2009).
12. T. Su, R. Scott, C. Ogden, S. T. Thurman, R. L. Kendrick, A. Duncan, R. Yu and S. J. B. Yoo, “Experimental demonstration of interferometric imaging using photonic integrated circuits,” Opt. Express 25, 12653-12665 (2017).
13. R. L. Kendrick, A. Duncan, C. Ogden, J. Wilm, D. M. Stubbs, S. T. Thurman, T. H. Su, R. P. Scott and S. J. B. Yoo, “Flat-Panel Space-Based Space Surveillance Sensor,,” in in Proc. Proceedings of the Advanced Maui Optical and Space Surveillance Technologies Conference, (Hawaii, USA, 2013). pp. 10-13.
14. J. W. Goodman in Statistical Optics, (Wiley-Interscience, USA, 2000).
15. T. Hashimoto and I. Ogawa, “Optical hybrid integration using planar lightwave circuit platform,” Proc. SPIE 4652, 58-67 (2002).
16. Q. Yu, B. Ge, Y. Li, Y. Yue, F. Chen and S. Sun, “ystem design for a "checkerboard" imager,” Appl. Opt. 57, 10218-10223 (2018).
17. G.-M. Lv, Q. Li, Y.-T. Chen, H.-J. Feng, Z.-H. Xu and J. Mu, “An improved scheme and numerical simulation of segmented planar imaging detector for electro-optical reconnaissance,” Opt. Rev. 26, 664-675 (2019).
18. C. S. Cho and J. H. Han, “Phase error reduction for multi-frequency Fringeprojection profilometry using adaptive compensation,” Curr. Opt. Photon. 2, 332-339 (2018).
19. C. Li, “An efficient algorithm for total variation regularization with applications to the single pixel camera and compressive sensing,” M.s Thesis, Rice University, USA, Rice (2009).
20. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13, 600-612 (2004).
21. B. G. Clark, “An efficient implementation of the algorithm 'CLEAN',” Astron. Astrophys. 89, 377-378 (1980).
22. V. J. Kreinovic and O. M. Kosheleva, “Matters arising,” Nature 281, 708-709 (1979).
23. Z. Leihong, Y. Xiao, Z. Dawei and C. Jian, “Research on multiple-image encryption scheme based on Fourier transform and ghost imaging algorithm,” Curr. Opt. Photon. 2, 315-323 (2018).
24. Y. Zhang, J. Deng, G. Liu, J. Fei and H. Yang, “Simultaneous estimation of spatial frequency and phase based on an improved component cross-correlation algorithm for structured illumination microscopy,” Curr. Opt. Photon. 4, 317-325 (2020).

### Article

#### Article

Curr. Opt. Photon. 2021; 5(6): 606-616

Published online December 25, 2021 https://doi.org/10.3807/COPP.2021.5.6.606

## Spatial Frequency Coverage and Image Reconstruction for Photonic Integrated Interferometric Imaging System

Wang Zhang , Hongliu Ma, Kang Huang

School of Mechanical and Aerospace Engineering, Jilin University, Changchun 130025, China

Correspondence to:*wangzhang@jlu.edu.cn, ORCID 0000-0001-9029-1320

Received: June 25, 2021; Revised: September 2, 2021; Accepted: September 7, 2021

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

### Abstract

A photonic integrated interferometric imaging system possesses the characteristics of small-scale, low weight, low power consumption, and better image quality. It has potential application for replacing conventional large space telescopes. In this paper, the principle of photonic integrated interferometric imaging is investigated. A novel lenslet array arrangement and lenslet pairing approach are proposed, which are helpful in improving spatial frequency coverage. For the novel lenslet array arrangement, two short interference arms were evenly distributed between two adjacent long interference arms. Each lens let in the array would be paired twice through the novel lenslet pairing approach. Moreover, the image reconstruction model for optical interferometric imaging based on compressed sensing was established. Image simulation results show that the peak signal to noise ratio (PSNR) of the reconstructed image based on compressive sensing is about 10 dB higher than that of the direct restored image. Meanwhile, the normalized mean square error (NMSE) of the direct restored image is approximately 0.38 higher than that of the reconstructed image. Structural similarity index measure (SSIM) of the reconstructed image based on compressed sensing is about 0.33 higher than that of the direct restored image. The increased spatial frequency coverage and image reconstruction approach jointly contribute to better image quality of the photonic integrated interferometric imaging system.

Keywords: Compressive sensing, Image reconstruction, Optical imaging, Optical interferometry, Photonic integrated circuits

### I. INTRODUCTION

In order to improve spatial resolution, conventional optical telescopes were usually designed with large apertures, leading to large volume, heavy weight and high power consumption [1, 2]. With the rapid development of photonic integrated circuit (PIC) technology and interferometric imaging technology, the concept known as the segmented planar imaging detector for electro-optical reconnaissance (SPIDER) has been proposed, which is of great importance for its potential applications in astronomy and space science [3]. For photonic integrated interferometric imaging (PIII), the large optical components, devices and mounting structures in conventional system are replaced by a densely packed interferometer array realized in an extremely thin layer of lenslets and PIC components, which offers substantial reductions in system size, weight, and cost. Using on-chip components, the high quality image is reconstructed by collecting and processing the interferometric information of Fourier domain at the output end [48]. At present, PIII technology is at the primary research stage and has not been transformed into production. Relevant research institutions only carried out some demonstrative experiments and theoretical investigations [9]. Key technologies concerning PIII need to be addressed, which include the principle of optical interferometric imaging and signal transmission mechanism, spatial frequency coverage and image reconstruction of the scene.

This paper focuses on spatial frequency coverage and image reconstruction algorithm of PIII. The former aspect determines the direct restored image, and the latter one influences the reconstructed image quality. Section 2 discusses the basic theory and concept of PIII. Section 3 describes a novel lenslet array arrangement and lenslet pairing approach that effectively improve spatial frequency coverage of the scene. Meanwhile, it introduces an image reconstruction approach based on compressive sensing (CS). Section 4 shows image reconstruction from a simulated PIII observation. Section 5 has the discussion. The conclusion is given in Section 6.

### II. THEORY AND CONCEPT

PIII system is to put a linear array of lenslets onto a PIC card, and the PIC cards are mounted as radial spokes on a disc. PIII system consists of three working modules: signal receiving module, signal transmission module and signal processing module. The signal receiving module mainly includes the lenslet array and its components. The signal transmission module consists of optical waveguides, arrayed waveguide gratings (AWGs), phase shifters, and multi-mode interferometers (MMIs). The signal processing module contains balanced detectors and digital signal processing. PICs transform the target intensity distribution into Fourier domain information. By scanning the phase delay on one of the arms in the MMI, the balanced detectors receive interferometric information, the phase and the amplitude of the complex visibility will be calculated. Then the image is restored by processing Fourier transform of the complex visibility information [10, 11]. Figure 1 shows the traditional structure of system and the working principle of a PIC.

Figure 1. The traditional structure of system and the working principle of a photonic integrated circuit (PIC) [12,13].

PIII is a new implementation approach of optical interferometric technology. According to the Van Cittert–Zernike theory [14], the measured complex visibility y between the paired lenslets (x1, y1) and (x2, y2) can be expressed as

$y(u,v)=∫ ∫ x(ξ,η)ρ ξ,ηexp −j2π(uξ+vη)dξdη,$

where ρ is the lenslet coupling efficiency projected onto the object plane. (ξ, η) is the coordinate of image intensity x. (u, v) is the spatial frequency corresponding to the distance vector between two lenslets (x1, y1) and (x2, y2):

$u=Δxλz=x2−x1λz,$

$u=Δyλz=y2−y1λz,$

where Δx, Δy are the distances of the chosen pair of lenslets in the x and y directions. λ is the current working wavelength of the system, and z is the scene distance.

For the PIII system, the paired beam passing through the phase shifters would interfere in MMI, and the interferometric information would be measured by the balanced detectors. As shown in Fig. 1, the optical signals input into the 90° optical hybrids can be expressed as ES and ER respectively, and the output signals of the 90° optical hybrids can be expressed as [15]

$Eout1=12ES+ER,$

$Eout2=12ES−ER,$

$Eout3=12ES+jER,$

$Eout4=12ES−jER.$

The output electrical signals I and Q of balanced detectors can be derived as follows:

$I=I1−I2=Eout1Eout1*−Eout2Eout2*=Re(ESER*),$

$Q=Q1−Q2=Eout3Eout3*−Eout4Eout4*=Im(ESER*),$

where E* is the conjugate of E. From equations above, the phase φ and amplitude E of the measured complex visibility can be expressed as

$E=I2+Q212,$

$φ=arctanQ1.$

The measured complex visibility y can be obtained by using the output information from the system as follows:

$y(u,v)=I+jQ=Eexp(−jφ).$

According to the Van Cittert-Zernike theorem, the complex visibility can be found by a two-dimensional Fourier transformation of the intensity distribution across the source [14]. Therefore, the restored image can be expressed as

$x(ξ,η)=F−1yu,v.$

### 3.1. Structure and Spatial Frequency Coverage

We designed a novel arrangement of lenslet array as shown in Fig. 2. This lenslet array arrangement is composed of thirty-seven long interference arms. Between two adjacent long interference arms, pairs of short interference arms are evenly distributed. Each long interference arm consists of twenty-nine lenslets (red color) and short interference arm contains fifteen lenslets (yellow color). The interval between two adjacent lenslets is equal to the diameter d of the lenslet. Considering the system is composed of P1 long interference arms, the angle between adjacent long interference arms is α1 = 2π / P1. Considering the system is composed of P2 short interference arms, the angle between adjacent short interference arms is α2 = 2π/(P1 + P2).

Figure 2. The schematic diagram of the new system: (a) the new structure of system and (b) the novel arrangement of lenslet array.

The spatial frequency coverage is influenced by the lenslet array arrangement and pairing approach of the lenslets in each radial spoke. As shown in Fig. 3, 1 × 2 beam splitters are designed to follow each lenslet except the middle and last lenslets. Two 1 × 3 beam splitters are designed to follow the middle and last lenslets respectively. Erbium-doped fibre amplifiers (EDFAs) are designed to amplify the signal intensity output from each 1 × 3 beam splitter to maintain equal signal intensity corresponding to the 1 × 2 beam splitter. The PIII system takes advantage of AWGs to divide broadband light into multiple narrow spectral channels to improve spatial frequency coverage. The paired narrow spectral beam passing through the phase shifters would be interfered in MMI, and the interferometric information would be measured by the balanced detectors. By setting up 1 × 2 and 1 × 3 beam splitters following the lenslets, all the lenslets can be paired twice in each radial spoke. For example, assuming each radial spoke contains 5 lenslets, the pairing approach is (1, 5), (2, 4), (3, 3), (1, 4), (2, 3), (5, 5).

Figure 3. The schematic diagram of the optimized photonic integrated device. (a) The working principle of a PIC, (b) the 1 × 2 beam splitter, (c) the 1 × 3 beam splitter, and (d) 5 lenslets pairing.

Based on setting up 1 × 2 and 1 × 3 beam splitters following the lenslets in Fig. 3, the novel pairing details of lenslets can be described as follows. Assuming each interference arm contains N lenslets, the lenslets were paired firstly following (1, N − 1), (2, N − 2), (3, N − 3)... ((N − 1) / 2, (N + 1) / 2), (N, N) as shown in Fig. 4(a). After that, the lenslets were secondly matched as (1, N), (2, N − 1), (3, N − 2)... ((N − 1) / 2, (N + 3) / 2), ((N + 1) / 2, (N + 1) / 2), as shown in Fig. 4(b). Zero spatial frequency sampling with the image average brightness information is obtained by pairing lenslets N and (N + 1) / 2 with itself [16]. Through this novel lenslet pairing approach, the length of the baseline is 0, d, 2d, 3d, …, (N − 1)d. The spatial frequency coverage is approximately twice than that of traditional lenslet pairing [17]. Therefore, this novel lenslet pairing approach can effectively improve the spatial frequency coverage of the scene. Consequently, low frequency and high frequency information of the scene can be achieved simultaneously, which is beneficial for the image restoration of both outline and detail of the scene.

Figure 4. The Schematic diagram of the novel lenslet pairing method: (a) the first pairing and (b) the second pairing.

### 3.2. Compressive Sensing Theory

CS is a novel signal sampling theory based on the sparsity and compression of an observed signal. CS is a robust framework for signal reconstruction. The theoretical framework of CS motivates sparse regularisation for solving inverse problems, through which, the quality of image reconstruction can be improved effectively [18].

### 3.2.1. Measurement Model

According to Van Cittert-Zernike theorem, the measurement equation takes the form of a Fourier transform. By inverse Fourier transform of the complex visibility measured by the system, intensity distribution of the scene will be achieved. Due to the imaging character of the PIII system, it is not possible to sample the entire (u, v) domain, leading to an ill-posed inverse problem of reconstructing image x from measured values y. The measurement equation for the PIII system can be represented as

$y=Ax+n,$

where x ϵ RT×1 is intensity of image. T is the total number of image pixels. y ϵ CM×1 is the measured complex visibility. M represents the number of measurements. A ϵ CM×T is the linear transformation matrix from the spatial domain to the frequency domain. n represents independently and identically distributed noise. Noise is not considered in this manuscript. A is the linear transformation matrix from the spatial domain to the frequency domain:

$A=SF,$

where S ϵ RM×T is the binary matrix for spatial frequency sampling of the object. F ϵ CT×T is the discrete Fourier transform. Then, we have the following linear model:

$y=SFx+n,$

where y can be calculated from the equation (12). S ϵ RM×T can be obtained through the spatial frequency coverage of the scene. F ϵ CT×T can be gained through the discrete Fourier transform basis.

### 3.2.2. Reconstruction Method

Fourier frequencies are not measured adequately through the PIII system, fitting the data alone does not uniquely define the image sought. Therefore, the image is reconstructed by using the total variation augmented Lagrangian alternating direction algorithm. The basic framework for image reconstruction is shown in Fig. 5.

Figure 5. The basic framework for image reconstruction.

Sparse regularization contributes to sparse solutions in terms of solving ill-posed inverse problems. To be specific, the ill-posed inverse problem can be solved by sparse regularization through imposing a priori constraints to select a unique image among all those that are consistent with the data. Moreover, this inverse problem of reconstructing image x from measured values y can be solved by building a total variation regularization mode, which can be expressed as

$minx ∑i D ix subject to Ax=y,$

where Dix is gradient of x at pixel i. The slack variable wi is introduced. The model of image reconstruction problem can be expressed as

$minwi,x ∑i wi subject to Ax=y, Dix=wi for all i.$

The constrained model (18) can be converted to an unconstrained objective function by the augmented Lagrangian method, which can be expressed as

$l˜A(wi,x)=∑ iwi−viTDix−wi+βi2 Dix−wi22−kT(Ax−y)+μ2 Ax−y22,$

where νi, κ are Lagrangian multipliers. βi, μ are penalty parameters. As shown in Fig. 5, the image reconstruction problem can be decomposed into two simple sub-problems by the alternating direction algorithm, the sub-problem wi and the sub-problem x. Through the iterative method, the sub-problem wi is solved firstly, then the sub-problem x is done. The sub-problem is to minimize the augmented Lagrangian function, which can be represented as

$minwi,xl˜Awi,x= ∑i wi−viT Dix−wi+ βi2 D ix− w i22−kT(Ax−y)+μ2 Ax−y22.$

Supposing xk denotes the minimizer of the sub-problem x at the k-th iteration, wik+1 can be obtained by minimizing the objective function with respect to wi on each edge as follows:

$minwi ∑i wi−viT Dixk−wi+ βi2 Dixk−wi22.$

Having wik+1 at hand, xk+1 can be achieved by minimizing the objective function with respect to x as follows:

$minxl˜Awik+1,x= ∑i wik+1−viT Dix−wik+1+ βi2 Dix−wik+122−kT(Ax−y)+μ2 Ax−y22.$

Lagrangian multipliers are updated through the well known formulas:

$vik+1=vik−βik(Dixk+1−wik+1),$

$κk+1=κk−μk(Axk+1−y).$

New penalty parameters μ and βi are chosen. The most effective way to choose new penalty parameters is to try different values and compare the corresponding image quality under different parameter values as shown in Fig. 6. The selected penalty parameters have better image quality.

Figure 6. Image quality under different parameter values: (a) image quality under different μ parameter values and (b) image quality under different βi parameter values.

Variables wi and x are solved alternatively through updating Lagrangian multipliers and choosing penalty parameters to ensure that the constraint condition is met in the final solution [19].

### IV. RESULTS

The PIII system parameters used for the simulations are listed in Table 1. The sampling pattern in the uv-plane is shown in Fig. 7. Since the acquisition of spatial frequency information is an under-sampling process, this results in the loss of the partial image content. Therefore, the PSNR, the NMSE and SSIM are used as the evaluation parameters of image reconstruction. For a test image Y and its original image X, the PSNR, NMSE and SSIM metrics are defined as [20]

TABLE 1. System parameters used for the simulations.

ParameterSymbol of ParametersParameter Values
Spectral Coverageλ380–760 nm
Lenslet Diameterd1 mm
Longest BaselineBmax28 mm
Number of Long PIC SpokesP137
Number of Lenslets per Long PIC SpokesN129
Number of Short PIC SpokesP274
Number of Lenslets per Short PIC SpokesN215
Scene Distancez100 km
Number of Spectral Binsm10

Figure 7. The sampling pattern of PIII system in the uv-plane.

$PSNR=10log10MAX2MSE,$

$NMSE=∑i=1W∑j=1H X i,j−Y i,j2∑i=1W∑j=1H X i,j2,$

$SSIM=(2μXμY+C1)(2σXY+C2)(μX2+μY2+C1)(σX2+σY2+C2),$

where MAX is the maximum possible pixel value of the image. MSE is the mean square error of the image. μX and μY are the averages of X and Y. σ2X and σ2Y are the variances of X and Y. σXY is the covariance of X and Y. C1 and C2 are two variables used to stabilize division involving a weak denominator. The larger PSNR and SSIM are, the better image quality is acquired. On the contrary, the smaller NMSE is, the better image quality is achieved.

Traditional lenslet array arrangement (as shown in Fig. 1) consists of radial spokes with equal length on a disc. The traditional lenslet pairing follows symmetrical scheme: (1, N), (2, N−1), (3, N−2)…(N/2, N/2+1). For comparison, the imaging simulation experiments were carried out for the PIII system in traditional and novel lenslet array arrangements and lenslet pairing method. All the images acquired through different approaches are shown in Figs. 810. The graphs of spatial frequency coverage are shown in Fig. 11. The calculated PSNR, NMSE and SSIM are listed in Table 2.

TABLE 2. The calculated PSNR, NMSE and SSIM.

The ImagePSNR(/dB)NMSESSIM
USAF1951(b)10.370.450.47
USAF1951(c)10.810.410.53
USAF1951(d)19.640.050.96
USAF1951(e)24.090.010.98
The Saturn(b)24.530.400.67
The Saturn(c)24.670.390.68
The Saturn(d)33.450.050.96
The Saturn(e)34.420.040.97
The Nebula(b)30.550.500.68
The Nebula(c)30.740.480.69
The Nebula(d)37.840.090.94
The Nebula(e)38.060.080.95

Figure 8. USAF1951: (a) the original image, (b) direct restored image with traditional lenslet array arrangement and symmetrical lenslet pairing method, (c) direct restored image with novel lenslet array arrangement and lenslet pairing method presented in this paper, (d) compressive sensing (CS) reconstruction image with traditional lenslet array arrangement and symmetrical lenslet pairing method, and (e) CS reconstruction image with novel lenslet array arrangement and lenslet pairing method.

Figure 9. The Saturn: (a) the original image, (b) direct restored image with traditional lenslet array arrangement and symmetrical lenslet pairing method, (c) direct restored image with novel lenslet array arrangement and lenslet pairing method presented in this paper, (d) compressive sensing (CS) reconstruction image with traditional lenslet array arrangement and symmetrical lenslet pairing method, and (e) CS reconstruction image with novel lenslet array arrangement and lenslet pairing method.

Figure 10. The Nebula: (a) the original image, (b) direct restored image with traditional lenslet array arrangement and symmetrical lenslet pairing method, (c) direct restored image with novel lenslet array arrangement and lenslet pairing method presented in this paper, (d) compressive sensing (CS) reconstruction image with traditional lenslet array arrangement and symmetrical lenslet pairing method, and (e) CS reconstruction image with novel lenslet array arrangement and lenslet pairing method.

Figure 11. The graphs of spatial frequency coverage: (a) the graph of spatial frequency coverage constructed by a conventional system, (b) the graph of spatial coverage constructed by our system, and (c) the graph of increased spatial frequency coverage.

As listed in Table 2, the PSNR of direct restored image using the novel lenslet array arrangement and pairing method increases approximately 0.25 dB than that of the direct restored image using the traditional lenslet array arrangement and symmetrical pairing method. The NMSE of the direct restored image using the novel lenslet array arrangement and pairing method proposed by this paper decreases approximately 0.02 more than that of the direct restored image using the traditional lenslet array arrangement and pairing method. The SSIM of the direct restored image using the novel lenslet array arrangement and pairing method increases approximately 0.02 than that of direct restored image by using traditional lenslet array arrangement and symmetrical pairing method.

Image reconstruction based on CS theory is carried out and uses the novel lenslet array arrangement and pairing method. The PSNR of CS reconstructed image increases by approximately 10 dB from that of the direct restored image. Meanwhile, the NMSE calculated from the direct restored image is approximately 0.38 higher than that of the reconstructed image based on CS. The SSIM of the CS reconstructed image increases approximately 0.33 than that of the direct restored image. The PSNR of the CS reconstructed image using the novel lenslet array arrangement and pairing method increases approximately 1.88 dB from that of CS reconstructed image using the traditional lenslet array arrangement and symmetrical pairing method. Meanwhile, the NMSE calculated from the CS reconstructed image using traditional lenslet array arrangement and symmetrical pairing method increases approximately 0.02 than that of CS reconstructed image using the novel lenslet array arrangement and the pairing method. The SSIM of the CS reconstructed image using the novel lenslet array arrangement and pairing method increases approximately 0.01 than that of the CS reconstructed image using traditional lenslet array arrangement and symmetrical pairing method.

Figures 810 are the images acquired through different approaches. According to Figs. 810, the image restoration and reconstruction simulation results show that the novel lenslet array arrangement and pairing method presented in this paper could effectively improve the spatial frequency coverage and have influence on direct restored image quality. Meanwhile, image reconstruction based on CS theory could improve the image quality of interferometric imaging.

The novel lenslet array arrangement adds lenslet arrays in more directions, which are different from the directions of the existing radial lenslets spokes. The novel lenslet pairing approach adds more spatial frequency sampling points. As shown in Fig. 11, the traditional lenslet array arrangement and lenslet pairing method can collect interferometric information in 37 directions (radial spokes) and obtains 5180 measured points, while this novel design can collect interferometric information in 111 directions and obtains 20720 measured points. The obtained spatial frequency points based on the novel lenslet array arrangement and lenslet pairing approach are approximately four times than that of traditional design. Therefore, this novel design improves spatial frequency coverage of the system. The novel lenslet array arrangement and lenslet pairing method designed in this paper can improve the image quality because spatial frequency coverage has the direct influence on image quality.

### V. DISCUSSION

This paper proposed the novel approach for improving spatial frequency coverage and image reconstruction. As shown in Fig. 1, the traditional lenslet array arrangement is composed of radial spokes with equal length on a disc. However, the novel lenslet array arrangement consists of different length spokes on a disc, as shown in Fig. 2. Compared with the traditional lenslet array arrangement, the obtained spatial frequency points based on the novel lenslet array arrangement are approximately twice than that of traditional design, which can effectively improve the spatial frequency coverage of the scene.

The traditional lenslet pairing follows symmetrical scheme: (1, N), (2, N−1), (3, N−2)…(N/2, N/2+1), the length of the baseline obtained through the traditional lenslet pairing approach is d, 3d, 5d, …, (N−1)d. Nevertheless, the length of the baseline obtained by the novel lenslet pairing approach is 0, d, 2d, 3d, …, (N−1)d, and the number of the baseline is approximately twice than that of traditional lenslet pairing. Therefore, the novel lenslet pairing approach can acquire the more spatial frequency information, which efficiently improve the image quality of the PIII system.

The traditional reconstruction algorithms mainly include CLEAN algorithm and maximum entropy method (MEM) algorithm. CLEAN algorithm can be seen as a greedy pursuit algorithm with an over-complete dictionary. Therefore, CLEAN algorithm seems unstable. The core idea of MEM algorithm is to obtain the least informative image that is consistent with the data [21, 22]. CS algorithm is applied to the PIII system in this paper, and it has significant impacts in other applications such as medical imaging, synthetic aperture radar imaging, remote sensing imaging, face recognition, speech recognition. Compared with traditional imaging methods, CS provides a framework for recovering image from the fewer measurements and has great advantages in flexibility [23, 24].

### VI. CONCLUSION

In this paper, a spatial frequency coverage method and image reconstruction algorithm for the PIII system were proposed. The novel lenslet array arrangement and lenslet pairing method positively influence the spatial frequency coverage. The acquired dense spatial frequencies efficiently improve the quality of the direct restored image. Based on CS theory, the image reconstruction model for the PIII system was established. By using this image reconstruction model, interferometric image with better quality could be acquired. The research content of this paper can provide technological support for optical interferometric imaging.

Although the image quality of the PIII system is improved based on the research content of this paper, there is still potential space for improving the quality of the image. In the future, the arrangement of lenslet array and the approach of lenslet pairing, which can increase the spatial frequency coverage, will still continue to be researched.

### Fig 1.

Figure 1.The traditional structure of system and the working principle of a photonic integrated circuit (PIC) [12,13].
Current Optics and Photonics 2021; 5: 606-616https://doi.org/10.3807/COPP.2021.5.6.606

### Fig 2.

Figure 2.The schematic diagram of the new system: (a) the new structure of system and (b) the novel arrangement of lenslet array.
Current Optics and Photonics 2021; 5: 606-616https://doi.org/10.3807/COPP.2021.5.6.606

### Fig 3.

Figure 3.The schematic diagram of the optimized photonic integrated device. (a) The working principle of a PIC, (b) the 1 × 2 beam splitter, (c) the 1 × 3 beam splitter, and (d) 5 lenslets pairing.
Current Optics and Photonics 2021; 5: 606-616https://doi.org/10.3807/COPP.2021.5.6.606

### Fig 4.

Figure 4.The Schematic diagram of the novel lenslet pairing method: (a) the first pairing and (b) the second pairing.
Current Optics and Photonics 2021; 5: 606-616https://doi.org/10.3807/COPP.2021.5.6.606

### Fig 5.

Figure 5.The basic framework for image reconstruction.
Current Optics and Photonics 2021; 5: 606-616https://doi.org/10.3807/COPP.2021.5.6.606

### Fig 6.

Figure 6.Image quality under different parameter values: (a) image quality under different μ parameter values and (b) image quality under different βi parameter values.
Current Optics and Photonics 2021; 5: 606-616https://doi.org/10.3807/COPP.2021.5.6.606

### Fig 7.

Figure 7.The sampling pattern of PIII system in the uv-plane.
Current Optics and Photonics 2021; 5: 606-616https://doi.org/10.3807/COPP.2021.5.6.606

### Fig 8.

Figure 8.USAF1951: (a) the original image, (b) direct restored image with traditional lenslet array arrangement and symmetrical lenslet pairing method, (c) direct restored image with novel lenslet array arrangement and lenslet pairing method presented in this paper, (d) compressive sensing (CS) reconstruction image with traditional lenslet array arrangement and symmetrical lenslet pairing method, and (e) CS reconstruction image with novel lenslet array arrangement and lenslet pairing method.
Current Optics and Photonics 2021; 5: 606-616https://doi.org/10.3807/COPP.2021.5.6.606

### Fig 9.

Figure 9.The Saturn: (a) the original image, (b) direct restored image with traditional lenslet array arrangement and symmetrical lenslet pairing method, (c) direct restored image with novel lenslet array arrangement and lenslet pairing method presented in this paper, (d) compressive sensing (CS) reconstruction image with traditional lenslet array arrangement and symmetrical lenslet pairing method, and (e) CS reconstruction image with novel lenslet array arrangement and lenslet pairing method.
Current Optics and Photonics 2021; 5: 606-616https://doi.org/10.3807/COPP.2021.5.6.606

### Fig 10.

Figure 10.The Nebula: (a) the original image, (b) direct restored image with traditional lenslet array arrangement and symmetrical lenslet pairing method, (c) direct restored image with novel lenslet array arrangement and lenslet pairing method presented in this paper, (d) compressive sensing (CS) reconstruction image with traditional lenslet array arrangement and symmetrical lenslet pairing method, and (e) CS reconstruction image with novel lenslet array arrangement and lenslet pairing method.
Current Optics and Photonics 2021; 5: 606-616https://doi.org/10.3807/COPP.2021.5.6.606

### Fig 11.

Figure 11.The graphs of spatial frequency coverage: (a) the graph of spatial frequency coverage constructed by a conventional system, (b) the graph of spatial coverage constructed by our system, and (c) the graph of increased spatial frequency coverage.
Current Optics and Photonics 2021; 5: 606-616https://doi.org/10.3807/COPP.2021.5.6.606

TABLE 1 System parameters used for the simulations

ParameterSymbol of ParametersParameter Values
Spectral Coverageλ380–760 nm
Lenslet Diameterd1 mm
Longest BaselineBmax28 mm
Number of Long PIC SpokesP137
Number of Lenslets per Long PIC SpokesN129
Number of Short PIC SpokesP274
Number of Lenslets per Short PIC SpokesN215
Scene Distancez100 km
Number of Spectral Binsm10

TABLE 2 The calculated PSNR, NMSE and SSIM

The ImagePSNR(/dB)NMSESSIM
USAF1951(b)10.370.450.47
USAF1951(c)10.810.410.53
USAF1951(d)19.640.050.96
USAF1951(e)24.090.010.98
The Saturn(b)24.530.400.67
The Saturn(c)24.670.390.68
The Saturn(d)33.450.050.96
The Saturn(e)34.420.040.97
The Nebula(b)30.550.500.68
The Nebula(c)30.740.480.69
The Nebula(d)37.840.090.94
The Nebula(e)38.060.080.95

### References

1. R. Athale, D. M. Healy, D. J. Brady and M. A. Neifeld, “Reinventing the camera,” Opt. Photonics News 19, 32-37 (2008).
2. J. E. Baldwin, M. G. Beckett, R. C. Boysen, D. Burns and T. R. Scott, “The first images from an optical aperture synthesis array: mapping of Capella with COAST at two epochs,” Astron. Astrophys. 306, L13-L16 (1996).
3. R. P. Scott, T. H. Su, C. Ogden, S. T. Thurman, R. L. Kendrick, A. Duncan, R. X. Yu and S. J. B. Yoo, “Demonstration of a photonic integrated circuit for multi-baseline interferometric imaging,,” in in Proc. IEEE Photonics Conference, (San Diego, CA, USA, 2015).
4. Q. H. Chu, Y. J. Shen, M. Yuan and M. L. Gong, “Numerical simulation and optimal design of segmented planar imaging detector for electro-optical reconnaissance,” Opt. Commun. 405, 288-296 (2017).
5. G. Liu, T. Su, S. Li, J. Chun, W. Lai, M. Prost, C. Ogden, S. T. Thurman, R. L. Kendrick, A. Duncan and S. J. B. Yoo, “Si3N4 photonic integrated circuit for multi-baseline interferometric imaging,,” in in Proc. IEEE Photonics Conference-IPC, (Orlando, FL, USA, 2017). pp. 51-52.
6. W. P. Gao, X. R. Wang, L. Ma, Y. Yuan and D. F. Guo, “Quantitative analysis of segmented planar imaging quality based on hierarchical multistage sampling lens array,” Opt. Express 27, 7955-7967 (2019).
7. R. L. Kendrick, A. Duncan, C. Ogden, J. Wilm and S. T. Thurman, “Segmented planar imaging detector for EO reconnaissance,” in Imaging and Applied Optics, paper CM4C.1 (2013).
8. S. T. Thurman, R. L. Kendrick, A. Duncan, D. Wuchenich and C. Ogden, “System design for a SPIDER imager,” in Frontiers in Optics, paper FM3E.3 (2015).
9. T. Su, G. Liu, K. E. Badham, S. T. Thurman, R. L. Kendrick, A. Duncan, D. Wuchenich, C. Ogden, G. Chriqui, S. Feng, J. Chun, W. Lai and S. J. B. Yoo, “Interferometric imaging using Si3N4 photonic integrated circuits for a SPIDER imager,” Opt. Express 26, 12801-12812 (2018).
10. N. K. Fontaine, R. P. Scott, J. P. Heritage and S. J. B. Yoo, “Near quantum-limited, single-shot coherent arbitrary optical waveform measurements,” Opt. Express 17, 12332-12344 (2009).
11. D. J. Geisler, N. K. Fontaine, R. P. Scott, J. P. Heritage, K. Okamoto and S. J. B. Yoo, “360-Gb/s optical transmitter with arbitrary modulation format and dispersion precompensation,” IEEE Photonics Technol. Lett. 21, 489-491 (2009).
12. T. Su, R. Scott, C. Ogden, S. T. Thurman, R. L. Kendrick, A. Duncan, R. Yu and S. J. B. Yoo, “Experimental demonstration of interferometric imaging using photonic integrated circuits,” Opt. Express 25, 12653-12665 (2017).
13. R. L. Kendrick, A. Duncan, C. Ogden, J. Wilm, D. M. Stubbs, S. T. Thurman, T. H. Su, R. P. Scott and S. J. B. Yoo, “Flat-Panel Space-Based Space Surveillance Sensor,,” in in Proc. Proceedings of the Advanced Maui Optical and Space Surveillance Technologies Conference, (Hawaii, USA, 2013). pp. 10-13.
14. J. W. Goodman in Statistical Optics, (Wiley-Interscience, USA, 2000).
15. T. Hashimoto and I. Ogawa, “Optical hybrid integration using planar lightwave circuit platform,” Proc. SPIE 4652, 58-67 (2002).
16. Q. Yu, B. Ge, Y. Li, Y. Yue, F. Chen and S. Sun, “ystem design for a "checkerboard" imager,” Appl. Opt. 57, 10218-10223 (2018).
17. G.-M. Lv, Q. Li, Y.-T. Chen, H.-J. Feng, Z.-H. Xu and J. Mu, “An improved scheme and numerical simulation of segmented planar imaging detector for electro-optical reconnaissance,” Opt. Rev. 26, 664-675 (2019).
18. C. S. Cho and J. H. Han, “Phase error reduction for multi-frequency Fringeprojection profilometry using adaptive compensation,” Curr. Opt. Photon. 2, 332-339 (2018).
19. C. Li, “An efficient algorithm for total variation regularization with applications to the single pixel camera and compressive sensing,” M.s Thesis, Rice University, USA, Rice (2009).
20. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13, 600-612 (2004).
21. B. G. Clark, “An efficient implementation of the algorithm 'CLEAN',” Astron. Astrophys. 89, 377-378 (1980).
22. V. J. Kreinovic and O. M. Kosheleva, “Matters arising,” Nature 281, 708-709 (1979).
23. Z. Leihong, Y. Xiao, Z. Dawei and C. Jian, “Research on multiple-image encryption scheme based on Fourier transform and ghost imaging algorithm,” Curr. Opt. Photon. 2, 315-323 (2018).
24. Y. Zhang, J. Deng, G. Liu, J. Fei and H. Yang, “Simultaneous estimation of spatial frequency and phase based on an improved component cross-correlation algorithm for structured illumination microscopy,” Curr. Opt. Photon. 4, 317-325 (2020).

Wonshik Choi,
Editor-in-chief