Ghost imaging (GI), a new non-traditional imaging method, uses light field’s high-order correlation to obtain the object’s spatial or phase distribution information [1]. Computational ghost imaging (CGI) technology has great application potential and has a particular application value in 3D imaging, remote sensing imaging, the medical field, optical encryption, national defense, the military [2–4] and so on. After the theory of compressed sensing (CS) was proposed [5], which broke through the limitation of the Nyquist sampling theorem and realized compressed sensing ghost imaging (CSGI) [6], it was also used to deal with the issue of random scattering [7]. It can be thought that in remote sensing imaging, astronomical imaging and other processes, the influencing factors of the environment cannot be ignored [8].

At present, the research on GI in atmospheric turbulence is extensive. In 2012, Zhang and Wang [9] studied the influence of atmospheric turbulence on GI. In 2013, Cheng and Lin [10] studied the effects of atmospheric turbulence intensity, light source coherence, light source size and arm detector size on GI. In 2016, Yang et al. [11] studied a situation where the light path of GI system contained a strong scattering medium. In 2018, Tang et al. [12] studied the position of the measured object in GI in turbulence. When GI is performed in fog and other complex weather or long-range imaging, light beams are absorbed or scattered in the randomly changing atmospheric turbulence, and the light signal is seriously affected, resulting in noise, distortion, blurring and other problems in the final imaging [13].

Computational imaging (CI) breaks through the limitations of traditional optical imaging. There are inherent problems in the imaging process, such as the diffraction limit, optical scattering limitations, and the constraints of optical systems [14]. CI combines an imaging system with information processing and can recover complete object information from incomplete and hidden measurement information. Thus, it has great practical significance. This technology has been widely used in a variety of imaging systems, and some typical inverse problems in imaging are solved, such as quantum imaging [15, 16], three-dimensional imaging technology [17], and multimode fiber imaging [18]. In recent years, deep learning (DL) has made unprecedented progress and has been used extensively in face recognition, machine translation, natural language processing and other fields [19, 20]. Some researchers have dealt with some problems by combining them with computational imaging (CI) and have solved many problems in the imaging field.

This paper first introduces the development and current situation of GI and the principle of the imaging process. Then we use the gamma-gamma (GG) model to simulate atmospheric turbulence within the medium to strong turbulent fluctuation rage. We research the performance of GI, and some influencing factors including turbulence intensity, transmission distance, and sampling rate. Finally, we compare various restoration methods and analyze the imaging results after restoration. The results show that the distorted imaging can be well restored by the proposed method. In this paper, we deal with the distortion of GI with advanced DL methods, and the results prove the superiority of DL in processing problems of GI.

2.1. CSGI in Atmospheric Turbulence

In CGI, we load a random speckle image into a digital micromirror device (DMD) instead of the traditional rotating ground glass [21, 22]. The imaging process is shown in Fig. 1. This greatly reduces the complexity of the procedure and operation in the experiment process and makes it more practical.

Figure 1. Schematic diagram of CGI.

The phase of the laser source is modulated by the DMD. Optical path information is collected from the target object and recorded by the bucket detector. The total strength is calculated as in Eq. (1), and the correlation function is shown in Eq. (2):

where δ_i(x, y) means the phase information loaded into the DMD matrix in advance [23, 24], T(x, y) is target object information, I_i is total light intensity measured by the bucket detector, M is the number of samples, and I means the ensemble average of the system. CS is widely used in the signal process to accurately reconstruct high-dimensional signals from low-dimensional signals [25]. If X ∈ R^N is sparse on a set of bases, it can be expressed as Eq. (3):

where φ denotes a sparse transformation matrix of size N × N, s is the signal after sparse transformation, and it is also the signal from sampling.

The model in this paper is a GG turbulence model [26]. This model is suitable for moderate and strong turbulence and has a wide range of applications and good practicality [27, 28]. The outer-scale effect I_x and the inner-scale effect I_y in atmospheric turbulence are both respectively affected by this distribution. The probability densities of these two effects are expressed as Eq. (4):

Then we can get the whole probability density expressed as Eq. (5):

where α represents large-scale vortexes in the scattering process, β is small-scale vortexes, Γ(∙) is gamma function, K_n(∙) is the second order modified Bezier function of n order, and I is the normalized received irradiance. In Eq. (6) σ21 = 1.23$C_n^2$k^7/6L^11/6, $C_n^2$ is the refractive index structure parameter, and it can be used to evaluate the intensity of atmospheric turbulence. Davis suggested a reference [29], as follows: $C_n^2$ = 10⁻¹⁷m^−2/3 for weak turbulence, $C_n^2$ = 10⁻¹⁵m^−2/3 for moderate turbulence, and $C_n^2$ = 10⁻¹³m^−2/3 for strong turbulence.

It is known that the turbulence behind the target object has little effect on imaging. Therefore, atmospheric turbulence is added to the receiving optical path of the target object. The experimental process is combined according to the principles and formulas mentioned above, and the situation of atmospheric turbulence in the optical path is simulated as shown in Fig. 2.

Figure 2. Schematic diagram of compressed sensing ghost imaging (CSGI) in atmospheric turbulence.

2.2. Network Structure

In this paper, we propose a method based on the generative adversarial network (GAN) and name it TGAN. It has advantages such as being able to save high texture details and create patterns close to real images, so it is the main force in image super-resolution and image restoration [30, 31]. GAN was first proposed by Goodfellow et al. [32]. Most of the DL algorithms before GAN needed many complex operations, and the time cost was very high. The most basic structure of the GAN is composed of one generator and one discriminator, where Z is the factor that causes image distortion such as noise, turbulence and so on. A schematic diagram is shown in Fig. 3.

Figure 3. Schematic diagram of the generative adversarial network (GAN).

The main purpose of this network is to simulate the distribution of actual data. Assume the data distribution is P_T(x), a series of actual data are distributed as {x¹, x², ..., x^m} the generator model distribution is P_G(x;θ), and θ is a parameter that controls generator distribution. In order to simulate actual data more accurately, the whole process is transformed into a problem of calculating maximum likelihood estimation (MLE), The likelihood estimation is L = ${\displaystyle {\prod }_{i=1}^m{P}_G\left(x^i;\theta \right)}$, and a θ* is found to maximize this likelihood estimate, θ^* = arg maxθ ${\displaystyle {\prod }_{i=1}^m{P}_G\left(x^i;\theta \right)}$. There is a way to solve this problem with the GAN, and the formula of the GAN is shown as Eq. (7):

In this formula, the maximum value of V(G, D) represents the difference between the data from the actual distribution and the data from the generator, where the former item in the formula demonstrates the datum from P_T, the latter indicates the datum from P_G, and max V(G, D) is used to optimize the discriminator.

The proposed network structure is shown in Fig. 4. Input an image of 256 × 256, perform one 2D convolution with 64 convolution kernels, a kernel size of 7 × 7, and a step size of 1 to extract the features of the input image, which can reduce the number of training parameters. Speed up the computation by batch normalization (BN) and use a rectified linear unit (ReLu) [33] as the activation function, which is more efficient and simplifies the calculation. Next, there are two down-sampling blocks with the same structure. As is shown in Fig. 4, in each down-sampling block there is a 3 × 3 convolutional layers and a 2 × 2 max pooling layers, which reduces the size of the feature map, simplifies the calculation complexity, and extracts the main features of the image. The last two layers are a BN layer and a ReLu function layer. After the down-sampling block, there are nine Res-Net blocks, and the residual network makes the training faster. In each Res-Net, the input data is first subjected to a convolution layer with 256 convolution kernels, a kernel size of 3 × 3, and a step size of 1. Then, do batch standardization and ReLu activation processing. The data is then passed to a dropout layer with a ratio of 0.5%, that is, 50% of neurons are randomly disconnected to prevent possible overfitting during training. After the data fusion processing, the two transposed convolution blocks are used to restore the small size feature map to the original size. Finally, the output data is passed to a 2D convolution layer with three convolution kernels, a kernel size of 7 × 7, and a step size of 1. Finally, the data is passed to the Tanh activation layer.

Figure 4. The structure of the network.

2.3. Loss Function

The generator network Gθ is trained to generate images. Our purpose is to restore the distorted image I_t to the real image I_T, and the training process is a process of optimizing the loss function.

The loss function is used to estimate the degree of inconsistency between the predicted value f (x) and the true value Y of the model. It is a non-negative value function, usually expressed by L(Y, f (x)), where the smaller the loss function, the better the model robustness. The loss function is a combination of adversarial loss and content loss, expressed as Eq. (8):

In Eq. (8) L_GAN is the adversarial loss and L_C is the content loss.

When training the original GAN (vanilla GAN) it is easy to encounter problems such as gradient disappearance and mode collapse, which is very difficult to train. The Wasserstein GAN (WGAN) proposed later uses the Wassertein-1 distance, making training less difficult. On this basis, the gradient penalty item is added to further improve the stability of training. WGAN-GP realizes stable training on multiple GAN structures, and there is almost no need to adjust hyperparameters. The expression is shown in Eq. (9):

There is content loss, that is, the gap between the assessment of the generated image and ground truth. Two common choices are L1 mean absolute error (MAE) loss, and L2 mean square error (MSE) loss. Recently, perceptual loss was proposed, which is essentially an L2 loss. It calculates the distance between the feature map generated and the ground truth feature map. The expression is shown in Eq. (10):

where φ_i,j represents the feature map output in the i-th max pooling layer and the j-th convolution layer (after activation) after inputting the image into VGG19 [34] (pre-trained on ImageNet). W_i,j H_i,j is the dimension of the feature map.

The TGAN algorithm in this paper uses the TensorFlow 1.1.5 [35] framework. During training, the batch size is 16. We use the Adam [36] to optimize the algorithm, where the learning rate is 0.001,beta1 is 0.9, beta2 is 0.999, and epsilon is 1e-08. After training, the time to restore one 256 × 256 image is about 0.4 s.

The whole system includes an imaging part and an algorithm part. The imaging system compresses sensing GI in atmospheric turbulence (TCSGI). The proposed method uses the GAN to restore previously distorted images, called TGAN. The general process is shown in Fig. 5.

Figure 5. Flowchart of the simulation system. (a) Imaging flow. Set original images as labels, and then input them to a TCSGI model to get distorted images, (b) training flow. If the discriminator shows real images, the generator training is finished; Otherwise, keep training. (c) Testing flow. The test images are input into the trained network to get the predicted images.

In order to show the situation of GI and the feasibility of the proposed method, we conducted simulation programs to analyze the results. The basic data set used in the study is the face image data collection IMDB-Wiki [37] and some images include numbers, letters and characters. The programs TCSGI run in MATLAB 2016b and TGAN algorithm run in python 3.7 are performed on a CX50-G30 with 2×Intel(R) Xeon(R)Gold 6132 CPU @2.60 GHz.

The experimental device is shown in Fig. 6. A laser diode is used as the lighting source, and the retractable lens model is the Nikon AF-S DX 55–200 mm f/4–5.6 G ED (68 mm × 79 mm; Nikon, Tokyo, Japan). A BFLY-PGE-50H5M experimental camera (29 × 29 × 30 mm; Teledyne FLIR, OR, USA) is used, and the acquisition card is the M2i.2030-exp (Spectrum Intrumentation, Grosshansdorf, Germany). The projection matrix is loaded onto the DMD chip in the experiment, and the image to be identified is placed in front of the lens, and then the signal trigger is activated to start the experiment.

Figure 6. Light path diagram of the experimental device.

3.1. Influence of Distance and Intensity on Imaging Quality

In this paper, the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) are used to evaluate image quality. PSNR is generally used for engineering projects between the maximum signal and background noise. The larger the value, the lower the distortion. The mathematical formula is as shown in Eq. (11):

where ${\text{MAX}}_I^2$ is the maximum pixel value possible for the image. If each pixel is 8-bit binary, then it is 255 SSIM is an indicator that measures the similarity of two images where the value range is [0,1], and the larger the value, the smaller the distortion. The formula is as shown in Eq. (12):

Next, different turbulence intensities and suitable transmission distances are set to study the influence of atmospheric turbulence, mainly the transmission distance d and atmospheric turbulence intensity $C_n^2$.

In Fig. 7(a), the measured PSNR values of the image are 15.08 dB, 9.72 dB, 9.42 dB, 8.97 dB, and 7.88 dB. In Fig. 7(b), the measured PSNR values are 14.89 dB, 12.78 dB, 8.40 dB, 8.22 dB, and 7.88 dB. According to the above, it can be found that at a certain intensity of turbulence or transmission distance, there will be a big difference in the imaging results at first, and then it will stabilize. At a certain intensity of turbulence, the increase in transmission distance will reduce the final imaging quality, and similarly, at a certain transmission distance, the stronger the turbulence intensity, the worse the image quality.

Figure 7. Influence of distance and intensity on imaging quality. (a) The turbulence intensity is set as Cn2 = 9 × 10⁻¹⁵m^−2/3, the transmission distance d is set as 500 m, 800 m, 3,000 m, 5,000 m, and 9,000 m, and the sampling rate is N = 100%. (b) The transmission distance d is set as 500 m and the turbulence intensity is set as Cn2 = 9 × 10⁻¹²m^−2/3, 9 × 10⁻¹³m^−2/3, 9 × 10⁻¹⁴m^−2/3, 9 × 10⁻¹⁵m^−2/3, 9 × 10⁻¹⁶m^−2/3 and N = 100%.

3.2. Influence of Sampling Rate on Imaging Quality

To some extent, the sampling rate determines the complexity and quality of the imaging process. The impact of the sampling rate on imaging performance was compared when the sampling rate was 12.5%, 25%, 50%, 75%, and 100%, as shown in Fig. 8.

Figure 8. Effect of different sampling rates on TCSGI.

The results show that at low sampling rates such as 12.5% and 25%, the patterns are almost unrecognizable. At 50%, the shape of images can be seen, and the images become clearer as the sampling rate increases. In TCSGI, as the sampling rate increases, the imaging quality is improved to a certain extent. Therefore, we choose the image with a sampling rate of 50% as the test sample in the process of image restoration.

3.3. Restoration with Filter Methods

In this section, we try to use a traditional filter to restore the image. Consider processing a distorted image with filters and using gray scale images of TCSGI (N = 50%, d = 300 m, $C_n^2$ = 9 × 10⁻¹⁵m^−2/3) as a test set. The simple filters are the median filter, Wiener filter, and Gaussian filter. It has a few effects on some images, but has little effect on those with excessive distortion, as shown in Fig. 9.

Figure 9. Restored by filters (a) ground truth, (b) the distorted images of TCSGI, (c) filtered by a Wiener filter, (d) filtered by a median filter, (e) filtered by a Gaussian filter.

It can be seen that the filters have a certain effect on image restoration, but the effect is limited. Restoration with filters can show the general outline of the image. Both PSNR and SSIM have been improved to some extent, and the test averages are shown in Table 1.

TABLE 1. Average values of peak signal-to-noise ratio (PSNR) and structural similarity index map (SSIM).

Image Quality Assessment Method	TCSGI	Wiener Filter	Median Filter	Gaussian Filter
PSNR (dB)	8.036	13.281	14.335	12.506
SSIM	0.086	0.195	0.252	0.169

3.4. Restoration with Deep Learning Methods

In this section, we mainly analyze the image quality of different imaging methods and need to use some indicators. Here, we use gray scale images by TCSGI (N = 50%, d = 300 m, $C_n^2$ = 9 × 10⁻¹⁵m^−2/3) as a test set, compared with a binary graph. We find that the gray images are seriously affected by turbulence, the quality is poor, and the original images can hardly be seen in the strong turbulence intensity. According to the above, we input the real image as a label in pairs with the distorted image, about 3,000 pairs, and choose 1,000 distorted images as the test set.

In the past related research [38], a deep neural network (DNN), which can complete the fitting task, has been widely used in image processing. We compared the imaging clarity of the four restoration methods to analyze their performance. In test process, three groups of pictures were selected to show, as shown in Fig. 10.

Figure 10. Comparison of different restoration methods. (a) Ground truth, (b) distorted images, (c) filtered by median filter (d) restored by deep neural network (DNN), and (e) restored by TGAN.

We observe the imaging results of median filters, DNN and TGAN. During the imaging process, it is inevitable that we find that there are certain differences between PSNR and SSIM due to different image complexity. We can see the general edges of the face because the main information of the image foreground is relatively large.

Combined with Table 2, Fig. 10 shows TCSGI, median filters, DNN and TGAN, in which the PSNR value of the image processed by TCSGI is between 8–9 dB, and the imaging effect is too fuzzy. Median filters can only slightly recover the fuzzy image, and the imaging effect is limited. For the DNN network, the recovered PSNR value is higher than 16 dB, and the SSIM is also around 0.5, and the imaging results are similar to the original image, but the local area has obvious change. The PSNR value of the TGAN method is more than 23 dB, and the SSIM is more than 0.7, so the similarity between the imaging and the original real image is higher.

TABLE 2. Average values of peak signal-to-noise ratio (PSNR) and structural similarity index map (SSIM).

Image Quality Assessment Method	TCSGI	Median Filter	DNN	TGAN
PSNR (dB)	8.036	14.335	18.729	21.902
SSIM	0.086	0.252	0.537	0.741

Figure 11 shows the imaging results of six groups. Finally, the entropy of image information restored by TGAN and the ground truth are compared. Image information entropy can be used to evaluate the overall information of the image, as shown in Eq. (13):

Figure 11. Peak signal-to-noise ratio (PSNR) and structural similarity index map (SSIM) values of six groups of images. (a) PSNR, (b) SSIM.

where p(i) is the probability that the gray value of the pixel is equal to i. It is found that the performance difference between the two imaging situations is very small, which means that the method can restore the image information well. We take six groups above as an example, as shown in Fig. 12.

Figure 12. Entropy of image information restored by TGAN and the ground truth.

The results show that with the observation of many test results, the overall PSNR value and SSIM value are extremely low for TCSGI imaging, and the imaging quality is very poor. The DNN can roughly restore the image, but the overall imaging quality is not as good as the TGAN imaging method proposed in this paper. TGAN is 17% higher than DNN on the PSNR and increases the SSIM by about 0.3. TGAN has a greater improvement effect on blurred and distorted images, the image quality is better, and the method can restore the image information well, but the texture and details are somewhat unnatural, which is the problem of the GAN.

In summary, we propose a restoration method based on deep learning for CSGI in atmospheric turbulence. The gamma-gamma model is used to simulate the medium to a strong range of turbulent fluctuation, called the TCSGI model. We study the influence of the transmission distance and turbulence intensity on imaging. We also do research and elaboration on the influence of the sampling rate on TCSGI clarity, find that the sampling rate is 50%, and see the outline of the images. We first discuss the effect of filters on restoration, which is limited. Compared with other restoration algorithms, the deep learning methods have excellent performance for image distortion restoration, especially the TGAN proposed. The restored image has good results in clarity and information entropy. It shows that deep learning has a good application in imaging restoration and can solve practical problems in computational imaging.

National Natural Science Foundation of China (No. 62275153, 62005165); Shanghai Industrial Collaborative Innovation Project (HCXBCY-2022-006); projects sponsored by the development fund for Shanghai talents (No: 2021005).

There are no conflicts of interest in the submitted manuscript, and all authors approved the manuscript for publication. The correcsponding author declares on behalf of the co-authors that the work described was original research that has not been published previously, and is not under consideration for publication elsewhere, in whole or in part. All the authors listed have approved the manuscript that is enclosed.

Data underlying the results presented in this paper are not publicly available at the time of publication but may be obtained from the authors upon reasonable request.