Most of the human eye’s lesions, such as macular degeneration and glaucoma, manifest in the retina. Additionally, there are many systemic diseases that affect the retina, such as diabetes [1, 2]. It is difficult to obtain high-resolution retinal cell and micro-vessel images with traditional optical imaging systems due to aberrations [3]. Adaptive optics (AO) technology, as an effective way to solve the problem, has been investigated by many researchers [4–14].

In 1997, Liang et al. [15] presented an AO imaging system for human retinal cells for the first time. In 2001, Fernández et al. [16] completed the real-time closed-loop wavefront correction of the human eye. Jian et al. [17] proposed wavefront sensor-less AO optical coherence tomography (OCT) and obtained images of mouse retinas. The final resultant wavefront had root mean square (RMS) of 52.2 nm, and the whole process took approximately 65 s. In 2009, Mu et al. [18] used a liquid crystal spatial light modulator (LC-SLM) to correct the high- and low-order aberrations of the human eye and obtained high-resolution images of retinal cells and micro-vessels [18, 19]. Wahl et al. [20] developed an open-loop sensorless AO retinal imaging system and used the coordinate search (CS) algorithm for large field-of-view imaging of the retina. The algorithm required two to three iterations to correct aberrations and each iteration took 12 s for 400 × 100 sampling. Azimipour et al. [21] used an AO OCT to measure the functional response of individual cone cells. Qin et al. [22] used an AO imaging system to investigate biological structures in the retina.

In recent years, the application of neural networks in AO systems has been investigated by many researchers [23–27]. Xu et al. [23] used a self-learning control model based on gradient back-propagation to control a deformable mirror (DM). Ma et al. [24] used the improved AlexNet to establish the relationship between light-intensity images and Zernike coefficients for aberration correction, greatly improving the Strehl ratio. In 2017, Fei et al. [25] used convolutional neural networks (CNN) to recover AO retinal images, eliminating the need to predict the point spread function (PSF) of the imaging system. However, in practical applications, closed-loop AO correction takes longer, and the traditional open-loop correction method has low accuracy in correction due to the nonlinear response of the DM.

In this paper, an open-loop wavefront correction method based on SH-U-net is presented for an AO retinal imaging system with the aim of enhancing the speed and accuracy of wavefront correction. In section 2, the principle of the traditional wavefront correction and SH-U-net are given. In section 3, the simulation results are analyzed and compared with other methods. Experimental results are given in section 4. Finally, conclusions are given in section 5.

2.1. Traditional Wavefront Correction Method

In the traditional wavefront correction algorithm [28], it is necessary to measure the interactive function of the DM. The corresponding slopes are measured using a Shack-Hartmann wavefront sensor (SHWFS) and the interaction matrix S of the DM’s actuators can be calculated as follows:

where N is the number of effective actuators of the DM, and M is the number of SHWFS microlens.

The response of the DM is generally assumed to be linear, and the measured slopes with SHWFS are assumed to be proportional to the voltages applied to the actuators. Therefore, the slope vector g of the aberrated wavefront measured by the SHWFS can be expressed as:

where S is the interactive matrix representing the fundamental response relationship of the control voltage, and V is the voltage vector applied to the DM. The measured response functions for DM69-15 (ALPAO Co., Montbonnot, France) with 69 actuators in our lab are shown in Fig. 1. Because of the DM’s nonlinear response, a small gain for the traditional closed-loop correction in the AO system is necessary [29].

Figure 1. Influence function of deformable mirror.

2.2. System Model Based on SH-U-net

In traditional closed-loop wavefront correction, several iterations are necessary to compensate for aberration. However, in the application of retinal cell AO imaging, closed-loop wavefront correction is not stable when the light is blocked during blinking. Sometimes, light spots with a low signal-to-noise ratio reflected from the retina also lead to poor closed-loop wavefront correction because of large wavefront measurement errors for traditional weight-of-center algorithms. Therefore, open-loop wavefront correction based on the SH-U-net is presented in Fig. 2. The deep learning network used in the work is an improved net based on U-net for a Shack-Hartmann wavefront sensor. We call the network SH-U-net. The light spots image from the SHWFS is used as the input for the SH-U-net network model as the control module, and the SH-U-net network model could calculate the control voltages of the DM from the input light spots image. By applying different control voltages to each actuator of the DM, the AO imaging system could correct the aberration and obtain high-resolution images of objects.

Figure 2. Wavefront correction based on SH-U-net.

The SH-U-net is shown in Fig. 3(a). To use in retinal aberration compensation, the U-net network [30] was improved. First, to enhance the voltage prediction accuracy of the SH-U-net, a module was incorporated for residual connection in the encoding part [31]. The residual block enables deep networks to better learn complex feature representations and helps to effectively train these networks with the backpropagation algorithm. In addition, the use of residual blocks can also reduce the number of parameters in the network and improve the computational efficiency of the model. The output of one convolution layer is concatenated with the output of the subsequent convolution layer, allowing for more effective feature extraction from the image. This also facilitates the propagation of gradients during the back-propagation process, making it easier for the gradient of small loss to reach the shallow neurons. Second, the parameters of each convolution layer are adjusted in detail. We increase the size of the convolution kernel to increase the receptive field. The encoding part consists of two 3 × 3 convolution layers, one 4 × 4, one 9 × 9 and two 2 × 2 pooling layers. In the decoding part, the image is first convoluted to reduce the depth of the feature by half without changing its width and height. It is then combined with some features extracted from the compression channel to form a feature map twice the size. Following this, two convolution layers are used to further extract features. Then, there is no fully connected layer in the network, and only the effective parts of each convolution layer are used.

Figure 3. Network structure and hysteresis loop: (a) SH-U-net control model, (b) measured peak-to-valley (PV) of tilt and tip as a function of theoretical ones with applied voltage.

For the ALPAO DM with 69 actuators in our laboratory, the measured hysteresis is shown in Fig. 3(b). In a closed-loop correction system, the voltages of the actuators are updated in real time, and wavefront correction error due to hysteresis of the DM could be overcome. However, in an open-loop AO system, the wavefront correction error induced by hysteresis could not be measured and compensated. The hysteresis effect observed in the DM is up to 7.7% as shown in Fig. 3(b), which can significantly affect the precision of wavefront correction in an AO open-loop system. To solve the problem, the SH-U-net takes into account the nonlinear and hysteresis response of the DM by obtaining the dataset that includes applied voltages of the DM and measured wavefront data obtained from the SHWFS. Based on the measured hysteresis dataset training, SH-U-net builds a pre-training model. In addition, the Sigmoid function is used as the activation function. The model can describe and predict the nonlinear response due to the hysteresis effect by appropriately adjusting the parameters of the Sigmoid function, such as slope and offset. This approach helps minimize the effect of hysteresis and improves the accuracy of wavefront correction. The least squares error as the loss function is used to evaluate and optimize the performance of the model. The loss function is given as follows:

where y_predict is the predicted voltages and y_true is the true inputted voltages. The loss function is used in the simulation to minimize the sum of all squared differences between the true and predicted values.

2.3. Characteristics of Human Eye Aberrations

The aberrations of the human eye are dominated by low-order aberrations, such as defocus and astigmatism. According to a statistical analysis of the wavefront aberration distribution of 109 test subjects by Porter et al. [32], the proportion of low-order aberrations in human eye wavefront aberrations is about 85–90%, and the remaining high-order aberrations are only 10–15%. In addition, aberrations of the human eye are also dynamic [33–35]. The human eye is not completely static when observing an object. Eye saccade [36, 37] introduces about 1 mrad. Drifts and microsaccades are less frequent, but larger in magnitude, up to 10 mrad. The training dataset is generated according to these aberration characteristics.

In the simulation, the SHWFS has a microlens array of 15 × 15, and there are 10 × 10 pixels for every microlens. The response of the DM was approximated using the Bessel function, which provided a reasonable estimation of the DM’s behavior. To train the SH-U-net, aberrations with different amplitudes and Zernike modes were generated based on the characteristics of ocular aberrations. A total of 100,000 sets of data were generated and 90,000 sets were used for training the SH-U-net model, and the rest were used for testing its performance. The simulations were completed on a PC with a 3.20 GHz Intel(R) Core i7-8700 GPU, 48 GB RAM, and NVIDIA GeForce RTX 1070 graphics.

After training, the SH-U-net achieved an accuracy of 98.95% in predicting the voltages required for wavefront correction. This high accuracy demonstrates the effectiveness of the model in accurately predicting the necessary control signals for the DM. Figure 4 shows the results of the predictions of the model and the wavefront correction effect. In Fig. 4(a), an original wavefront randomly selected from the test dataset is given. The light spots image corresponding to the wavefront was inputted into the SH-U-net, and the corresponding DM control voltages were predicted, as shown in Fig. 4(d). Driven by this voltage, we can obtain the corresponding surface shape of the DM in Fig. 4(b). The residual wavefront, which represents the difference between the original wavefront and the corrected one, is shown in Fig. 4(c). The wavelength is 785 nm in the simulation, and its RMS is 0.08λ, and peak-to-valley (PV) is 0.44λ. Figure 4(e) shows the voltage prediction error of the SH-U-net. The prediction error is less than 0.01 V, which indicates that the trained SH-U-net model predicts voltages accurately. From these simulation results, the accuracy of the SH-U-net model in predicting control voltages is verified, which also validates its effectiveness in wavefront correction.

Figure 4. Wavefront correction with SH-U-net: (a) Original wavefront, (b) predicted wavefront, (c) residual wavefront, (d) predicted voltage, and (e) voltage error.

The same test dataset was also used in iterative wavefront correction using the classic PID controlling algorithm, and the results are shown in Fig. 5. The original wavefront is shown in Fig. 5(a), and the residual wavefront results after 5 and 16 iterations of AO correction are shown in Figs. 5(b) and 5(c), respectively. Figure 6 shows the RMS and PV as a function of the AO iterative number. The blue curve represents the RMS, and the red one represents the PV. The RMS of the residual wavefront is decreased from 0.31λ to 0.08λ, and PV is decreased from 1.40λ to 0.44λ. Although the SH-U-net corrects the wavefront only one time, its residual wavefront after AO correction is smaller than the traditional method.

Figure 5. Residual wavefront obtained using PID algorithm: (a) Original wavefront, (b) wavefront after 5 iterations, and (c) wavefront after 16 iterations.

Figure 6. Root mean square (RMS) and peak-to-valley (PV) during iteration.

Since everything is ideal during the simulation, the corrections are all close to the diffraction limit. In a real test, there will be a difference between the results of the two correction methods.

To get the AO correction results statistically, 1,000 sets of test data were used to verify the wavefront correction performance of the SH-U-net. The RMS of the original wavefront and the compensated wavefront are shown in Fig. 7. The red dots represent the RMS of the original wavefront, and the black dots represent the RMS of the compensated wavefront. It can be seen that the RMS of the original wavefront is around 2.55λ–5.10λ, and the average is 3.67λ. The RMS after compensation is around 0.19λ–0.57λ, and the average is 0.34λ. This indicates that the prediction is stable and the correction is accurate.

Figure 7. Root mean square (RMS) of original wavefront and compensated ones.

To validate the AO wavefront correction method based on the SH-U-net model, a retinal imaging AO system was built in our laboratory [38]. A schematic diagram of the optical layout of the experimental AO imaging system is shown in Fig. 8.

Figure 8. Schematic diagram of experimental optical layout. DB, dichroic beam splitter; BS, beam splitter; L, lens; CCD, charge-coupled device; SHWFS, Shack-Hartmann wavefront sensor.

A laser with a wavelength of 785 nm generates a wavefront that enters the model eye through the lenses L2, L3, and L4. The dotted box in the Fig. 8 is the Badal system, which can image human eyes with different diopters. The wavefront light spots are collected by WFS150-7AR SHWFS with 29 × 29 microlens array (Thorslabs Inc., NJ, USA). For retinal imaging, light with 635 nm enters the model eye through L1, L3 and L4. After being reflected by the model eye, the light passes through L4, the Badal system and L5 to the DM (ALPAO DM-69, the number of actuators is 69, and the clear aperture is 10.5 mm; ALPAO). After being reflected by the DM, the light beam reaches the charge-coupled-device (CCD) (PCO Co., Munich, Germany, 1,024 × 1,024 pixels, pixel size is 4.65 μm × 4.65 μm) through L5, L6 and L7. The experimental optical layout is shown in Fig. 9. And the optical instrument parameters are shown in the Table 1.

Figure 9. Experimental optical layout.

TABLE 1. Optical instrument parameters.

Instrument			Parameter
Laser Source (nm)		Upper	785
		Left	635
SHWFS			WFS150-7AR, 29 × 29 Lenslets Array
DM			ALPAO DM-69 (Number of Drivers = 69/ Clear Aperture = 10.5 mm)
Lenses Width (mm)	L1		30
	L2		300
	L3		125
	L4		200
	L5		75
	L6		75
	L7		125

Firstly, 5,000 sets of random control voltages ranging from −1 to 1 were generated for DMs. Secondly, a WFS as shown in Fig. 9, was used to measure the reference wavefront when no voltage was applied to the DM, and then control voltage was applied to the DM to collect 5,000 sets of corresponding wavefronts. The reference wavefront was subtracted from the one measured wavefront to obtain the wavefront and light spots corresponding to the DM under a specific set of control voltages. Next, 4,500 sets of data were used for training, and the other 500 sets were used for testing. Network model training was completed, and the test results are shown in Fig. 10.

Figure 10. Test results after training: (a) Real voltage, (b) predicted voltage, and (c) prediction error.

Figures 10(a)–10(c) show the real voltage in the experiment, the predicted voltage by the network after inputting the light spots, and the error value between the real voltage and the predicted one with an RMS of only 0.023 V, respectively. It can be seen that the predictions of the network are accurate enough. The DM is driven by this set of voltages to correct the aberration, and the residual wavefront is shown in Fig. 11. Figure 11(a) shows the original distorted wavefront with an RMS of 11.05λ and PV of 22.94λ. The distorted wavefront is generated by a liquid crystal spatial light modulator. Figure 11(b) shows the AO-corrected residual wavefront with an RMS of 0.88λ and PV of 5.47λ. It took 50 ms to predict and correct. The image before and after AO correction is shown in Figs. 11(f) and 11(g), respectively. A USAF 1951 resolution test pattern is used as the object. The group number of the resolution target is 3 and the resolution is 8.9797 lp/mm after correction by our method. It can be seen that the image quality has been significantly improved after open-loop AO correction based on the SH-U-net model, thus verifying the feasibility of the correction method. However, due to component accuracy and system errors, we cannot achieve the theoretical resolution of 25 lp/mm.

Figure 11. Distorted wavefront, residual wavefront and corresponding image of Negative 1951 USAF Target: (a) Distorted wavefront, (b) residual wavefront with SH-U-net, (c) residual wavefront after 15 iterations with closed-loop correction, (d) residual wavefront after 100 iterations with closed loop correction, and (e) residual wavefront with open-loop correction. (f), (g), (h), (i) and (j) are images of the corresponding wavefronts, respectively.

As a comparison, the results for the traditional closed-loop and open-loop AO correction are also shown in Fig. 11. In our experiments, open-loop means to correct a wavefront only one time. Figures 11(c) and 11(d) show the results of closed-loop correction. Figure 11(c) is the wavefront after 15 iterations with an RMS of 1.73λ and PV of 7.67λ. After 100 iterations, the residual wavefront is as shown in Fig. 11(d), with an RMS of 1.00λ and PV of 4.36λ, which is close to that of the SH-U-net model. The whole process took about 30 s. It can be seen in Figs. 11(h) and 11(i) that the resolution of the image is continuously improved during correction, and the imaging effect after 100 iterations is close to that in Fig. 11(g). Figure 12 shows RMS and PV as a function of the iterative number during AO closed-loop correction. It can be seen that RMS in the first 20 iterations decreases rapidly, the curve converges after about 40 iterations, and the changes in RMS in the last 50 iterations are small.

Figure 12. Root mean square (RMS) and peak-to-valley (PV) during iteration.

After traditional open-loop correction, that is, after it is corrected one time with a proportional parameter of one, the residual wavefront is shown in Fig. 11(e) with an RMS of 1.52λ and PV of 7.22λ. The effect of open-loop correction is shown in Fig. 11(j). It can be seen from the image of the USAF 1951 resolution test pattern that traditional open-loop correction has a certain correction effect, but the correction accuracy is not very high, and it is significantly lower than that of the SH-U-net method. This further verifies the feasibility of our proposed method.

A total of 500 sets of experimental data were used to correct with SH-U-net, and the measured results obtained are shown in Fig. 13. The red triangle represents the wavefront before correction, and the blue circle represents the wavefront after correction. The results indicate that our method can perform effective correction. Smaller original aberrations and more experimental results for the SH-U-net training would improve the wavefront correction precision further.

Figure 13. 500 sets of wavefront correction results.

The SH-U-net model for open-loop wavefront correction is presented and used in adaptive optics retinal imaging. After data training, the network has a direct nonlinear mapping relationship between the light spots on the wavefront and the control voltage of the DM. Therefore, the accurate control voltage of the DM could be calculated from the light spots of the wavefront sensor. Theoretical and experimental results validate the feasibility of the method. Compared with traditional closed-loop and open-loop AO correction algorithms, it can complete the correction of human eye aberration with higher precision in a shorter time, making it a promising application prospect in many fields.

National Natural Science Foundation of China (No. 61475152, No. 62205127); Fund for Key Laboratory of Electro-Optical Countermeasures Test & Evaluation Technology (GKCP2021001).

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.