In conventional optical microscopy, large field of view (FoV) and high resolution usually cannot be achieved simultaneously. Thus, some computational imaging methods have been proposed to circumvent this physical limit by merging information from multiple images computationally [1, 2]. Fourier ptychography (FP) is one such computational imaging method, which is based on a coded illuminator. Fourier ptychographic microscopy (FPM) achieves intensity and phase images with both high-resolution (HR) and large FoV by combining the FP method with optical microscopy. Recently, FPM has been widely studied in three-dimensional imaging [3], multiplexed imaging [4-6], diffraction tomography [7], fluorescence imaging [8, 9], digital pathology [10], and in vitro cell-culture imaging [11].

The hardware setup of a typical FPM platform involves a simple replacement of the light source for conventional optical microscopy with a programmable LED array, as shown in Fig. 1. The LED matrix is used to successively illuminate the sample at different angles of incidence, and the low-resolution (LR) images are captured sequentially. Due to the limited numerical aperture (NA) of the objective, the high-frequency parts of the sample that exceed the passband are captured as dark-field images. The low-frequency parts of the sample that do not exceed the pass-band are captured as bright-field images. These captured LR dark-field images contain information about features of sub-diffraction-limit size, which correspond to shifted regions of the sample’s Fourier domain. The HR complex image is reconstructed by stitching all of the LR images together coherently in the Fourier domain, using a phase-retrieval algorithm. However, in view of the much lower illumination efficiency of high-angle incident light, the dark-field images always have a low signal-to-noise ratio (SNR), which can significantly affect FP stability and degrade the reconstruction result, based on a previous study [12]. Although some improved algorithms have been proposed recently to minimize the negative influence of noise during FP reconstruction [13-17], they all operate from the perspective of improving an algorithm’s convergence properties, but the noise-reduction parts have not been discussed.

Figure 1. FPM experimental setup

Generally speaking, noise reduction is indispensable for computational imaging methods. In FPM, high data quality can always provide a good reconstruction result and improve convergence speed. In practice, the thresholding method [5] is usually used for FPM data processing. For this method, no prior knowledge of noise characteristics is needed. Several subregions of each dark-field LR image are selected as background to calculate the average intensity level, and this average level is set as a noise-reduction threshold, to distinguish meaningful signal from background noise. Although this method is simple and in some cases works well, the measurement of the average noise level is somewhat ambiguous, because the selection of subregions is usually empirically determined and cannot be applied universally to an arbitrary sample. To solve the uncertainty in background-noise-level measurement in the conventional thresholding noise-reduction method, in this work we propose an improved thresholding method that provides a reliable background-noise threshold for noise reduction. In the proposed method, the noise-reduction threshold for each measured LR image is calculated individually, by comparing the average intensity difference between target image and measured image in every iteration. Our method takes advantage of the FPM calculation framework, and there is no need to select subregions specifically for different samples to calculate the average background-noise level. Experimental results show that the proposed method is more efficient and robust, compare to the conventional thresholding method.

2.1. Principle of FPM

To illustrate our proposed method clearly, we first give a brief review of the basic principle and framework of FPM. Conventional optical microscopy can achieve either large FoV or high resolution, but not both. Usually the sample is scanned mechanically to address this problem. Unlike the mechanical method, the FPM achieves the same space-bandwidth product by scanning the source in the Fourier domain with a programmable LED array. The absence of mechanical scanning makes the FPM technique easy to apply to conventional optical microscopy. By applying FPM, both a large FoV and high resolution can be achieved. In addition, quantitative phase information about a sample can be also acquired [18].

The hardware setup of a typical FPM platform involves a programmable LED array and optical microscopy with a low-NA objective lens. The sample is illuminated from different angles by lighting up each individual LED successively, and the corresponding LR image is captured by the camera. Although the resolutions of the captured LR images individually do not go beyond the objective’s NA limits, they do contain information about sub-diffractionlimit-sized features, which correspond to shifted regions of the sample’s Fourier domain. A resolution surpassing the objective’s diffraction limit can be achieved by coherently stitching these subregions together with a phase-retrieval algorithm in the Fourier domain. The final resolution of FPM is determined by the sum of objective NA and illumination NA.

The FPM framework can be summarized in five steps. First, the FPM method makes an initial guess of the HR sample’s complex amplitude distribution in the spatial domain, u₀(r), where r=(x, y) denotes the coordinates in the focal plane. Usually the amplitude of an LR image with a vertical incident plane wave is up-sampled to generate an initial guess about the HR image. This HR initial guess is then transformed to the Fourier domain. Second, a small subregion of the initial guess’s Fourier spectrum is intercepted and an inverse Fourier transform is applied, to generate an LR target complex image (where the superscript n represents the number of the n^th LR image, and the subscript t denotes the target image). This process is equivalent to a low-pass filtering of the coherent imaging system, the position of the low-pass filter corresponding to a particular oblique illumination angle. Third, following phase retrieval concepts, the target image’s amplitude is updated with the square root of the LR measurement captured under the corresponding illumination angle, (where the subscript m denotes the measured image), while the phase component remains unchanged. The Fourier transform is then applied to the updated target image and is used to replace the corresponding subregion of the HR-sample estimate’s Fourier spectrum. Fourth, steps 2 and 3 are repeated for different illumination angles. Note that a sufficient overlapping proportion of adjacent subregions in the Fourier domain is significant for maintaining data redundancy and convergence. The iterative update process continues for all N images. Finally, steps 2-4 are repeated until either a terminating condition is met, or a fixed number of iterations have been completed.

2.2. Thresholding Method

Due to the lower illumination efficiency of high-angle incident light, the dark-field images always have a lower signal-to-noise ratio (SNR) than the bright-field images. Thus a noise-reduction process has to be applied to the dark-field images before FP reconstruction. The conceivable background noise of the dark-field images can be a mixture of stray light and inevitable noise, such as the image sensor’s dark current, thermal or readout noise, dead pixels, etc. [19-22]. The thresholding method is a method for background noise reduction that is commonly used in FPM data processing. In this method, several small subregions of each dark-field LR image are selected, and their average intensity levels are calculated as a threshold for noise reduction. For the n^th dark-field image, the threshold is calculated as follows:

where (x, y) represents the i^th subregion of the n^th LR dark-field image, and 〈⋅〉 denotes the arithmetic averaging operator. The noise-reduction process is performed by subtracting the noise threshold from the LR dark-field image:

where (r) represents the image to be used for the updating process (the subscript u stands for updating process). To remove the residual background noise, for each processed image, pixels with values below 0 are set to 0 to maintain background continuity.

The principle of the thresholding method is demonstrated in Fig. 2. Figure 2(a) shows one dark-field image of the FPM dataset, and the nonuniform distribution of background noise can be observed. Two subregions (both 100 × 100 pixels) are selected to properly calculate the noise threshold, as shown in Figs. 2(a1) and 2(a2). Figure 2(b) shows the LR image after noise reduction. For the conventional thresholding method, a clean and uniform background can be achieved only when the subregions used for noise-threshold calculation are appropriately selected. However, for samples with complicated distributions, selecting proper subregions from the dark-field images can be ambiguous and inefficient

Figure 2. Thresholding method for background-noise reduction.

2.3. Improved Thresholding Method for Fourier Ptychographic Microscopy

To establish proper thresholds for the datasets of any sample, we first investigate the amplitude replacement and updating processes of the FPM framework in steps 2 and 3. Usually the Fourier-spectrum subregion interception in step 2 is ranked according to ascending order of illumination NA. Thus, at the beginning of each iteration subregions within the objective NA are intercepted to generate the LR target images, and the bright-field images are used for initial updating. Due to the high SNR of the bright-field images, the corresponding updated subregions in the Fourier domain also have good SNR. An assessment of the sufficient overlapping percentage of adjacent subregions is needed for the FPM. When the interception and updating processes come to the beginning of the dark-field regions, the generated target images also have better SNR than the measured images. Therefore, the initially generated target image of the dark-field image tends to be less noisy, and the noise threshold can be found by comparing the difference in arithmetic mean intensity values between the target and measured images:

where represents the average intensity value of the nth target image and 〈(r)〉 is the average intensity value of the n^th measured image. The noise-reduction process in Eq. (2) is applied to the measured image to obtain (r), and is then used for updating in step 3. After applying this noise-reduction approach, the following adjacent subregion to be intercepted and updated in the Fourier domain can also maintain good SNR. This improved SNR consistency means the noise-level-estimation approach can be applied to the remaining update processes. A general SNR improvement of the FPM reconstruction can be ultimately achieved.

Based on the above discussion, our proposed noise-reduction method can be easily integrated with the conventional FPM reconstruction framework. The FPM framework combined with the proposed method is illustrated in Fig. 3. In the actual noise-reduction process, we introduce a general weighting factor α to balance the noise-reduction process:

Figure 3. The FPM reconstruction framework, combined with the proposed noise-reduction method.

Since the SNR of the measured dark-field image is related to exposure time, illumination efficiency, image-sensor performance, etc., it is difficult to unify the SNR performance of different FPM datasets. Therefore, we introduce the weighting factor, to make the proposed method easier to apply to different experimental setups. A larger value of will α enhance the noise-reduction performance, but lose more of the high-frequency information in the dark-field images. In general, a value of α in the range of 1 to 1.05 shows a nice balance between noise-reduction and reconstruction-resolution performance, as will be demonstrated in the next section.

For most noise-reduction methods, the noise-reduction performance and capability to preserve details are contradictory. Thus it is necessary to investigate the resolution performance of the FPM combined with different noise-reduction methods. We first apply the conventional thresholding method and our proposed method to the public FPM dataset of a USAF 1951 resolution target (downloaded from http://www.laurawaller.com/opensource/) for comparison. This dataset is captured under a 4×, 0.1-NA objective lens, and contains 293 LR images. The detailed experimental setup can be found in Ref. [5]. The EPRY algorithm [13] is popularly used for a robust FPM reconstruction process that recovers the HR complex distribution and pupil function jointly. In this work, the EPRY algorithm is combined with noise-reduction methods for testing purposes. Note that all of the noise-reduction processes are only applied to the LR dark-field images.

Figure 4 demonstrates the FPM reconstruction results for the conventional thresholding method, with different subregions selected for noise reduction. Figure 4(a) shows the full FoV of one LR bright-field image, and a small segment (100 × 100 pixels, as shown in Fig. 4(a1)) is cropped for reconstruction and comparison. Panel groups (b), (c), and (d) show the reconstructed amplitude, phase, and pupil function respectively, using the EPRY-FPM for 10 iterations. Three pairs of different subregions (100 × 100 pixels, marked with green, yellow, and blue boxes, as shown in Fig. 4(a)) are selected for noise estimation, and the corresponding reconstruction results are placed in the same colored boxes. In contrast, (b1)-(d1) show the results without any noise reduction. It is clear that the reconstruction without noise reduction is significantly degraded by the intense noise and resulting nonuniform background distribution. By applying the thresholding method, the reconstruction quality can be improved, as shown in the rest of the reconstruction results in Fig. 4. However, it is also clear that the reconstruction results obtained with the conventional thresholding method can be easily affected by the selection of subregions used for noise estimation. Therefore, FPM reconstruction with the conventional thresholding method cannot always maintain good consistency.

Figure 4. Reconstruction results using the conventional thresholding method, with different subregions selected for noise reduction.

The FPM reconstruction results with the proposed noise-reduction method are presented in Fig. 5. The same small region is used for reconstruction, with the corresponding α ranging from 0.95 to 1.15. As in Fig. 4, panel groups (a), (b), and (c) show the recovered amplitude, phase, and pupil function respectively. By comparing the reconstruction results, it is easy to see that when the weighting factor α is less than 1, the residual noise cannot be removed effectively, resulting in artifacts in the recovered amplitude and phase images. On the contrary, when α is greater than or equal to 1, the artifacts fade away and the backgrounds of reconstructed amplitude and phase maintain good uniformity. By comparing the reconstruction results, it is easy to see that the proposed method maintains better noise-reduction consistency when the weighting factor is not less than 1. Moreover, the reconstructed amplitude and phase images using the proposed method have improved background uniformity.

Figure 5. Reconstruction results using the proposed noise-reduction method with the corresponding weighting factor ranging from 0.95 to 1.15. Panel groups (a), (b), and (c) present the recovered amplitude, phase, and pupil function respectively, using the EPRY-FPM for 10 iterations.

To further investigate the noise-reduction and resolution properties of our proposed method, the phase distributions for different weight factor values along the dashed lines indicated in Figs. 5(b1)~5(b5) are shown in Fig. 6. Here the background is taken as the zero-phase point. The phase-distribution curves in Fig. 6 clearly show nine sharp peaks along the horizontal axis, corresponding to the nine scribed bars in the USAF resolution target. As for the red curve, for which the weighting factor α is less than 1, many parts of the curve below the zero-phase point can be seen. This is because more residual noise is preserved, which degrades the consistency of the recovered phase. For the remaining curves in Fig. 6, when the weighting factor is greater or equal than 1, the recovered phase distributions maintain good consistency. It can also be observed that the contrast of the phase distribution decreases slightly as α increases. This is due to more of the meaningful signal being removed with higher weighting factors. Thus, in practice, the weighting factor can be flexibly adjusted to balance noise reduction and resolution.

Figure 6. The phase distribution for different weighting factors, along the dotted lines in Figs. 5(b1)~5(b5).

In addition, the proposed method was also tested with biological samples, based on our experimental setup shown in Fig. 1. This setup is modified from a standard optical-microscopy configuration (ZEISS Axio Scope) by replacing its illumination component with a 16 × 16 array of WS2812 programmable RGB LEDs (distance between adjacent LED elements is 10 mm, the green channel is used for experiments; the central wavelength for green is λ = 525 nm). The optical system is equipped with a 10×, 0.25-NA objective lens, and an Arduino UNO R3 SCM is used to control the LED array. The LED array is placed 100 mm below the sample, and is synchronized with the external trigger of the camera (Point Grey, CM3-U3-50S5M-CS). The camera in the experiment has 5 megapixels (2448 × 2048 resolution, pixel size 3.45 µm) and a 12-bit output depth. For all experiments reported in the following section, a circular LED area is used for illumination and the diametrical number of the rounded LED is 15, resulting in a final synthetic NA_syn of approximately 0.84 in the lateral directions. The theoretical maximum resolution of our setup is λ/NA_syn ≈ 0.625 µm.

In this work, a blood smear with a relatively complex background distribution is used for the FPM sampling. This dataset contains 177 LR images, including 9 bright-field and 168 dark-field images. In Fig. 7 we present the reconstruction results for the dataset, for different treatments. A small section (300 × 300 pixels, as shown in Fig. 7(a1)) is cropped from the full FoV for reconstruction and comparison. Panel groups (b), (c), and (d) show the reconstructed amplitude, phase, and pupil function respectively. For the conventional thresholding method, two subregions are randomly selected (100 × 100 pixels each, marked with green boxes, as shown in Fig. 7(a)) and their average intensity value is calculated as the noise-reduction threshold, per Eq. (1). (b2)-(d2) show the reconstruction using the conventional thresholding method, while (b3)-(d3) show the results using the proposed method with weighting factor α = 1.01. The reconstructed amplitude and phase distributions suggest that, compared to the conventional thresholding method, our proposed method provides clearer detail in the sample and a more uniform background. We also investigated the variation of noise-level estimation of the proposed method with iteration. The noise-level estimate for each dark-field image along with the number of iterations is shown in Fig. 8.

Figure 7. Reconstruction results for a blood smear, for different treatments.

Figure 8. Noise-level estimation of each dark-field image, along with number of iterations.

In Fig. 8, all the dark-field images are labeled according to ascending order of illumination NA. It can be seen that in the early iterations the noise-level estimates are relatively high, and as the iteration time increases the noise levels slowly decrease and finally reach stable levels. This phenomenon is predictable, according to the FPM model. During the early iterations, due to a lack of high-frequency components in the HR-spectrum estimation, the generated LR target images have relatively low energy, resulting in high noise-level estimates. As the number of iterations increases, however, the high-frequency parts of the HR spectrum are gradually reconstructed and finally converge. The noise-level estimates also ultimately reach stable values. This property of gradually decreasing noise-level estimation is efficient for balancing noise reduction with preserving detail, as demonstrated by earlier experimental results.

Based on the above experimental comparison, it is clear that the proposed method maintains good resolution and noise-reduction performance when it is applied to a FPM dataset captured with different experimental setup. We would like to note that the selection of the weighting factor is not very sensitive to different FPM datasets, according to our experiments. In general, to balance the backgroundquality and resolution performance, a weighting factor value in the range of 1 to1.05 is recommended.

In summary, we have proposed a background-noise-reduction method for Fourier ptychographic microscopy, based on an improved thresholding method that estimates a reliable background-noise threshold for the noise-reduction process. Our proposed method can be easily combined with the FPM calculation framework, and the noise-reduction and reconstruction-resolution performance can be flexibly balanced by adjusting only one weighting factor. Experimental results show that the proposed method yields clearer details of the sample and a more uniform background, compared to the conventional thresholding method. It is also demonstrated that the proposed method possesses good noise-reduction consistency for different samples. Therefore, by applying the proposed method, no preprocessing or prior knowledge of the sample dataset is needed; this improves the applicability and robustness of the FPM technique.