Before the advent of spectral imaging, the images obtained by traditional imaging technology were usually grayscale or color images. In such cases the amount of information was relatively limited, and the related processing or analysis tasks were mostly based on spatial information such as grayscale and texture distribution, thus making it impossible to distinguish ground-object categories from images [1]. With the development of imaging techniques that integrate spectral information into image data on the basis of traditional optical imaging techniques, spectral imaging has aroused much interest. Spectral imaging solves the “only the image without spectrum” or “only the spectrum without image” problems of traditional optical imaging technology, and makes it possible to analyze the object categories in an image [2, 3].

Hyperspectral imaging technology is an advanced image-data acquisition technology that records the spatial and spectral information of ground objects through an imaging spectrometer [4]. Hyperspectral images contain two-dimensional spatial and one-dimensional spectral information; Thus hyperspectral data can be considered three-dimensional data space. The spectral resolution of hyperspectral imaging technology is usually below 10 nm, and its number of spectral bands can reach the hundreds. Such technology can obtain the diagnostic spectral features of ground objects, thus achieving accurate classification. In recent years, hyperspectral imaging (HSI) has rapidly developed in various fields, due to the development of science and technology and the practical needs in the information field.

By extracting spectral and spatial information contained in HSIs, researchers obtained many research results regarding spectral demixing [5], feature extraction [6], and target detection (TD) [7]. The spectra of different ground objects in HSIs are often quite different; Therefore, different types of ground objects can be diagnosed in HSIs. The field of TD has received a great deal of attention. TD is often based on similarity measurements, establishing a similarity relation between the test spectrum vector (the spectral vector of unknown pixels in HSI) and the reference spectrum vector (the spectral vector of the known target). With that, the target distributions with high levels of similarity to the reference spectrum can be obtained [8].

In the ideal state, the category and spectrum of the target have a one-to-one correspondence; That is, the spectra should be the same if the target types are the same, while the spectra are different when the target types are different. However, achieving this state is impossible in practice. Traditional space-based imaging methods have high altitudes and fixed times, and their detection directions are basically perpendicular to the ground. Thus hyperspectral data obtained are less affected by imaging conditions [9], and the spectrum is relatively stable. Land-based imaging methods typically use small unmanned aerial vehicles, or human-powered methods. Imaging time and direction are arbitrary, and the imaging conditions clearly affect the hyperspectral data. As a result, the uncertain characteristics of the spectra of targets are significant, usually manifesting as same object, different spectrum.

The spectral curves of targets show uncertainty when imaging conditions change. On the other hand, the spatial distribution of targets is usually not uniform or regular, and obvious uncertainties exist when taking the whole large-area average spectrum as the spectrum of the targets. This approach weakens TD accuracy, in terms of the similarity between the test and reference spectrum vectors. Therefore, it is worth discussing the effectiveness of the TD method under various land-based imaging conditions. Traditional detection evaluations are often based on detection results for a single imaging condition, without taking into account the influence of imaging conditions on hyperspectral data, which results in TD methods that may not be directly applied to land-based imaging conditions. Therefore, from the perspective of applicability, detection effectiveness should be analyzed to find a new evaluation method that can comprehensively evaluate performance under land-based conditions. Subsequently, a universal detection method that can be adapted to different imaging conditions with better results will be needed.

To solve this problem, on the premise of fully analyzing the advantages and disadvantages of detection and evaluation methods, this paper proposes the concepts of the stability coefficient (SC) and effective area under the curve (EAUC) to comprehensively evaluate the TD effectiveness under land-based imaging conditions, while taking into account both detection accuracy and stability. The experimental results show that the evaluation results of this method are consistent with those of the qualitative analysis, and have good universality.

Section 2 summarizes the principles of classical TD, and analyzes the strengths and weaknesses of each method. Section 3 introduces the principles of the classic evaluation methods. Section 4 analyzes the performance of the classical detection methods under various imaging conditions, and analyzes their applicability using the EAUC value. Section 5 summarizes the content of this paper.

The basic form of TD in HSI is a binary hypothesis-testing problem, formulated as Eq. (1):

where x represents the known reference-spectrum vector, y represents the unknown test-spectrum vector, η represents the threshold, and D(x, y) represents the detection function. The TD methods work by comparing the known target spectrum to the test pixel spectrum in an HSI. The pixels with high levels of similarity are regarded as target pixels, and the pixels with low levels of similarity are regarded as background pixels.

In general, TD methods can be categorized into projection-based, distance-based, information-based, and statistics-based methods. In addition, the constrained-energy-minimization (CEM) method is also a classical TD method [10] and is widely used. The idea behind this method is to enhance the information in the direction of interest and suppress the information in the other direction, thus highlighting the target.

There are various methods through which to improve classical TD methods. However, most of these methods are based on the similarity across spectral vectors. This paper takes traditional TD methods as examples, and analyzes their applicability under land-based imaging conditions. A brief introduction to classical TD methods is presented following section.

2.1. Projection-based TD Methods

A projection-based TD method mainly exploits the shape differences between the reference-spectrum vector and the test-spectrum vector. This method is insensitive to differences in spectral amplitude, can overcome the influence on TD of the amplitude differences caused by light intensity, terrain, shadows, particle size, and other factors, and has the characteristic of multiplier-factor invariance. The disadvantage of this method is that it can measure only the difference in overall shape between the reference-spectrum vector and test-spectrum vector, and cannot reflect local differences; Usually the contrast of the detection results is not at a high level.

Projection-based TD methods include the spectral-angle-metric (SAM) [11] method, the spectral-angle-cosine (SAC) method, the spectral-gradient-angle (SGA) method, the normalized SGA (NSGA) method, the kernel-spectral-angle (KSA) method [12], and the orthogonal-projection-divergence (OPD) method [13]. Among them, the SAM, SAC, SGA, and NSGA methods are the most classic. The SAM method reflects the shape differences between spectral vectors and measures the inclusive angle between reference-spectrum vector and test-spectrum vector. The SAM method is defined as Eq. (2):

where x^T and y^T represent the transpositions of x and y respectively. The SAM values fall in the range [0, π]. The smaller the value is, the higher the degree of similarity. Similarly the SAC method converts the range to [−1, 1]. The higher the value is, the higher the degree of similarity between reference-spectrum vector and test-spectrum vector. The SAC method is defined as follows:

The SGA method measures the similarity between spectral gradient vectors [14], as defined in Eq. (4). This method takes into account the tilt of the vector, and is robust to geometric distortions and intensity variations. As mentioned, the range of the SGA value is [−1, 1]; To convert its range to [0, 1], the NSGA method is defined as in Eq. (5):

where x′ = [x₂ − x₁, x₃ − x₂, ..., x_n − x_n−1], y′ = [y₂ − y₁, y₃ − y₂, ..., y_n − y_n−1]. x_i and y_i represent the reflectivity of the i^th spectral band of x and y, respectively. n represents the number of spectral bands. The spectral gradient reflects the change in the slope of the spectral vector, and describes the morphological features of the spectral vector. Typically these changes are associated with the absorption features of the targets, which are essential features of an image.

2.2. Distance-based TD Methods

Distance-based TD methods assume that the spectrum is a higher-dimensional vector in Euclidean space, where the dimension is denoted as the number of spectral bands. The TD problem is transformed into a similarity-measurement problem for distance: the smaller the distance, the greater the degree of spectral similarity. The principle of this method is simple, and there are many correlation-distance indices with which to describe measurement for distance. In fact, other types of detection methods are similar to distance-based detection methods, and their results can be transformed into one another. Compared to projection-based methods, a distance-based method has difficulty overcoming the influence on TD of light intensity, terrain, shadows, particle size, and other factors, especially when the target is in a shadow.

Distance-based TD methods include the Euclidean-distance (ED) method, the normalized ED (NED) [15] method, the Mahalanobis-distance (MaD) method [16], the Tchebyshev-distance (TcD) method [17], and the Hamming-distance (HD) method [18]. The ED method is a frequently used distance-based TD method, but its results are largely dependent on spectral amplitude and are insensitive to differences in shape between test-spectrum vector and reference-spectrum vector. To solve this problem, the NED method is defined in Eq. (6):

where $\tilde{x}$ = x / ||x||, $\tilde{y}$ = y / ||y||. This method normalizes the spectral vector before calculating the ED, such that the range of NED values is [0, 1]; The smaller the value, the more similar the two spectral vectors. The degree of robustness is enhanced after normalization.

2.3. Information-based TD Methods

Information-based TD methods are based on evaluating the information-entropy (IE) characteristics of the spectral vectors. This method analyzes the similarity between vectors based on the amount of information provided, and is not affected by amplitude or shape. However, the false-alarm rate is higher in the detection results of this method, compared to that for other methods.

IE describes the uncertainty of a signal and reflects the amount of information it carries. Mutual information (MI) is used to describe the correlation between two systems; that is, the amount of information contained in both systems. The MI between the test-spectrum vector and the reference-spectrum vector is defined as Eq. (7):

where $p(x_i)=x_i/{\displaystyle \sum _{i=1}^n{x}_i}$, $p(y_j)=y_j/{\displaystyle \sum _{j=1}^n{y}_j}$. p(x_i, y_j) represents the joint probability density of x_i and y_i. The larger the value of MI, the larger the amount of duplicated information, and thus the greater the correlation between the two spectral vectors and the more similar the spectra.

2.4. Statistics-based TD Methods

Statistics-based TD methods measure similarity by calculating the correlation between vectors [19]. These methods are not affected by light intensity, shadows, or other conditions. Representative methods include the normalized-correlation-coefficient (NCC) and spectral-correlation-angle (SCA) methods. The NCC and SCA methods are defined as follows:

where

where cov(x, y) represents the covariance between x and y. R_xy represents the correlation coefficient between x and y. σ_x and σ_y represent the standard deviations of x and y respectively. The range of the NCC value is [0, 1]; The larger the value is, the higher the degree of similarity. The range of the SCA value is [0, π / 2]; The smaller the value is, the higher the degree of similarity.

2.5. CEM method

The principle of the CEM method is to design a linear filter vector w = [w₁, w₂, ..., w_n]^T that minimizes the energy output of the hyperspectral data and satisfies Eq. (12):

The output of test spectrum y after passing through the filter vector is given by Eq. (13):

Therefore, the average output energy of all pixels in HSI is given by Eq. (14):

where $R=1/N{\displaystyle \sum _{j=1}^N{y}_j{y}_j^{\mathrm{T}}}$ represents the autocorrelation matrix of all pixels in the HSI. N represents the number of pixels in the HSI. z_j represents the j^th output of the j^th pixel in the HSI. Then the filter-design problem is transformed into the minimum-value problem satisfying Eq. (13), as in Eq. (15):

Eq. (15) can be solved by the Lagrange-multiplier method:

The CEM method is applied to each pixel to achieve TD.

3.1. Classical Evaluation Method

To successfully evaluate the detection effectiveness of TD methods, the detection results are analyzed qualitatively and quantitatively. Qualitative analysis is used mainly to visually measure the number, contour, and integrity of targets in the detection results with the naked eye. The closer the detection results are to the real target distribution, the clearer the contour and the higher the integrity, and exhibit better detection.

In fact, a quantitative analysis of the detection results is more convincing. The receiver-operating-characteristic (ROC) curves [20] and area-under-the-curve (AUC) values are used to measure the detection results. The ROC curve reflects the relationship between the detection rate and the false-alarm rate; The abscissa indicates the false-alarm rate P_f [21], while the ordinate indicates the detection rate P_d. This two-dimensional relationship is based on the three-dimensional ROC curve containing a common threshold τ (the number of equal intervals between 0 and 1). The three-dimensional ROC curve and its projection are shown in Fig. 1.

Figure 1. Three-dimensional receiver-operating-characteristic (ROC) curve and its projection.

The three-dimensional ROC curve is projected into three vertical directions to obtain the τ − P_f (false-alarm rate) curve, τ − P_d (detection rate) curve, and P_d − P_f (ROC) curve. The τ − P_f curve reflects the ability of the detector to suppress the background. The τ − P_d curve reflects the ability of the detector to enhance the target. The P_d − P_f curve reflects the overall performance of the detector.

The two-dimensional ROC curve is essentially composed of τ − P_f and τ − P_d curves. Multiple groups of P_f and P_d values can be obtained by setting different thresholds τ. The ROC curve can be obtained by combining the values of P_f and P_d corresponding to each threshold. P_f and P_d are defined as follows:

where N_d represents the number of detected true target pixels, which is the number of pixels that actually belong to the target and are considered as such by the detector. N_t represents the total number of target pixels in the image, N_f represents the number of false-alarm pixels detected, and N_tot represents the total number of pixels in the image.

In fact the ROC curve is not continuous, but consists of discrete points, each of which represents the false-alarm rate and detection rate corresponding to a threshold. The AUC value quantitatively describes the degree of inclination of the ROC curve toward the upper left. The AUC value is the area surrounded by the ROC curve and the abscissa. The more the ROC curve is bent toward the upper left, the larger the AUC value, the better detection of the detection method, and the higher the detection reliability. The smaller the AUC value, the worse the detection results and the lower the detection reliability.

3.2. Analysis of Applicability

It is not entirely accurate to determine the merits of a detection method based on the ROC curve or AUC value. Especially in land-based applications, spectral data show a certain degree of uncertainty depending on the imaging conditions. The performance of TD methods under multiple imaging conditions is variable. It is obviously inappropriate to consider only TD performance under a single imaging condition. For example, a detection method can have a better detection effect under some imaging conditions yet face difficulty in detecting the target under other conditions, which cannot confirm the performance of the detection method and cannot describe the applicability of the method under land-based imaging conditions. Therefore, we need to comprehensively consider the average value and degree of dispersion of the detection results. The larger the average value and the smaller the degree of dispersion, the better the detection effect and the better applicability under land-based imaging conditions.

Detection results are diverse depending on land-based imaging conditions. Therefore, the analysis of detection results under land-based imaging conditions is based on the comprehensive analysis of multiple detection results. This approach requires that the detection method meet not only the requirements of detection accuracy, but also those of stability; That is, the detection method with a higher level of detection accuracy under each imaging condition has a greater degree of applicability. Based on this idea, this paper proposes the SC and effective AUC (EAUC) concepts. The calculation process is as follows.

First, the τ − P_f and τ − P_d curves are analyzed. The step size of the threshold value is constant, and thus the τ − P_f and τ − P_d curves are evenly distributed on the horizontal axis. Therefore, the average false-alarm and detection curves can be obtained by taking both the τ − P_f and τ − P_d curves under various imaging conditions:

where K represents the number of imaging conditions. P_fi and P_di represent the false-alarm rate and detection rate curves under the i^th imaging condition respectively.

Then the SC value is calculated to combine the stability characteristics of the detection results, and the AUC value under different imaging conditions is weighted according to the SC value to calculate the EAUC value. The SC value combines the average value and degree of dispersion, which more comprehensively reflects the performance of the detection methods under various imaging conditions. The larger the SC value, the more similar the ROC curve of the detection results to the average ROC curve, and the greater the weight involved in calculating the EAUC value. The SC value is calculated as follows:

where σ_i represents the standard deviation for the i^th imaging condition. P_fik and P_dik represent the k^th element of false-alarm rate P_fi and detection rate P_di for the i^th imaging condition respectively. Q represents the number of the threshold τ (equal intervals between 0 and 1).

The EAUC values of all detection methods are calculated by combining the SC and AUC values under all imaging conditions:

By analyzing the calculation process for the EAUC, it is found that the EAUC can reflect both the average value and degree of dispersion of the detection results. Therefore, in theory this approach can comprehensively evaluate the applicability of a detection method under land-based imaging conditions.

4.1. Experimental Data

The experimental data are obtained by an HSI-300 hyperspectral imaging system based on acousto-optic tunable filter (Gooch & Housego Co., Ilminster, England). The size of each hyperspectral image data is 712 × 1,002 pixels. The spectral resolution (band interval) of the HSI-300 spectrometer is 4 nm, its bandwidth is 2.3 nm, and its band range is 449–801 nm. A total of 89 bands are obtained. The experiment takes place in Shijiazhuang, Hebei, China, on June 1, 2023. The geographical coordinates are 38° 27′ N, 114° 30′ E. The weather is clear during the experiment. The simulation platform consists of an Intel Core i7-7700HQ CPU with a dominant frequency of 2.80 GHz, 8 GB RAM, and MATLAB R2017a software.

The targets include two pieces of camouflage clothing, sample 1, and a brown plate, sample 2. The background is composed mainly of green leaves, soil, leaves of Chinese ilex, and weeds. Samples 1 and 2 are almost integrated into the background, and it is difficult to detect their specific positions with the naked eye. The distributions and spectral curves of the two samples are shown in Fig. 2, and their real locations are in Fig. 3.

Figure 2. Distributions and spectral curves of the two samples.

Figure 3. Real locations of the two samples.

Imaging conditions are arbitrary in land-based applications, meaning that the use of any imaging condition is possible; Therefore, the more imaging conditions, the better the analysis of the detection results. However, it is impossible to obtain all imaging conditions completely, and only representative groups of imaging conditions can be selected. Experiments are conducted at 10:00, 10:30, 11:00, 11:30, 12:00, 12:30, 13:00, 13:30, 14:00, and 14:30, and ten groups of hyperspectral data are obtained. The imaging conditions are listed in Table 1.

TABLE 1. Imaging conditions.

Group No.	Time	Solar Zeinth Angle (˚)	View Zeinth Angle (˚)	Relative Azimuth Angle (˚)
1	10:00	34.5	62	148.5
2	10:30	29.1	62	139.2
3	11:00	24.2	62	130.1
4	11:30	20.0	62	107.6
5	12:00	17.2	62	85.3
6	12:30	16.6	62	66.4
7	13:00	18.5	62	35.5
8	13:30	22.1	62	20.2
9	14:00	26.7	62	13.7
10	14:30	32.0	62	6.1

Since the obtained hyperspectral data denote the reflected intensity values, radiometric calibration of the data is needed; That is, the normalization of the obtained hyperspectral data to the form of reflectance. On the one hand, this approach can overcome the effect of radiation intensity, and on the other hand, it can also reduce the effect of noise. We use a standard whiteboard to normalize the obtained hyperspectral data, as shown in Fig. 4. The standard whiteboard used in this experiment is a polytetrafluoroethylene (PTFE) board that has been calibrated. Its reflection properties are uniform in all directions, and it can be approximated as a Lambertian reflector. The reflection intensity of this board is higher than that of the other pixels in each band. The spectral reflectance r(λ) of each pixel in this experiment can be calculated by Eq. (22):

Figure 4. Standard whiteboard.

where r_s(λ) represents the reflectance of the standard whiteboard and value of the reflectance was 0.98. I_s(λ) represents the obtained spectral intensity value of the standard whiteboard. I(λ) represents the obtained spectral intensity value of each pixel. Then the spectral of each pixel in the hyperspectral data is transformed into reflectance, and the spectral curve represents the spectral reflectance as determined by the standard whiteboard. The spectral curves of two samples under all imaging conditions are calculated, as shown in Fig. 5.

Figure 5. Spectral curves of two samples under all imaging conditions: (a) Spectra of sample 1, (b) spectra of sample 2.

4.2. Detection Results

We take two samples as the desired targets, and the applicability of each is analyzed. The mean spectra of sample 1 and sample 2 under all conditions are taken as known reference spectra. The detection results of the SAC, NED, NCC, CEM, MI, and NSGA methods under all conditions are analyzed. Since the measured spectral data consists solely of positive values, the ranges of the six detection methods are all in [0, 1]. Thus it is unnecessary to unify the dimensions of the different detection methods. The closer the detection results are to 1 for the SAC, CEM, NCC, MI, and NSGA methods, the more similar the test spectrum is to the reference spectrum. The NED method has a larger calculation result when the test spectrum is more similar to the reference spectrum. Therefore we reversed the calculation result of the NED method using D_new = 1 − D_origin, where D_origin represents the calculation result before reversal and D_new represents the calculation result after reversal. The detection results for two samples under all imaging conditions are shown in Figs. 6 and 7.

Figure 6. Detection results for sample 1, under (a)–(j) 10 conditions. Detection methods are SAC, NED, NCC, CEM, MI, and NSGA from left to right, respectively.

Figure 7. Detection results for sample 2 under (a)– (j) 10 conditions. Detection methods are SAC, NED, NCC, CEM, MI, and NSGA from left to right, respectively.

4.3. Analysis of Applicability

First, we qualitatively analyze the detection results. As shown in Fig. 6, the NSGA method hardly detects the target. The NSGA method measures the change rate of the spectra, and the rate of sample 1 under visible light is very similar to those of the other objects in the background. As a result, the NSGA method detection results are not proper in distinguishing different ground objects. The results of the MI method under some conditions are clearly visible and show a large contrast with the background. However, under conditions 7 and 10, the target (sample 1) is obliterated by the background and it is almost impossible to detect the target, indicating that the stability of the MI method is low and that the detection results are greatly affected by the imaging conditions. The SAC, NED, NCC, and CEM methods can detect the target under all conditions. The gray value of the target position is the largest, and its detection results are less affected by the imaging conditions and are more stable. Therefore, for the detection of sample 1 against this background, the SAC, NED, NCC, and CEM methods exhibit better detection effects, followed by the MI method, and the NSGA method exhibits the worst detection effects.

Similarly, the detection results for the target (sample 2) with the NSGA method shown in Fig. 7 are worse than those of other methods. The MI method can detect the target under conditions 1, 3, 4, 5, 6, and 8, but the detection results are poor under other conditions, which also indicates that the degree of stability of the MI method is low. The SAC and NED methods can detect the target under all conditions, but there is an obviously serious false alarm rate in the detection results. The CEM method can detect the target under all conditions, and the contrast between the target and the background is obvious. However, there are also some false alarms in the detection results. The results of the NCC method under the first nine conditions are clearly visible, but under condition 10 the target is obliterated by the background and is almost impossible to detect.

According to the comprehensive analysis, the reason for the poor detection results of the NCC method under condition 10 can be caused by errors in the data. However, this method shows significantly better detection results than those of the other methods under the first nine groups of conditions, and the false alarm rate is low. The CEM method can detect the target under almost all imaging conditions, but it is also accompanied by a certain degree of false alarms. Although the SAC and NED methods also show detection effects, false alarms are more obvious when these methods are used. The MI method has a detection effect only under a few conditions, while the NSGA method has no detection effect at all. In summary, the detection methods for sample 2 can be listed in order of effectiveness: CC, CEM, SAC, NED, MI, and NSGA methods.

Next, we quantitatively analyze the results. The step-in threshold is set to 0.001, and the number of steps is 1,000. Figure 8 shows the ROC curves for all detection methods under all imaging conditions.

Figure 8. Receiver-operating-characteristic (ROC) curves for each method, under 10 conditions. (a) Sample 1 and (b) sample 2.

By analyzing the trend of the ROC curves in Fig. 8(a), we find that the ROC curves for the SAC, NED, NCC and CEM methods are significantly curved toward the upper left, proportional to the increasing detection performance. As seen from Fig. 8(b), the ROC curves for the MI method are greatly affected by the imaging conditions, and the detection performance is unstable. The ROC curves for the SAC, NED, NCC, and CEM methods are significantly curved toward the upper left, compared to those for the NSGA and MI methods. This finding is consistent with that of the previous qualitative analysis. The AUC values are shown in Table 2.

TABLE 2. Area-under-the-curve (AUC) values for different methods.

AUC		SAC	NED	NCC	CEM	MI	NSGA
Sample 1	Condition 1	0.9959	0.9959	0.9998	0.9999	0.9878	0.5914
	Condition 2	0.9899	0.9899	0.9989	0.9997	0.9868	0.5719
	Condition 3	0.9796	0.9796	0.9992	0.9995	0.9875	0.5661
	Condition 4	0.9645	0.9645	0.9985	0.9974	0.9795	0.5403
	Condition 5	0.9616	0.9616	0.9983	0.9976	0.9843	0.5754
	Condition 6	0.9565	0.9565	0.9985	0.9967	0.9819	0.5319
	Condition 7	0.9834	0.9834	0.9974	0.9959	0.9698	0.5200
	Condition 8	0.9578	0.9578	0.9981	0.9932	0.9734	0.5296
	Condition 9	0.9776	0.9776	0.9970	0.9904	0.9618	0.5350
	Condition10	0.9965	0.9965	0.9891	0.9897	0.8959	0.4609
	Mean	0.9763	0.9763	0.9975	0.9960	0.9709	0.5423
Sample 2	Condition 1	0.9717	0.9718	0.9980	0.9909	0.9389	0.2829
	Condition 2	0.9579	0.9578	0.9969	0.9898	0.9281	0.2409
	Condition 3	0.9495	0.9495	0.9965	0.9903	0.9221	0.2212
	Condition 4	0.9385	0.9385	0.9960	0.9876	0.8791	0.2466
	Condition 5	0.9320	0.9320	0.9961	0.9845	0.8527	0.2451
	Condition 6	0.9275	0.9275	0.9926	0.9845	0.9037	0.2489
	Condition 7	0.9416	0.9416	0.9914	0.9799	0.7585	0.2693
	Condition 8	0.9320	0.9320	0.9923	0.9790	0.8371	0.2916
	Condition 9	0.9506	0.9506	0.9895	0.9744	0.7023	0.3174
	Condition10	0.9669	0.9670	0.7583	0.9686	0.3787	0.3077
	Mean	0.9468	0.9468	0.9708	0.9832	0.8101	0.2672

Table 2 shows that the AUC values of the NCC and CEM methods are almost always the largest among all imaging conditions, with a small range of variation. This finding shows that the NCC and CEM methods perform better and have the best level of applicability under various land-based imaging conditions, compared to other methods. When detecting sample 2, the average AUC value of the NCC method is smaller than that of the CEM method. This is not consistent with that of our previous qualitative analysis, because only the average AUC value is calculated without combining the stability characteristics of all detection results. Consequentially, the comprehensive evaluation result is greatly affected by a certain result. Thus large deviation occurs in the comprehensive evaluation result, which calculates only the average AUC value.

Notably, the source of the average AUC values is obtained only by averaging the AUC values in Table 2, not by averaging the ROC curves. Since the distribution of the points in the ROC curve on the coordinate axis is not continuous but discrete, the ROC curve cannot be averaged directly. To comprehensively evaluate the detection effect under land-based imaging conditions, this study uses the EAUC method to comprehensively analyze and evaluate the applicability of the detection methods.

First, the average false alarm and detection curves are shown in Fig. 9. Since the average false-alarm curves and average detection curves have the same threshold value τ, the average false-alarm-rate curves and average detection-rate curves can be combined into the average ROC curves according to Eq. (22), as shown in Fig. 10. Then, the SC values of all detection methods under various imaging conditions are calculated according to Eq. (23), as shown in Table 3. Finally, the calculation results for the EAUC values are shown in Table 4.

Figure 9. Average detection and false-alarm curves of detection methods in (a) sample 1 and (b) sample 2.

Figure 10. Average receiver-operating-characteristic (ROC) curves of all detection methods in (a) sample 1 and (b) sample 2.

TABLE 3. Stability coefficient (SC) values.

SC		SAC	NED	NCC	CEM	MI	NSGA
Sample 1	Condition 1	0.0396	0.0421	0.0461	0.0626	0.0489	0.0834
	Condition 2	0.0630	0.0657	0.0693	0.0537	0.0727	0.1016
	Condition 3	0.1405	0.1485	0.1588	0.0737	0.1065	0.1829
	Condition 4	0.1773	0.1579	0.2257	0.1391	0.2922	0.1116
	Condition 5	0.1292	0.1224	0.1213	0.1434	0.0579	0.1230
	Condition 6	0.0865	0.0978	0.0807	0.1797	0.0971	0.2064
	Condition 7	0.1397	0.1324	0.1309	0.0987	0.1200	0.0569
	Condition 8	0.0876	0.0865	0.0424	0.0777	0.0896	0.0474
	Condition 9	0.0849	0.0822	0.0886	0.0554	0.0618	0.0643
	Condition 10	0.0518	0.0645	0.0363	0.1160	0.0533	0.0226
Sample 2	Condition 1	0.0513	0.0534	0.0538	0.0489	0.0671	0.1333
	Condition 2	0.0689	0.0666	0.0732	0.0608	0.0469	0.0809
	Condition 3	0.0683	0.0830	0.1027	0.0601	0.0859	0.0381
	Condition 4	0.1473	0.1619	0.1575	0.0713	0.1576	0.0927
	Condition 5	0.1615	0.1469	0.1673	0.2311	0.1380	0.1393
	Condition 6	0.1209	0.0961	0.1022	0.1745	0.1039	0.1552
	Condition 7	0.1499	0.1418	0.1899	0.0917	0.1529	0.0823
	Condition 8	0.0925	0.0932	0.0451	0.1281	0.1243	0.0774
	Condition 9	0.1074	0.1210	0.0803	0.0670	0.0935	0.0557
	Condition 10	0.0321	0.0362	0.0281	0.0665	0.0300	0.1451

TABLE 4. Effective area under the curve (EAUC) values.

EAUC	SAC	NED	NCC	CEM	MI	NSGA
Sample 1	0.9732	0.9737	0.9980	0.9960	0.9745	0.5513
Sample 2	0.9425	0.9432	0.9870	0.9834	0.8309	0.2635

As seen from Table 4, the EAUC value is higher in order of NCC, CEM, MI, NED, SAC, and NSGA when the target is sample 1. This is consistent with the previous qualitative analysis. However, it is inconsistent with the sequence of average AUC values, because the EAUC method combines the stability characteristics of the detection results rather than considering only the average value of the detection, which can reflect the detection effect more objectively. At the same time, the detection results under all imaging conditions are relatively stable, and there is no large degree of variation. The use of the EAUC-value method does not affect the average size of the detection results. When sample 2 is the target, the EAUC value is higher in order of NCC, CEM, NED, SAC, MI, and NSGA. This sequence is inconsistent with the average AUC value but consistent with the qualitative analysis. This also proves that the EAUC-value method is more reasonable for the evaluation of detection results and can effectively realize the evaluation of the detection method under land-based imaging conditions, compared to the other methods considered.

The results of the above analysis of the test results based on the EAUC value are consistent with those of the qualitative analysis, indicating that the EAUC value is more reasonable than other values for measuring detection effectiveness. Compared to the direct calculation of the average AUC value, the calculation of the EAUC value combined with the SC value gives different weights to the AUC value under different imaging conditions. Thus the value of EAUC is combined with the evaluation results under various imaging conditions, reducing the contingency of the evaluation, which is conducive to our selection of TD methods with high accuracy and strong stability under land-based imaging conditions.

From the perspective of land-based applications, this paper first introduces several different types of detection methods and classical evaluation methods, and then analyzes the detection results of TD methods and their applicability under land-based imaging conditions. The levels of detection performance of TD methods under different imaging conditions are not constant. Therefore, a traditional evaluation method that measures only the detection performance of the TD method under one condition is not suitable under land-based imaging conditions. Finally, the proposed EAUC-value method is used to comprehensively analyze the applicability of each detection method under land-based imaging conditions. This method fully combines detection accuracy and stability; That is, a method with high levels of accuracy and stability has a larger EAUC value and can realize the evaluation of TD methods under various imaging conditions. This method is suitable for the evaluation of detection performance under land-based imaging conditions. However, due to the limited number of imaging conditions, many groups of imaging conditions are selected based only on subjective experience. The more imaging conditions there are, the more conducive the method is to the more comprehensive calculation of EAUC values; The more able it is to reflect the applicability of the detection method under land-based conditions; And the more conducive it is to selecting appropriate detection methods for land-based applications. This paper also proposes objective and rational detection methods through the EAUC-value method, considering the uncertainty characteristics of the spectra into account.

We would like to thank the Army Engineering University of PLA, Electronic and Optical Engineering Department for financial and equipment support in developing this work. We would also like to thank the anonymous reviewer for their helpful and insightful comments, which significantly improved the quality of the manuscript.

National Natural Science Foundation of China (Grant no. 62005319).

The authors declare that there are no conflicts of interest related to this article.

Data underlying the results presented in this paper are not publicly available at this time, but may be obtained from the authors upon reasonable request.