Ex) Article Title, Author, Keywords
Current Optics
and Photonics
Ex) Article Title, Author, Keywords
Curr. Opt. Photon. 2024; 8(3): 246-258
Published online June 25, 2024 https://doi.org/10.3807/COPP.2024.8.3.246
Copyright © Optical Society of Korea.
Shenxiang Feng, Xiaojian Hao , Tong Wei, Xiaodong Huang, Pan Pei, Chenyang Xu
Corresponding author: ^{*}haoxiaojian@nuc.edu.cn, ORCID 0000-0003-0295-701X
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
In aerospace and energy engineering, the reconstruction of three-dimensional (3D) temperature distributions is crucial. Traditional methods like algebraic iterative reconstruction and filtered back-projection depend on voxel division for resolution. Our algorithm, blending deep learning with computer graphics rendering, converts 2D projections into light rays for uniform sampling, using a fully connected neural network to depict the 3D temperature field. Although effective in capturing internal details, it demands multiple cameras for varied angle projections, increasing cost and computational needs. We assess the impact of camera number on reconstruction accuracy and efficiency, conducting butane-flame simulations with different camera setups (6 to 18 cameras). The results show improved accuracy with more cameras, with 12 cameras achieving optimal computational efficiency (1.263) and low error rates. Verification experiments with 9, 12, and 15 cameras, using thermocouples, confirm that the 12-camera setup as the best, balancing efficiency and accuracy. This offers a feasible, cost-effective solution for real-world applications like engine testing and environmental monitoring, improving accuracy and resource management in temperature measurement.
Keywords: 3D temperature filed, Deep learning, Non-contact temperature measurement, Radiation thermometry
OCIS codes: (100.0100) Image processing; (110.0110) Imaging systems
Pyrometry holds great importance in modern industry and scientific research. It serves as an accurate method for determining the temperature of objects in extreme environments, playing a critical role in high-temperature processes within fields such as materials science, energy development, and aerospace. For example, in the domain of material synthesis, the meticulous control of temperature in high-temperature environments directly affects the quality and properties of the materials produced. Moreover, the utilization of three-dimensional (3D) temperature-field provides valuable information about surface temperatures and temperature distributions within objects. This technique proves indispensable in areas such as combustion, chemical reaction engineering, and biomedical research, where complex heat flows and transfer processes are often involved. By employing 3D temperature-field measurements, we can enhance the accuracy of modeling and comprehending these complex processes [1–3].
Mainstream methods for advanced optical diagnosis include one-and two-dimensional techniques, such as traditional laser-induced fluorescence (LIF) and particle image velocimetry (PIV). These techniques find widespread use in measuring flame temperature and velocity [4–6]. However, the limited spatial resolution of the detection equipment hampers high-resolution 3D reconstruction of the combustion field. Furthermore, the complex test systems require sufficient space in the experimental environment to arrange equipment. While infrared-radiation thermometry serves to obtain the surface temperature of high-temperature objects, it falls short in accurately reconstructing the 3D temperature field of a flame.
Cai et al. [7–9] innovatively used a single camera equipped with an endoscope based on chemiluminescence-computational-tomography technology, to achieve high spatial and temporal resolutions in reconstructing a flame’s luminescence field. This approach enabled the comprehensive study of combustion processes in turbulent and swirling flames. It is important to note that endoscopes introduce limitations in image resolution and significant image distortion due to their optical lenses, which need correction prior to reconstruction. Conventional 3D reconstruction algorithms based on computed tomography of chemiluminescence (CTC) include algebraic reconstruction technology (ART), simultaneous algebraic reconstruction technology (SART), and multiplicative algebraic reconstruction technologies (MART). These algorithms share similar reconstruction mechanisms, with the resolution of their results being affected by pixel partitioning. In recent times there has been a growing interest in harnessing deep-learning algorithms for various computational tasks. Consequently, attempts have been made to apply deep learning to CTC reconstruction applications. For example, Worth and Dawson [10] used CTC technology to study vortex-flame interactions in turbulent flames, showcasing the potential of deep learning in this context.
Despite the successful outcomes of previous research endeavors, the inherent inherently unsteady nature of flames necessitates the use of camera sensors with a wide dynamic range, resulting in high equipment costs. This cost challenge also extends to related studies that aim to employ multiple cameras in reconstructing the 3D structure of flames [11, 12]. While this limitation is not a technical problem, it greatly hinders widespread application of these techniques in basic combustion research. Therefore, selecting the most cost-effective camera solution for 3D flame reconstruction becomes imperative. However, an insufficient number of cameras inevitably results in inadequate spatial sampling and reduced reconstruction accuracy. Therefore, using quantitative evaluation to determine the appropriate number of cameras for 3D reconstruction is important.
To study the impact of camera quantity on reconstruction results, we must formulate the best experimental plan. Based on the sampling methodology inherent to the reconstruction algorithm in spatial ray tracing, we introduce the spatial sampling rate as a quantitative index to evaluate the relationship between the number of cameras and reconstruction errors. As the camera count increases, the time required for model training also linearly increases. To strike a balance between reconstruction accuracy and efficiency, we propose computational efficiency as an additional evaluation metric, taking into account both the spatial sampling rate and model-training time.
In our study, we conduct experiments that combine numerical simulations with butane-flame reconstruction. In the numerical simulation, we use a Gaussian temperature distribution to simulate temperature fields of varying complexity. We employ 6, 9, 12, 15, and 18 cameras for the reconstruction process and calculate corresponding metrics, including root-mean-square error, spatial sampling rate, and computational efficiency. Our findings lead us to conclude that the optimal number of cameras for this task is 12. In the subsequent butane-flame reconstruction experiment, we seek to further verify the choice of 12 cameras as a solution that balances cost considerations with reconstruction accuracy. We conduct the reconstruction experiments using 9, 12, and 15 cameras. To assess the accuracy we use thermocouples to measure temperatures at different heights of the flame. We then calculate the relative error by comparing the reconstruction results to the corresponding actual temperature measurements, ultimately confirming that the use of 12 cameras is optimal.
A flame constitutes a highly complex gas-phase chemical-reaction system, with its internal radiation mainly coming from combustion reaction products and partially burned fuel particles. When thermally excited particles relax from a high-energy state to a low-energy state, they emit photons and produce flame radiation [13].
Our flame-temperature-measurement technology, based on Planck’s blackbody radiation law and calibrated through experiment, determines a relationship between the radiation intensity of the target under measurement and the readings from the optical detector. It calculates the temperature of an object by measuring the intensity of radiation energy emitted by that object. Various radiation temperature-measurement methods, predicated on wavelength selection, fall into three categories: The total-radiation method, single-wavelength-radiation temperature-measurement method, and temperature-measurement methods based on radiation-energy ratios for multiple wavelengths. In our study, we employ single-wavelength radiation thermometry. According to Planck’s Law, the relationship between the radiation intensity E(λ, T) at wavelength λ within a flame and its temperature T is expressed as follows [14]:
where λ is the wavelength, c is the speed of light, h is Planck’s constant, and k is Boltzmann’s constant. The integral formula depicting the radiation generated by the flame as it passes through the camera lens onto the imaging plane is [7]:
where I(p) is the integrated radiation intensity received by pixel p on the imaging plane. E(p) and ω(x) are respectively the radiation intensity and the corresponding emissivity at point x in space. Following photoelectric conversion within the camera, the integrated radiation intensity I(p) is converted to a grayscale value G(p). Through calibration tests involving a blackbody furnace, we can establish the relationship between the grayscale value G(p) and radiation intensity I(p) as Eq. (3):
where A and B are constants. In our approach, a two-dimensional (2D) temperature projection can be calculated from a flame-projection image collected by a camera, leveraging the principles mentioned above. Our prior research improved grayscale imaging by doping K element from K_{2}SO_{4} into the flame [15–17]. Therefore, in this study we conduct single-spectral-line temperature measurements by installing a narrowband filter with a central wavelength of 768 nm and a bandwidth of 10 nm in front of the camera lens.
Based on the principle of radiation thermometry, differentiable rendering technology from computer graphics and universal approximation theorem, we employ a multilayer perceptron (MLP) to represent the 3D temperature field [18]. Spatial-position coding technology is used to map input spatial coordinates to a high-dimensional space, improving the network’s ability to represent rapidly changing regions within the temperature field. Meanwhile, differentiable rendering technology expresses the flame-radiation projection process as a differentiable function. This function enables the calculation of the reprojection result for the light, and MLP training is conducted by reducing the mean square error (MSE) between the reprojection result and the actual projection.
As shown in Fig. 1, we generate a dataset of rays based on the intrinsic and extrinsic parameters of the camera, and the images collected by it. The light dataset consists of the origin, direction, and pixel projection values. Sampling occurs at equal steps along the ray.
The network structure of the MLP is shown in Fig. 2, with the input vector shown in green, the middle-hidden layers in yellow, and the output vector in orange. The input Γ(x) is the 3D coordinate of the sample point x encoded by Eq. (4), for L = 5. The output of the MLP is the temperature T. The numbers within each block indicate the dimensions of the respective vectors. All layers are fully connected and employ the rectified-linear-unit (ReLU) activation function. The output layer also utilizes ReLU to ensure that output results remain non-negative. Following position encoding, the input passes through six fully-connected layers, each with a width of 256, using the ReLU activation function. Subsequently, two downsampling layers with widths of 128 and 64 are concatenated to reduce dimensionality. Ultimately, the temperature corresponding to the given point serves as the output. To address issues related to vanishing and exploding gradients in deep networks, we introduce a residual connection. This connection involves the concatenation of the input vector with the vector from the fourth layer. In the diagram, the symbol + represents the concatenation of vectors.
In this study, we employ a simplified forsward-imaging pinhole model to describe the mapping relationship between 3D space and 2D projection. To facilitate the reconstruction of the 3D temperature field, we simplify the camera as a viewpoint and an imaging plane. Light is defined by the viewpoint coordinates and pixel direction. We establish a flame thermal and optical radiation model, as shown in Eq. (5). The projection value for each pixel p is the light integral of the radiation intensity from the sample point across the reconstructed target domain:
where n and f represent the minimum and maximum values along the path from the sample point on the ray to the imaging plane, respectively. t(s) represents the temperature value at sample point s, while T(p) is the temperature value of pixel p. The internal and external parameter matrices of the camera determine the mapping relationship between pixel p and sample point s, as shown in Eq. (6) [19]:
where (u, v) is the position of pixel p in the pixel coordinate system and Z_{c} is the distance from the spatial sample point in the camera coordinate system to the imaging plane. f_{x} and f_{y} represent the focal lengths of pixels in the x- and y-directions respectively. c_{x}, c_{y} is the center coordinate of the imaging plane, a fixed camera parameter obtained through calibration experiments [19]. R and T represent the rotation and translation matrices of the camera respectively. (X_{w}, Y_{w}, Z_{w}) is the coordinate of the spatial sample points s in the world coordinate system.
Based on the above model, we reconstruct the 3D flame temperature field by controlling the number of cameras. This allows us to conduct further investigations of the impact of camera quantity on the reconstruction outcome. The camera performs uniform sampling along the light ray, as shown in Fig. 3. Consequently, we obtain the coordinates of the sample point at any position along the light ray within the reconstructed target domain, corresponding to each pixel of the camera. Figure 3 shows the sampling effect on the reconstructed target domain, using four cameras in a 2D schematic diagram.
The accuracy of temperature-field reconstruction is closely linked to the distribution of sample points. A higher density of sample points leads to improved accuracy in capturing the finer details of the 3D flame structure. More accurate camera angles are achieved when the camera’s field of view comprehensively covers the flame. However, increasing the number of cameras escalates experimental costs and computational-resource requirements. Therefore, we propose a spatial sampling rate related to the number of cameras, as a metric to evaluate the impact of camera quantity on reconstruction errors. To quantify the spatial sampling rate accurately, we discretize the reconstruction target domain into a voxel set of dimensions 100 × 100 × 100. The spatial sampling rate corresponding to each camera number can be evaluated by calculating the percentage of voxels containing sample points relative to the total number of pixels.
Initially, numerical simulations are conducted to study the impact of the spatial sampling rate on the results of temperature-field reconstruction. In this numerical simulation, a 3D Gaussian distribution is employed to simulate the temperature field of an explosive fireball. We generate single-, double-, and triple-fireball configurations to represent temperature fields of varying complexity. A single spherical temperature distribution is generated based on Eq. (7):
where T_{max} is the maximum value of the temperature field, set to 1,000 K for this study. The parameter N is used to control the temperature distribution, as shown in Eq. (8):
where (x_{0}, y_{0}, z_{0}) is the center coordinate of a single spherical temperature field, and R is the fireball radius.
To study the impact of the spatial sampling rate on the reconstruction results, we conduct temperature-field simulations using 6, 9, 12, 15, and 18 cameras. These cameras are centered on the origin coordinates of the temperature field and evenly arranged at intervals of 60°, 40°, 30°, 24°, and 20° respectively, at the same horizontal height, as shown in Fig. 4. In Fig. 4, it is evident that the temperature field of a single Gaussian distribution exhibits smooth and continuous characteristics internally. The temperature fields of the double and triple Gaussian distributions contain both a smooth region within the temperature field and rapidly changing regions at the junctions of two Gaussian distributions.
To evaluate the impact of different spatial sampling rates on the 3D temperature-field-reconstruction results, we employ the root-mean-square error (RMSE) to calculate the difference between the reconstructed and original temperature fields. The calculation formula is shown in Eq. (9):
where T_{i} and T–_{i} represent respectively the reconstructed and original temperature fields, while N indicates the number of voxels into which the reconstructed temperature field is divided.
Figures 5–7 show 2D slices of the three temperature fields reconstructed using 6, 9, 12, 15, and 18 cameras at height z = 0, along with their corresponding relative errors. As evident from the figures, the errors mainly cluster at the flame boundary, with minimal errors observed inside and outside the flame. The relative-error map provides further insight into the effectiveness of temperature-field reconstruction.
To conduct a more in-depth analysis of the impact of the spatial sampling rate on temperature-field reconstruction, we calculate both the spatial sampling rate corresponding to different camera numbers, and the RMSE of the reconstruction results. Figure 8 displays the reconstruction error and spatial sampling rate corresponding to the number of cameras utilized. The histogram in Fig. 8 shows the number of cameras increasing, while the temperature-field-reconstruction error consistently decreases. Compared to the double- and triple-fireball temperature fields, the single-fireball temperature fields exhibit minor reconstruction errors, and the spatial sampling rate has minimal impact on the reconstruction results. Due to their more complex structures, the double- and triple-fireball temperature fields have relatively larger reconstruction errors, which notably diminish as the number of cameras increases. The line graph in Fig. 8 illustrates that the spatial sampling rate increases as the number of cameras rises. Notably, the increment in sampling rate is most substantial when the number of cameras increases from 9 to 12.
Although increasing the number of cameras can improve reconstruction accuracy, it also extends the model-training time and consumes more computational resources. Therefore, we conduct a further analysis of the model’s computational efficiency under different spatial sampling rates. In order to better evaluate the calculation efficiency, we normalize the spatial sampling rate and calculation time respectively. The computational efficiency is the ratio of the normalized spatial sampling rate to the normalized training time, as listed in Table 1. We set the normalized minimum value of the spatial sampling rate as 0.265, and the normalized minimum value of the training time is 11 min. As shown in Table 1, the model achieves the highest computational efficiency of 1.263 when using 12 cameras. After careful analysis, considering both reconstruction accuracy and computational efficiency, we select 12 cameras spaced 30° apart as the optimal number for temperature-field reconstruction.
TABLE 1 Spatial sampling rate and computational efficiency
Number of Cameras | Spatial Sampling Rate | Training Time (min) | Computational Efficiency (%) |
---|---|---|---|
6 | 0.2659 | 12 | 0.329 |
9 | 0.2819 | 19 | 0.773 |
12 | 0.3133 | 25 | 1.263 |
15 | 0.3284 | 32 | 1.105 |
18 | 0.3415 | 39 | 1.000 |
To verify the effect of different camera numbers on an actual flame’s temperature-field reconstruction, we conduct a 3D temperature-field-reconstruction experiment on a butane flame. The maximum temperature of a butane flame can reach 1,400 K [20]. In this study we use thermocouples to measure temperatures at five distinct points, at different flame heights. We then compare the reconstructed temperature values at the corresponding positions to evaluate the reconstruction error of the 3D temperature field.
To address the rapid changes in the jet flame resulting from airflow disturbances, we adopt a synchronous control system, as shown in Fig. 9(a). This system ensures that all cameras can simultaneously collect flame projection data. The synchronous control system mainly relies on a signal generator to transmit acquisition signals to all cameras simultaneously. Subsequently, it transmits the image data captured by all cameras to the workstation for processing via a high-speed data acquisition card. We use 9, 12, and 15 CMOS cameras for the reconstruction experiments, maintaining a distance of 50 cm between each camera and the flame. The cameras are centered on the target flame and evenly spaced 30° apart, as shown in Fig. 9(b). In addition, each camera’s lens is equipped with a narrowband filter featuring a central wavelength of 768 nm and a bandwidth of 10 nm.
According to the established grayscale-to-temperature fitting relationship from prior calibration experiments, we sequentially process the grayscale images collected by the 12 cameras. This allows us to obtain a 2D temperature projection of the butane flame from every direction. Figure 10(a) displays a grayscale image of the flame captured by the camera on the far right; Figure 10(b) showcases the corresponding 2D temperature map. The maximum temperature recorded was 1,242.85 K.
Figure 11 shows the change in MSE loss values with the number of epochs during neural-network training. In this study, one traversal through all rays is considered to constitute one epoch. As depicted in the figure, the final convergence values for 9, 12, and 15 cameras were 1,398, 804, and 630 respectively. A comparative analysis of these three curves reveals that the convergence performance for 12 and 15 cameras is better than that for 9 cameras. Taking into account the cost considerations related to the cameras, it is determined that 12 cameras represent the best experimental solution.
Figure 12 shows the reconstruction results for the butane flame’s temperature field, employing different numbers of cameras. The figure reveals that the shapes of the three reconstructed temperature fields are relatively consistent, but with varying temperature distributions. To verify the accuracy of the temperature-field reconstruction, we use a K-type thermocouple to measure temperature values at the five sample points depicted in Fig. 10. We then calculate the relative error. Table 2 shows the findings, indicating that when the camera numbers were 9, 12, and 15, the maximum relative errors between the reconstruction results and the thermocouple measurements were 15.06%, 5.76%, and 2.59% respectively. Among the three sets of reconstruction results, the relative error at sample point 1 is the smallest among the five sample points. Additionally, the changing trend of flame temperature is consistent with the thermocouple measurements, with temperature decreasing as height decreases. Comparing the reconstruction results obtained using 12 and 15 cameras, the maximum relative errors are 5.76% and 2.59% respectively, falling within an acceptable range.
TABLE 2 Verification of radiation-temperature-measurement results based on thermocouples
Number of Cameras | Position | Thermocouple Measurements (K) | Reconstructed Value (K) | Relative Error (%) |
---|---|---|---|---|
9 | 1 | 1,132 | 1,191 | 5.21 |
2 | 903 | 839 | 7.09 | |
3 | 769 | 712 | 7.41 | |
4 | 742 | 695 | 6.33 | |
5 | 704 | 598 | 15.06 | |
12 | 1 | 1,140 | 1,145 | 0.44 |
2 | 942 | 912 | 3.18 | |
3 | 747 | 724 | 3.08 | |
4 | 733 | 714 | 2.59 | |
5 | 712 | 671 | 5.76 | |
15 | 1 | 1,141 | 1,143 | 0.18 |
2 | 927 | 903 | 2.59 | |
3 | 755 | 745 | 1.32 | |
4 | 739 | 720 | 2.57 | |
5 | 695 | 682 | 1.87 |
In this study, our main focus was on analyzing the impact of the number of cameras on the results of 3D temperature-field reconstruction. This is because the reconstruction results are heavily affected by the density of spatial-sampling points. A higher coverage of the reconstruction area by sample points generally leads to smaller reconstruction errors. Therefore, we proposed the spatial sampling rate as a quantitative index to evaluate how the number of cameras affects reconstruction results. However, in addition to the number of cameras, the factors influencing the spatial sampling rate are also related to factors such as the resolution of the camera’s imaging plane and the field of view of the lens. However, considering that our model of the experimental equipment was relatively simple, we focused on studying the impact of the number of cameras on the spatial sampling rate. We analyzed this by conducting numerical simulations and butane-flame-reconstruction experiments. In our assessment, we considered factors such as computational efficiency, equipment cost, and reconstruction accuracy. The results determined that the setup with 12 cameras spaced 30° apart was optimal.
The number of cameras and the chosen sampling method along the ray mainly affect the spatial sampling rate. Under identical camera-parameter settings, an increase in the number of cameras results in denser sampling points generated in space, thereby increasing the spatial sampling rate. Similarly, when camera settings remain unchanged, reducing the sampling step along the ray generates more sample points and increases the corresponding spatial sampling rate. Following the discretization of the reconstruction target domain into 100 × 100 × 100 voxels, we calculated the corresponding spatial sampling rates for camera configurations of 6, 9, 12, 15, and 18. Our calculations verified that the spatial sampling rate increased with an increase in the number of cameras. To further analyze the impact of the number of cameras on reconstruction results, we conducted both simulated temperature-field reconstruction experiments and empirical butane-flame temperature-field reconstruction experiments.
In our numerical simulation, we simulated a fireball with a spherical temperature distribution that adhered to a Gaussian distribution. The maximum temperature at the center of this fireball was set to 1,000 K. We simulated temperature fields with different complexities by creating single, double, and triple fireballs. The numerically simulated temperature field included both smooth temperature changes within the flame and abrupt temperature changes at the flame’s boundary. This allowed us to verify the impact of different spatial sampling rates on the reconstruction results. The simulation results showed that the temperature field of a single fireball, characterized by a relatively simple structure, was less affected by the number of cameras. Conversely, the temperature fields of the double- and triple-fireball scenarios exhibited a decrease in reconstruction error as the number of cameras increased. To select the optimal number of cameras, we used the normalized ratio of spatial sampling rate to model-training time as a measure of computational efficiency, with 12 cameras achieving the highest reconstruction efficiency of 1.263.
For the butane flame’s temperature-field reconstruction, we used three experimental schemes employing 9, 12, and 15 cameras, with thermocouples serving as verification tools to determine reconstruction errors. The maximum relative errors observed in the reconstruction results for the three schemes were 15.06%, 5.76%, and 2.59% respectively. Additionally, the average relative errors were 8.22%, 3.01%, and 1.71% respectively. After a comprehensive evaluation considering both reconstruction accuracy and cost, it was determined that the use of 12 cameras represented the best solution.
Our approach involved evaluating different experimental plans based on the number of cameras and the light-sampling method in the reconstruction algorithm. We proposed two key indicators, namely the spatial sampling rate and computational efficiency, to evaluate their impact on reconstruction results. This evaluation method helps to reduce equipment cost and improve calculation efficiency in temperature measurement, while ensuring reconstruction accuracy.
The authors thank all reviewers, editors, and contributors for their contributions and suggestions, as well as all members of the OSEC Laboratory.
Supported in part by the National Natural Science Foundation of China (NSFC) (Grant no. 52075504); the Fund for Shanxi 1331 Project Key Subject Construction, State Key Laboratory of Quantum Optics, and Quantum Optics Devices (Grant no. KF202301); the Shanxi Key Laboratory of Advanced Semiconductor Optoelectronic Devices and System Integration (Grant no. 2023SZKF11).
The authors declare no conflicts of interest.
The data underlying the results presented in this paper are not publicly available at this time, but may be obtained from the authors upon reasonable request.
Curr. Opt. Photon. 2024; 8(3): 246-258
Published online June 25, 2024 https://doi.org/10.3807/COPP.2024.8.3.246
Copyright © Optical Society of Korea.
Shenxiang Feng, Xiaojian Hao , Tong Wei, Xiaodong Huang, Pan Pei, Chenyang Xu
The State Key Laboratory of Dynamic Measurement Technology, North University of China, Taiyuan 030051 Shanxi, China
Correspondence to:^{*}haoxiaojian@nuc.edu.cn, ORCID 0000-0003-0295-701X
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
In aerospace and energy engineering, the reconstruction of three-dimensional (3D) temperature distributions is crucial. Traditional methods like algebraic iterative reconstruction and filtered back-projection depend on voxel division for resolution. Our algorithm, blending deep learning with computer graphics rendering, converts 2D projections into light rays for uniform sampling, using a fully connected neural network to depict the 3D temperature field. Although effective in capturing internal details, it demands multiple cameras for varied angle projections, increasing cost and computational needs. We assess the impact of camera number on reconstruction accuracy and efficiency, conducting butane-flame simulations with different camera setups (6 to 18 cameras). The results show improved accuracy with more cameras, with 12 cameras achieving optimal computational efficiency (1.263) and low error rates. Verification experiments with 9, 12, and 15 cameras, using thermocouples, confirm that the 12-camera setup as the best, balancing efficiency and accuracy. This offers a feasible, cost-effective solution for real-world applications like engine testing and environmental monitoring, improving accuracy and resource management in temperature measurement.
Keywords: 3D temperature filed, Deep learning, Non-contact temperature measurement, Radiation thermometry
Pyrometry holds great importance in modern industry and scientific research. It serves as an accurate method for determining the temperature of objects in extreme environments, playing a critical role in high-temperature processes within fields such as materials science, energy development, and aerospace. For example, in the domain of material synthesis, the meticulous control of temperature in high-temperature environments directly affects the quality and properties of the materials produced. Moreover, the utilization of three-dimensional (3D) temperature-field provides valuable information about surface temperatures and temperature distributions within objects. This technique proves indispensable in areas such as combustion, chemical reaction engineering, and biomedical research, where complex heat flows and transfer processes are often involved. By employing 3D temperature-field measurements, we can enhance the accuracy of modeling and comprehending these complex processes [1–3].
Mainstream methods for advanced optical diagnosis include one-and two-dimensional techniques, such as traditional laser-induced fluorescence (LIF) and particle image velocimetry (PIV). These techniques find widespread use in measuring flame temperature and velocity [4–6]. However, the limited spatial resolution of the detection equipment hampers high-resolution 3D reconstruction of the combustion field. Furthermore, the complex test systems require sufficient space in the experimental environment to arrange equipment. While infrared-radiation thermometry serves to obtain the surface temperature of high-temperature objects, it falls short in accurately reconstructing the 3D temperature field of a flame.
Cai et al. [7–9] innovatively used a single camera equipped with an endoscope based on chemiluminescence-computational-tomography technology, to achieve high spatial and temporal resolutions in reconstructing a flame’s luminescence field. This approach enabled the comprehensive study of combustion processes in turbulent and swirling flames. It is important to note that endoscopes introduce limitations in image resolution and significant image distortion due to their optical lenses, which need correction prior to reconstruction. Conventional 3D reconstruction algorithms based on computed tomography of chemiluminescence (CTC) include algebraic reconstruction technology (ART), simultaneous algebraic reconstruction technology (SART), and multiplicative algebraic reconstruction technologies (MART). These algorithms share similar reconstruction mechanisms, with the resolution of their results being affected by pixel partitioning. In recent times there has been a growing interest in harnessing deep-learning algorithms for various computational tasks. Consequently, attempts have been made to apply deep learning to CTC reconstruction applications. For example, Worth and Dawson [10] used CTC technology to study vortex-flame interactions in turbulent flames, showcasing the potential of deep learning in this context.
Despite the successful outcomes of previous research endeavors, the inherent inherently unsteady nature of flames necessitates the use of camera sensors with a wide dynamic range, resulting in high equipment costs. This cost challenge also extends to related studies that aim to employ multiple cameras in reconstructing the 3D structure of flames [11, 12]. While this limitation is not a technical problem, it greatly hinders widespread application of these techniques in basic combustion research. Therefore, selecting the most cost-effective camera solution for 3D flame reconstruction becomes imperative. However, an insufficient number of cameras inevitably results in inadequate spatial sampling and reduced reconstruction accuracy. Therefore, using quantitative evaluation to determine the appropriate number of cameras for 3D reconstruction is important.
To study the impact of camera quantity on reconstruction results, we must formulate the best experimental plan. Based on the sampling methodology inherent to the reconstruction algorithm in spatial ray tracing, we introduce the spatial sampling rate as a quantitative index to evaluate the relationship between the number of cameras and reconstruction errors. As the camera count increases, the time required for model training also linearly increases. To strike a balance between reconstruction accuracy and efficiency, we propose computational efficiency as an additional evaluation metric, taking into account both the spatial sampling rate and model-training time.
In our study, we conduct experiments that combine numerical simulations with butane-flame reconstruction. In the numerical simulation, we use a Gaussian temperature distribution to simulate temperature fields of varying complexity. We employ 6, 9, 12, 15, and 18 cameras for the reconstruction process and calculate corresponding metrics, including root-mean-square error, spatial sampling rate, and computational efficiency. Our findings lead us to conclude that the optimal number of cameras for this task is 12. In the subsequent butane-flame reconstruction experiment, we seek to further verify the choice of 12 cameras as a solution that balances cost considerations with reconstruction accuracy. We conduct the reconstruction experiments using 9, 12, and 15 cameras. To assess the accuracy we use thermocouples to measure temperatures at different heights of the flame. We then calculate the relative error by comparing the reconstruction results to the corresponding actual temperature measurements, ultimately confirming that the use of 12 cameras is optimal.
A flame constitutes a highly complex gas-phase chemical-reaction system, with its internal radiation mainly coming from combustion reaction products and partially burned fuel particles. When thermally excited particles relax from a high-energy state to a low-energy state, they emit photons and produce flame radiation [13].
Our flame-temperature-measurement technology, based on Planck’s blackbody radiation law and calibrated through experiment, determines a relationship between the radiation intensity of the target under measurement and the readings from the optical detector. It calculates the temperature of an object by measuring the intensity of radiation energy emitted by that object. Various radiation temperature-measurement methods, predicated on wavelength selection, fall into three categories: The total-radiation method, single-wavelength-radiation temperature-measurement method, and temperature-measurement methods based on radiation-energy ratios for multiple wavelengths. In our study, we employ single-wavelength radiation thermometry. According to Planck’s Law, the relationship between the radiation intensity E(λ, T) at wavelength λ within a flame and its temperature T is expressed as follows [14]:
where λ is the wavelength, c is the speed of light, h is Planck’s constant, and k is Boltzmann’s constant. The integral formula depicting the radiation generated by the flame as it passes through the camera lens onto the imaging plane is [7]:
where I(p) is the integrated radiation intensity received by pixel p on the imaging plane. E(p) and ω(x) are respectively the radiation intensity and the corresponding emissivity at point x in space. Following photoelectric conversion within the camera, the integrated radiation intensity I(p) is converted to a grayscale value G(p). Through calibration tests involving a blackbody furnace, we can establish the relationship between the grayscale value G(p) and radiation intensity I(p) as Eq. (3):
where A and B are constants. In our approach, a two-dimensional (2D) temperature projection can be calculated from a flame-projection image collected by a camera, leveraging the principles mentioned above. Our prior research improved grayscale imaging by doping K element from K_{2}SO_{4} into the flame [15–17]. Therefore, in this study we conduct single-spectral-line temperature measurements by installing a narrowband filter with a central wavelength of 768 nm and a bandwidth of 10 nm in front of the camera lens.
Based on the principle of radiation thermometry, differentiable rendering technology from computer graphics and universal approximation theorem, we employ a multilayer perceptron (MLP) to represent the 3D temperature field [18]. Spatial-position coding technology is used to map input spatial coordinates to a high-dimensional space, improving the network’s ability to represent rapidly changing regions within the temperature field. Meanwhile, differentiable rendering technology expresses the flame-radiation projection process as a differentiable function. This function enables the calculation of the reprojection result for the light, and MLP training is conducted by reducing the mean square error (MSE) between the reprojection result and the actual projection.
As shown in Fig. 1, we generate a dataset of rays based on the intrinsic and extrinsic parameters of the camera, and the images collected by it. The light dataset consists of the origin, direction, and pixel projection values. Sampling occurs at equal steps along the ray.
The network structure of the MLP is shown in Fig. 2, with the input vector shown in green, the middle-hidden layers in yellow, and the output vector in orange. The input Γ(x) is the 3D coordinate of the sample point x encoded by Eq. (4), for L = 5. The output of the MLP is the temperature T. The numbers within each block indicate the dimensions of the respective vectors. All layers are fully connected and employ the rectified-linear-unit (ReLU) activation function. The output layer also utilizes ReLU to ensure that output results remain non-negative. Following position encoding, the input passes through six fully-connected layers, each with a width of 256, using the ReLU activation function. Subsequently, two downsampling layers with widths of 128 and 64 are concatenated to reduce dimensionality. Ultimately, the temperature corresponding to the given point serves as the output. To address issues related to vanishing and exploding gradients in deep networks, we introduce a residual connection. This connection involves the concatenation of the input vector with the vector from the fourth layer. In the diagram, the symbol + represents the concatenation of vectors.
In this study, we employ a simplified forsward-imaging pinhole model to describe the mapping relationship between 3D space and 2D projection. To facilitate the reconstruction of the 3D temperature field, we simplify the camera as a viewpoint and an imaging plane. Light is defined by the viewpoint coordinates and pixel direction. We establish a flame thermal and optical radiation model, as shown in Eq. (5). The projection value for each pixel p is the light integral of the radiation intensity from the sample point across the reconstructed target domain:
where n and f represent the minimum and maximum values along the path from the sample point on the ray to the imaging plane, respectively. t(s) represents the temperature value at sample point s, while T(p) is the temperature value of pixel p. The internal and external parameter matrices of the camera determine the mapping relationship between pixel p and sample point s, as shown in Eq. (6) [19]:
where (u, v) is the position of pixel p in the pixel coordinate system and Z_{c} is the distance from the spatial sample point in the camera coordinate system to the imaging plane. f_{x} and f_{y} represent the focal lengths of pixels in the x- and y-directions respectively. c_{x}, c_{y} is the center coordinate of the imaging plane, a fixed camera parameter obtained through calibration experiments [19]. R and T represent the rotation and translation matrices of the camera respectively. (X_{w}, Y_{w}, Z_{w}) is the coordinate of the spatial sample points s in the world coordinate system.
Based on the above model, we reconstruct the 3D flame temperature field by controlling the number of cameras. This allows us to conduct further investigations of the impact of camera quantity on the reconstruction outcome. The camera performs uniform sampling along the light ray, as shown in Fig. 3. Consequently, we obtain the coordinates of the sample point at any position along the light ray within the reconstructed target domain, corresponding to each pixel of the camera. Figure 3 shows the sampling effect on the reconstructed target domain, using four cameras in a 2D schematic diagram.
The accuracy of temperature-field reconstruction is closely linked to the distribution of sample points. A higher density of sample points leads to improved accuracy in capturing the finer details of the 3D flame structure. More accurate camera angles are achieved when the camera’s field of view comprehensively covers the flame. However, increasing the number of cameras escalates experimental costs and computational-resource requirements. Therefore, we propose a spatial sampling rate related to the number of cameras, as a metric to evaluate the impact of camera quantity on reconstruction errors. To quantify the spatial sampling rate accurately, we discretize the reconstruction target domain into a voxel set of dimensions 100 × 100 × 100. The spatial sampling rate corresponding to each camera number can be evaluated by calculating the percentage of voxels containing sample points relative to the total number of pixels.
Initially, numerical simulations are conducted to study the impact of the spatial sampling rate on the results of temperature-field reconstruction. In this numerical simulation, a 3D Gaussian distribution is employed to simulate the temperature field of an explosive fireball. We generate single-, double-, and triple-fireball configurations to represent temperature fields of varying complexity. A single spherical temperature distribution is generated based on Eq. (7):
where T_{max} is the maximum value of the temperature field, set to 1,000 K for this study. The parameter N is used to control the temperature distribution, as shown in Eq. (8):
where (x_{0}, y_{0}, z_{0}) is the center coordinate of a single spherical temperature field, and R is the fireball radius.
To study the impact of the spatial sampling rate on the reconstruction results, we conduct temperature-field simulations using 6, 9, 12, 15, and 18 cameras. These cameras are centered on the origin coordinates of the temperature field and evenly arranged at intervals of 60°, 40°, 30°, 24°, and 20° respectively, at the same horizontal height, as shown in Fig. 4. In Fig. 4, it is evident that the temperature field of a single Gaussian distribution exhibits smooth and continuous characteristics internally. The temperature fields of the double and triple Gaussian distributions contain both a smooth region within the temperature field and rapidly changing regions at the junctions of two Gaussian distributions.
To evaluate the impact of different spatial sampling rates on the 3D temperature-field-reconstruction results, we employ the root-mean-square error (RMSE) to calculate the difference between the reconstructed and original temperature fields. The calculation formula is shown in Eq. (9):
where T_{i} and T–_{i} represent respectively the reconstructed and original temperature fields, while N indicates the number of voxels into which the reconstructed temperature field is divided.
Figures 5–7 show 2D slices of the three temperature fields reconstructed using 6, 9, 12, 15, and 18 cameras at height z = 0, along with their corresponding relative errors. As evident from the figures, the errors mainly cluster at the flame boundary, with minimal errors observed inside and outside the flame. The relative-error map provides further insight into the effectiveness of temperature-field reconstruction.
To conduct a more in-depth analysis of the impact of the spatial sampling rate on temperature-field reconstruction, we calculate both the spatial sampling rate corresponding to different camera numbers, and the RMSE of the reconstruction results. Figure 8 displays the reconstruction error and spatial sampling rate corresponding to the number of cameras utilized. The histogram in Fig. 8 shows the number of cameras increasing, while the temperature-field-reconstruction error consistently decreases. Compared to the double- and triple-fireball temperature fields, the single-fireball temperature fields exhibit minor reconstruction errors, and the spatial sampling rate has minimal impact on the reconstruction results. Due to their more complex structures, the double- and triple-fireball temperature fields have relatively larger reconstruction errors, which notably diminish as the number of cameras increases. The line graph in Fig. 8 illustrates that the spatial sampling rate increases as the number of cameras rises. Notably, the increment in sampling rate is most substantial when the number of cameras increases from 9 to 12.
Although increasing the number of cameras can improve reconstruction accuracy, it also extends the model-training time and consumes more computational resources. Therefore, we conduct a further analysis of the model’s computational efficiency under different spatial sampling rates. In order to better evaluate the calculation efficiency, we normalize the spatial sampling rate and calculation time respectively. The computational efficiency is the ratio of the normalized spatial sampling rate to the normalized training time, as listed in Table 1. We set the normalized minimum value of the spatial sampling rate as 0.265, and the normalized minimum value of the training time is 11 min. As shown in Table 1, the model achieves the highest computational efficiency of 1.263 when using 12 cameras. After careful analysis, considering both reconstruction accuracy and computational efficiency, we select 12 cameras spaced 30° apart as the optimal number for temperature-field reconstruction.
TABLE 1. Spatial sampling rate and computational efficiency.
Number of Cameras | Spatial Sampling Rate | Training Time (min) | Computational Efficiency (%) |
---|---|---|---|
6 | 0.2659 | 12 | 0.329 |
9 | 0.2819 | 19 | 0.773 |
12 | 0.3133 | 25 | 1.263 |
15 | 0.3284 | 32 | 1.105 |
18 | 0.3415 | 39 | 1.000 |
To verify the effect of different camera numbers on an actual flame’s temperature-field reconstruction, we conduct a 3D temperature-field-reconstruction experiment on a butane flame. The maximum temperature of a butane flame can reach 1,400 K [20]. In this study we use thermocouples to measure temperatures at five distinct points, at different flame heights. We then compare the reconstructed temperature values at the corresponding positions to evaluate the reconstruction error of the 3D temperature field.
To address the rapid changes in the jet flame resulting from airflow disturbances, we adopt a synchronous control system, as shown in Fig. 9(a). This system ensures that all cameras can simultaneously collect flame projection data. The synchronous control system mainly relies on a signal generator to transmit acquisition signals to all cameras simultaneously. Subsequently, it transmits the image data captured by all cameras to the workstation for processing via a high-speed data acquisition card. We use 9, 12, and 15 CMOS cameras for the reconstruction experiments, maintaining a distance of 50 cm between each camera and the flame. The cameras are centered on the target flame and evenly spaced 30° apart, as shown in Fig. 9(b). In addition, each camera’s lens is equipped with a narrowband filter featuring a central wavelength of 768 nm and a bandwidth of 10 nm.
According to the established grayscale-to-temperature fitting relationship from prior calibration experiments, we sequentially process the grayscale images collected by the 12 cameras. This allows us to obtain a 2D temperature projection of the butane flame from every direction. Figure 10(a) displays a grayscale image of the flame captured by the camera on the far right; Figure 10(b) showcases the corresponding 2D temperature map. The maximum temperature recorded was 1,242.85 K.
Figure 11 shows the change in MSE loss values with the number of epochs during neural-network training. In this study, one traversal through all rays is considered to constitute one epoch. As depicted in the figure, the final convergence values for 9, 12, and 15 cameras were 1,398, 804, and 630 respectively. A comparative analysis of these three curves reveals that the convergence performance for 12 and 15 cameras is better than that for 9 cameras. Taking into account the cost considerations related to the cameras, it is determined that 12 cameras represent the best experimental solution.
Figure 12 shows the reconstruction results for the butane flame’s temperature field, employing different numbers of cameras. The figure reveals that the shapes of the three reconstructed temperature fields are relatively consistent, but with varying temperature distributions. To verify the accuracy of the temperature-field reconstruction, we use a K-type thermocouple to measure temperature values at the five sample points depicted in Fig. 10. We then calculate the relative error. Table 2 shows the findings, indicating that when the camera numbers were 9, 12, and 15, the maximum relative errors between the reconstruction results and the thermocouple measurements were 15.06%, 5.76%, and 2.59% respectively. Among the three sets of reconstruction results, the relative error at sample point 1 is the smallest among the five sample points. Additionally, the changing trend of flame temperature is consistent with the thermocouple measurements, with temperature decreasing as height decreases. Comparing the reconstruction results obtained using 12 and 15 cameras, the maximum relative errors are 5.76% and 2.59% respectively, falling within an acceptable range.
TABLE 2. Verification of radiation-temperature-measurement results based on thermocouples.
Number of Cameras | Position | Thermocouple Measurements (K) | Reconstructed Value (K) | Relative Error (%) |
---|---|---|---|---|
9 | 1 | 1,132 | 1,191 | 5.21 |
2 | 903 | 839 | 7.09 | |
3 | 769 | 712 | 7.41 | |
4 | 742 | 695 | 6.33 | |
5 | 704 | 598 | 15.06 | |
12 | 1 | 1,140 | 1,145 | 0.44 |
2 | 942 | 912 | 3.18 | |
3 | 747 | 724 | 3.08 | |
4 | 733 | 714 | 2.59 | |
5 | 712 | 671 | 5.76 | |
15 | 1 | 1,141 | 1,143 | 0.18 |
2 | 927 | 903 | 2.59 | |
3 | 755 | 745 | 1.32 | |
4 | 739 | 720 | 2.57 | |
5 | 695 | 682 | 1.87 |
In this study, our main focus was on analyzing the impact of the number of cameras on the results of 3D temperature-field reconstruction. This is because the reconstruction results are heavily affected by the density of spatial-sampling points. A higher coverage of the reconstruction area by sample points generally leads to smaller reconstruction errors. Therefore, we proposed the spatial sampling rate as a quantitative index to evaluate how the number of cameras affects reconstruction results. However, in addition to the number of cameras, the factors influencing the spatial sampling rate are also related to factors such as the resolution of the camera’s imaging plane and the field of view of the lens. However, considering that our model of the experimental equipment was relatively simple, we focused on studying the impact of the number of cameras on the spatial sampling rate. We analyzed this by conducting numerical simulations and butane-flame-reconstruction experiments. In our assessment, we considered factors such as computational efficiency, equipment cost, and reconstruction accuracy. The results determined that the setup with 12 cameras spaced 30° apart was optimal.
The number of cameras and the chosen sampling method along the ray mainly affect the spatial sampling rate. Under identical camera-parameter settings, an increase in the number of cameras results in denser sampling points generated in space, thereby increasing the spatial sampling rate. Similarly, when camera settings remain unchanged, reducing the sampling step along the ray generates more sample points and increases the corresponding spatial sampling rate. Following the discretization of the reconstruction target domain into 100 × 100 × 100 voxels, we calculated the corresponding spatial sampling rates for camera configurations of 6, 9, 12, 15, and 18. Our calculations verified that the spatial sampling rate increased with an increase in the number of cameras. To further analyze the impact of the number of cameras on reconstruction results, we conducted both simulated temperature-field reconstruction experiments and empirical butane-flame temperature-field reconstruction experiments.
In our numerical simulation, we simulated a fireball with a spherical temperature distribution that adhered to a Gaussian distribution. The maximum temperature at the center of this fireball was set to 1,000 K. We simulated temperature fields with different complexities by creating single, double, and triple fireballs. The numerically simulated temperature field included both smooth temperature changes within the flame and abrupt temperature changes at the flame’s boundary. This allowed us to verify the impact of different spatial sampling rates on the reconstruction results. The simulation results showed that the temperature field of a single fireball, characterized by a relatively simple structure, was less affected by the number of cameras. Conversely, the temperature fields of the double- and triple-fireball scenarios exhibited a decrease in reconstruction error as the number of cameras increased. To select the optimal number of cameras, we used the normalized ratio of spatial sampling rate to model-training time as a measure of computational efficiency, with 12 cameras achieving the highest reconstruction efficiency of 1.263.
For the butane flame’s temperature-field reconstruction, we used three experimental schemes employing 9, 12, and 15 cameras, with thermocouples serving as verification tools to determine reconstruction errors. The maximum relative errors observed in the reconstruction results for the three schemes were 15.06%, 5.76%, and 2.59% respectively. Additionally, the average relative errors were 8.22%, 3.01%, and 1.71% respectively. After a comprehensive evaluation considering both reconstruction accuracy and cost, it was determined that the use of 12 cameras represented the best solution.
Our approach involved evaluating different experimental plans based on the number of cameras and the light-sampling method in the reconstruction algorithm. We proposed two key indicators, namely the spatial sampling rate and computational efficiency, to evaluate their impact on reconstruction results. This evaluation method helps to reduce equipment cost and improve calculation efficiency in temperature measurement, while ensuring reconstruction accuracy.
The authors thank all reviewers, editors, and contributors for their contributions and suggestions, as well as all members of the OSEC Laboratory.
Supported in part by the National Natural Science Foundation of China (NSFC) (Grant no. 52075504); the Fund for Shanxi 1331 Project Key Subject Construction, State Key Laboratory of Quantum Optics, and Quantum Optics Devices (Grant no. KF202301); the Shanxi Key Laboratory of Advanced Semiconductor Optoelectronic Devices and System Integration (Grant no. 2023SZKF11).
The authors declare no conflicts of interest.
The data underlying the results presented in this paper are not publicly available at this time, but may be obtained from the authors upon reasonable request.
TABLE 1 Spatial sampling rate and computational efficiency
Number of Cameras | Spatial Sampling Rate | Training Time (min) | Computational Efficiency (%) |
---|---|---|---|
6 | 0.2659 | 12 | 0.329 |
9 | 0.2819 | 19 | 0.773 |
12 | 0.3133 | 25 | 1.263 |
15 | 0.3284 | 32 | 1.105 |
18 | 0.3415 | 39 | 1.000 |
TABLE 2 Verification of radiation-temperature-measurement results based on thermocouples
Number of Cameras | Position | Thermocouple Measurements (K) | Reconstructed Value (K) | Relative Error (%) |
---|---|---|---|---|
9 | 1 | 1,132 | 1,191 | 5.21 |
2 | 903 | 839 | 7.09 | |
3 | 769 | 712 | 7.41 | |
4 | 742 | 695 | 6.33 | |
5 | 704 | 598 | 15.06 | |
12 | 1 | 1,140 | 1,145 | 0.44 |
2 | 942 | 912 | 3.18 | |
3 | 747 | 724 | 3.08 | |
4 | 733 | 714 | 2.59 | |
5 | 712 | 671 | 5.76 | |
15 | 1 | 1,141 | 1,143 | 0.18 |
2 | 927 | 903 | 2.59 | |
3 | 755 | 745 | 1.32 | |
4 | 739 | 720 | 2.57 | |
5 | 695 | 682 | 1.87 |