Deep learning has been extensively and significantly applied in a variety of fields, including non-line-of-sight (NLOS) imaging [1], medical imaging [2], optical communications [3–5], and more. Recently, this interest has extended to fiber optic sensors [6–9], whose applications are rapidly expanding.

Intrusion detection typically employs sensing methods such as radar, infrared surveillance systems, and CCD cameras [10–12]. However, these methods are often costly, have low sensitivity in bad weather, and are limited to clear line-of-sight conditions, making it difficult to detect a wide range of intrusions. In contrast, fiber optic sensors offer distinct advantages, including high sensitivity, remote sensing capability, and immunity to electromagnetic interference. As a result, they have been successfully applied in intrusion detection systems. These sensors are developed using technologies such as the optical time domain reflectometer (OTDR) [13] and the optical fiber interferometer [14]. The former can only be detected when the fiber is cut or the light reflection is sufficient to be detected, while the latter often faces reliability issues.

Methods using speckle patterns in multimode fibers, which are somewhat immune to such problems, have been introduced for a variety of applications, including intrusion detection [15–19]. These methods have the additional benefits of being cost-effective and compact. This paper presents a deep learning-based weight sensor that uses the speckle pattern of a multimode fiber and demonstrates its applicability to intrusion detection. The detection system integrates the weight sensor with an image classifier using the Google Teachable Machine [20] and facilitates real-time intrusion notification via the ThingSpeak cloud platform [21], an open source Internet of Things (IoT) application developed by MathWorks.

When coherent light emitted from a laser enters a multimode fiber, interference between modes occurs as it propagates through the fiber. This results in the formation of bright and dark regions on a screen at a certain distance from the fiber’s end-face, which is known as the optical speckle pattern. If M is the total number of modes of light transmitted in the optical fiber, then the initial intensity of light at a point I₀ (x, y) on the screen can be expressed as follows [15, 18]:

where a_0m (x, y) and φ_0m (x, y) are m-th mode amplitude and phase distributions over the x-y projection plane, respectively. When external perturbations are applied to the optical fiber, the modified intensity I(x, y) can be expressed as

where ∆a_m (x, y) and ∆φ_m (x, y) are the amplitude and phase variations of the m-th mode, respectively. It can be assumed that the beam propagating through the optical fiber experiences minimal loss (assuming a lossless waveguide), and thus the amplitude variation can be considered as ∆a_m (x, y) ≈ 0. External pressure changes the refractive index or cross-sectional size of the fiber, thereby affecting the path lengths of the modes and causing a phase-related change, ∆φ_m (x, y). Consequently, alterations to the speckle patterns occur. However, inappropriate application of external pressure may cause an afterimage to persist even after the pressure is removed. This phenomenon, known as hysteresis, is a result of the interaction between the applied pressure and the underlying material.

Two types of optical weight sensors were constructed for the purpose of investigating the hysteresis characteristics, as depicted in Fig. 1. In Fig. 1(a), the weight is applied directly to the optical fiber using a metal plate. In contrast, in Fig. 1(b), it is designed so that the elastic plate can flex slightly, indirectly affecting the optical fiber without applying direct pressure. The elastic plate is supported at its four corners by pedestals, thereby maintaining a small gap between the plate and the underlying surface. In the direct method illustrated in Fig. 1(a), the metal plate had a top dimension of 300 × 200 mm, a thickness of 1.8 mm, and a weight of 0.8 kg. In Fig. 1(b), the indirect method involved a top dimension of 400 × 400 mm, a thickness of 7.0 mm, and support from four pedestals, each 10.0 mm high. The plate is composed of polypropylene, with an elastic modulus of 16 × 10³ kgf/cm². To investigate hysteresis, a 3.0 kg weight was applied for 10 minutes and then removed to observe the optical response return to its initial state. Figure 2 shows the speckle patterns observed in this experiment, with zero-mean normalized correlation coefficients (ZNCC) of 0.94 and 0.99, respectively. The comparison in Fig. 2 clearly demonstrates that the indirect method in Fig. 1(b) exhibits superior hysteresis performance compared to the direct method in Fig. 1(a).

Figure 1. Two types of optical weight sensors for studying hysteresis characteristics. (a) Direct method, where pressure is applied directly to the multimode fiber using a metal plate, and (b) indirect method, where an elastic plate is directly under pressure and indirectly transmitted to the multimode fiber.

Figure 2. Results of a hysteresis test to measure the return to the initial state after applying a 3 kg weight for 10 minutes: (a) Speckle pattern for the direct method, which has changed greatly (red dotted circles), and (b) speckle pattern for the indirect method, which has changed very little.

This paper employs the classification capabilities of a deep learning neural network to quantify weight based on the optical speckle pattern of a weight sensor constructed with a multimode fiber, as illustrated in Fig. 1(b). While AlexNet [22], VGG [23], ResNet [24], and similar models are commonly used for pattern classification tasks in the field of convolutional neural networks (CNNs), a Teachable Machine [20], a web-based, real-time pattern classification tool developed by Google, was used for its simplicity. The Teachable Machine is a no-code AI learning tool based on the CNN architecture known as MobileNet [25]. It operates in a transfer learning mode using pre-trained results, which facilitates rapid learning and achieving desired results even with limited data. While creating an ideal weight sensor would require extensive data collection and training, this study illustrates the promise of a simplified deep learning-based weight sensor and its potential utility, including applications in intrusion detection systems.

Figure 3 depicts the experimental setup. A laser beam with a wavelength of 532 nm is incident into the multimode fiber. A thermoelectric cooler (TEC) temperature compensator and optical power controller circuit are provided to stabilize the wavelength and optical power. The laser beam, with a measured power of 32.5 mW, was directly applied to an optical connector. The beam then traverses a 15-meter-long multimode fiber with a 62.5 μm core diameter and encounters a weight sensor configuration as depicted in Fig. 1(b). The optical speckle is detected by projecting it from the optical connector end and directly collecting it through a Raspberry Pi camera, which has an image size of 2,450 × 2,450 pixels. Once entered as input data in the Teachable Machine, the image is automatically resized to a 224 × 224 image.

Figure 3. Experimental setup of the proposed deep-learning weight sensor with multimode fiber speckle patterns, and thermoelectric cooler (TEC).

For training purposes, 30 speckle images, ranging from 0.0 to 2.0 kg, were obtained for each 200 g weight increment. This resulted in a total of 330 images. The learning rate was set at 0.01, with 50 epochs, resulting in a final training loss of 0.021. Figure 4(a) presents the confusion matrix, which summarizes the model’s prediction accuracy for each class. Figure 4(b) illustrates the loss per epoch, showing that the model learns well without underfitting or overfitting. The output of the Teachable Machine is expressed as the percentage of similarity with the 11 trained class patterns, indicating the degree to which the input pattern resembles each class.

Figure 4. Confusion matrix and loss-per-epoch plots on the training and test sets. (a) The loss confusion matrix, which provides a summary of the model’s prediction accuracy for each class. (b) The resulting graph of loss per epoch demonstrates that the model learns effectively without underfitting or overfitting.

In order to function as a weight sensor, it is necessary to convert the classified percentage output into a weight value. The predicted weight value, W_pred, can be calculated by considering only the classes with the highest output values, $O_i^{\text{max}}$, and their neighboring classes, O_i−1 and O_i+1, of the Teachable Machine as follows

where W_i represents the measured weight corresponding to the i-th class. As an illustrative example from Fig. 5, the weight can be estimated as W_pred = (0.0 × 9 + 0.2 × 52 + 0.4 × 6) / (9 + 52 + 6) = 0.191 kg, based on the data obtained.

Figure 5. An example of measuring any arbitrary weight on the fiber optic weight sensor using the Teachable Machine.

Figure 6 illustrates the output of the fiber optic weight sensor, which was obtained by 100 g increments of applied weight, ranging from 0.0 to 2.0 kg, over five iterations. The trained data points exhibited an average error of 11.4 g, indicating highly favorable results. In contrast, the untrained points exhibited a larger average error of 243.8 g. Although the output shows some level of consistency overall, the measured results suggest that it may not be suitable for precise weight sensing. However, because sudden weight changes can be detected, it was possible to implement an intrusion detection system, which will be explained in the next section.

Figure 6. Results of the predicted weight using the proposed weight sensor, which was obtained by 100 g increments of applied weight, ranging from 0.0 to 2.0 kg, over five iterations. The solid red circles represent the trained weights, while the blue dotted circles represent the predictions for the untrained weights.

For the purpose of simple intrusion detection, the system can be trained to operate in just two states: The normal state and the intrusion state. The normal state comprises a limited number of speckle patterns, while the intrusion state exhibits countless types of speckle patterns. Despite the difficulty in clearly grasping the internal principles of the deep learning of the Teachable Machine, all patterns other than the normal speckle patterns are classified as intrusions. Two classes were trained for verification purposes. A total of 30 speckle patterns (representing the normal class) were used at a weight of 0.0 kg, while 300 randomly selected speckle patterns (representing the intrusion class) were used in a range of weights from 0.1 to 2.0 kg. Figure 7 illustrates the test results obtained with 20 g increments of applied weight, ranging from 0.0 to 0.1 kg, and 100 g increments of applied weight, ranging from 0.1 to 2.0 kg, over five iterations. It can be observed that even a slight deviation from 0.0 kg results in classification as an intrusion, indicating high sensitivity to even minor noise and the potential for malfunction. This issue could be mitigated by slightly expanding the range of normal weights. Additionally, the training time of the proposed system is typically less than two minutes, which would facilitate the implementation of an adaptive system using regularly collected speckle patterns of the normal state. Further research in this area will be published in the near future.

Figure 7. Output of intrusion class (%) vs. measured weight (g; log scale), showing the results obtained with 20 g increments of applied weight ranging from 0.0 to 0.1 kg, and with 100 g increments of applied weight ranging from 0.1 to 2.0 kg, over five iterations.

Figure 8(a) depicts the architecture of the real-time intrusion detection system proposed in this paper, which employs a deep learning-based weight sensor. The trained Teachable Machine is exported as a TensorFlow.js model, enabling its integration with Node.js for the development of a customized weight-sensing system [26]. The measured weight is transmitted to the ThingSpeak^TM data collection platform over the Internet. ThingSpeak^TM is a cloud-based platform service that collects and visualizes live data streams, as well as enabling analysis of the collected data. The platform offers the ability to program a real-time notification signal to be sent to individual Pushover accounts when a predefined threshold is exceeded. Additionally, users who are licensed to use the platform have the ability to send data once per second on ThingSpeak^TM.

Figure 8. One potential application of the proposed real-time intrusion detection system. (a) Architecture of the real-time intrusion detection system, (b) data sent to the ThingSpeak^TM platform when a 1.83 kg cat crossed the weight sensor, and (c) example of a message sent to a Pushover app.

To verify the implemented system, real-time notifications were examined to ascertain whether they were transmitted to the Pushover account upon the passage of a 1.83 kg cat across the weight sensor. The data collected and transmitted to the ThingSpeak^TM platform is illustrated in Fig. 8(b). The data indicates that sudden changes following consecutive small fluctuations around 0.0 kg indicate an intrusion. Given the possibility of malfunction due to noise, the system is configured to send alarm signals to the Pushover app when weights exceeding the 500 g threshold are detected cumulatively three times or more. Although a relatively simple algorithm has been employed here, it can be designed to suit the application’s purpose. The alarm signal sent to the Pushover app is illustrated in Fig. 8(c).

The Google Teachable Machine, as applied to the weight sensor in this paper, is based on MobileNet’s transfer learning model. This model facilitates the reuse of a pre-trained model with a new problem for rapid learning of new patterns. However, it may not perform well when generalized to patterns that are dissimilar to the pre-trained pattern. A notable discrepancy was observed in the predicted weights that were not learned, as shown in Fig. 5. Notwithstanding, it is noteworthy that the results did not deviate completely. Based on these results, although it is difficult to use as an accurate weight sensor, it demonstrated the potential for application as a system that can detect intrusion in real time. ThingSpeak^TM, a cloud-based IoT analytics platform, was used for real-time intrusion detection, and real-time intrusion alerts were successfully implemented to notify the user’s Pushover app.

In order to implement an accurate weight sensor, a substantial quantity of sample data is required, and it is necessary to train a deep learning model from scratch. As a means of achieving this, the vision transformer (ViT) model [27], which has recently received considerable attention, appears to be capable of attaining satisfactory performance.

The author would like to thank Prof. Chang-Soo Park of GIST for his helpful advice.

This work was supported by Kyungnam University Foundation Grant 2021.

The author declares that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

All data generated or analyzed during this study are included in this published article. The Teachable Machine model that supports the findings of this study is available at https://teachablemachine.withgoogle.com/models/ADgTnzy4i/.