Information technologies and computer sciences

Using a TensorFlow-Based Neural Network for Gesture Recognition and Control of a Bionic Prosthesis

Authors

R. I. Bilyy
Vinnytsia National Technical University ROR
DOI https://doi.org/10.31649/1997-9266-2024-174-3-71-77

Keywords

neural network TensorFlow gesture recognition bionic prosthesis electromyography recognition accuracy

Abstract

This study analyzes in detail the recognition accuracy of various gestures using a neural network built on TensorFlow to control a bionic prosthesis. By conducting a series of experiments using data sets and model parameters, it is investigated how effectively the neural network recognizes various gestures in the context of prosthesis control. The results of the experiments will allow us to understand the potential limitations and opportunities of this technology for practical application in real conditions. The neural network model and architecture, activation functions and parameters used for training are described. Learning parameters that affect the effectiveness of model learning are also considered. To evaluate the effectiveness of the model, results such as model evaluation metrics, model performance graphs, and confusion matrix are used to evaluate the level of reliability and performance of the trained neural network. Method of collecting electromyography (EMG) data is proposed, which consists in the use of electrical signals arising in muscles during their contraction. This process involves placing electrodes on the surface of the skin over the muscles being analyzed to record the electrical signals that occur at the moment of muscle activity. This approach allows collecting objective data on muscle activity and their movements, which can then be used to train a neural network and further use it in controlling a bionic prosthesis. A series of experiments was conducted to assess the accuracy of recognition of various movements using the trained model. Analyzing the results of the experiments, it is possible to understand how efficiently and reliably the trained model works in real conditions and how suitable it is for practical use in control systems of bionic prostheses.

381 163

How to Cite

[1]
“Using a TensorFlow-Based Neural Network for Gesture Recognition and Control of a Bionic Prosthesis”, Вісник ВПІ, no. 3, pp. 71–77, Jun. 2024, doi: 10.31649/1997-9266-2024-174-3-71-77.

Author Biography

R. I. Bilyy, Vinnytsia National Technical University

Post-Graduate Student t of the Chair of Biomedical Engineering and Opto-Electronic Systems

References

A. M. Elbreki, S. Ramdan, F. Mohamed, K. Alshari, Z. Rajab, and B. Elhub, “Practical Design of an Upper Prosthetic Limb Using Three Dimensional Printer with an Artificial Intelligence Based Controller,” in Proceedings of the 2022 International Conference on Engineering & MIS (ICEMIS), Istanbul, Turkey, Jul. 4-6, 2022, pp. 1-6.
O. Kerdjidj, K. Amara, F. Harizi, and H. Boumridja, “Implementing Hand Gesture Recognition using EMG on the Zynq Circuit,” IEEE Sensors Journal, vol. 9, pp. 10054-10061, 2023. https://doi.org/10.1109/JSEN.2023.3259150 .
N. M. Hye, U. Hany, S. Chakravarty, L. Akter, and I. Ahmed, “Artificial Intelligence for sEMG-based Muscular Movement Recognition for Hand Prosthesis," IEEE Access, vol. 11, pp. 38850-38863, 2023. https://doi.org/ 10.1109/ACCESS.2023.3267674 .
L. C. Zayia, and P. Tadi, “Neuroanatomy, motor neuron,” StatPearls, StatPearls Publishing, Tampa, FL, USA, 2021.
L. Mc Manus, G. De Vito, and M.M. Lowery, “Analysis and biophysics of surface EMG for physiotherapists and kinesiologists: Toward a common language with rehabilitation engineers,” Frontiers in Neurology, vol. 11, p. 576729, 2020. https://doi.org/10.3389/fneur.2020.576729 .
D. Mahmood, H. N. Riaz, and H. Nisar, “Advances in Non-Invasive Biomedical Signal Sensing and Processing with Machine Learning: Introduction to Non-Invasive Biomedical Signals for Healthcare,” Springer International Publishing, Cham, Switzerland, 2023.
V. Asanza, E. Peláez, F. Loayza, L. L. Lorente-Leyva, and D. H. Peluffo-Ordóñez, “Identification of Lower-Limb Motor Tasks via Brain–Computer Interfaces: A Topical Overview,” Sensors, vol. 22, p. 2028, 2022. https://doi.org/ 10.3390/s22052028 .
I. Daimiel Naranjo et al., “Radiomics and Machine Learning with Multiparametric Breast MRI for Improved Diagnostic Accuracy in Breast Cancer Diagnosis,” Diagnostics, vol. 11, p. 919, 2021. https://doi.org/ 10.3390/diagnostics11060919 .
M. A. Gulum, C. M. Trombley, and M. Kantardzic, “A Review of Explainable Deep Learning Cancer Detection Models in Medical Imaging,” Applied Sciences, vol. 11, p. 4573, 2021. https://doi.org/10.3390/app11104573 .
M. Zivkovic, et al. “A Hybrid CNN and XGBoost Model Tuned by Modified Arithmetic Optimization Algorithm for COVID-19 Early Diagnostics from X-ray Images,” Electronics, vol. 11, p. 3798, 2022. https://doi.org/ 10.3390/electronics11223798
A. Toro-Ossaba, J. Jaramillo-Tigreros, J. C. Tejada, A. Peña, A. López-González, and R.A. Castanho, “LSTM Recurrent Neural Network for Hand Gesture Recognition Using EMG Signals,” Applied Sciences, vol. 12, p. 9700, 2022. https://doi.org/10.3390/app12199700 .
Á. L. Valdivieso Caraguay, J. P. Vásconez, L. I. Barona López, and M. E. Benalcázar, “Recognition of Hand Gestures Based on EMG Signals with Deep and Double-Deep Q-Networks,” Sensors, vol. 23, p. 3905, 2023. https://doi.org/10.3390/s23083905 .
Y. Xu, P. C. Barbosa, J. S. da Cunha Neto, L. Zhang, V. H. C. de Albuquerque, V. Shanmuganathan, and S. Pasupathi, “Development of intelligent and integrated technology for pattern recognition in EMG signals for robotic prosthesis command,” Expert Systems, vol. 40, p. e13109, 2023. https://doi.org/10.1111/exsy.13109 .
K. Dokic, M. Martinovic, and D. Mandusic, “Inference speed and quantisation of neural networks with TensorFlow Lite for Microcontrollers framework,” in Proceedings of the 2020 5th South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference (SEEDA-CECNSM), Corfu, Greece, Sep. 25-27, 2020, pp. 1-5.
Y. Fang, J. Yang, D. Zhou, and Z. Ju, “Modelling EMG driven wrist movements using a bio-inspired neural network,” Neurocomputing, vol. 470, pp. 89-98, 2022. https://doi.org/10.1016/j.neucom.2021.10.104 .
A. K. Mukhopadhyay, S. Samui, “An experimental study on upper limb position invariant EMG signal classification based on deep neural network,” Biomed. Signal Process. Control. 2020;55:101669. https://doi.org/10.1016/j.bspc.2019.101669 .
D. Buongiorno, et al., “Deep learning for processing electromyographic signals: A taxonomy-based survey,” Neurocomputing, vol. 452, pp. 549-565, 2021. https://doi.org/10.1016/j.neucom.2020.06.139 .
V. Asanza, et al., “SSVEP-EEG Signal Classification based on Emotiv EPOC BCI and Raspberry Pi,” IFAC-PapersOnLine, vol. 54, pp. 388-393, 2021. https://doi.org/10.1016/j.ifacol.2021.10.287 .
A. Constantine, V. Asanza, F. L. Loayza, E. Peláez, and D. Peluffo-Ordóñez, “BCI System using a Novel Processing Technique Based on Electrodes Selection for Hand Prosthesis Control,” IFAC-PapersOnLine, vol. 54, pp. 364-369, 2021. https://doi.org/10.1016/j.ifacol.2021.10.283.
D. Tinoco-Varela, J. A. Ferrer-Varela, R. D. Cruz-Morales, and E. A. Padilla-García, “Design and Implementation of a Prosthesis System Controlled by Electromyographic Signals Means, Characterized with Artificial Neural Networks,” Micromachines, vol. 13, p. 1681, 2022. https://doi.org/10.3390/mi13101681 .
O. K. Kolesnytskyj, I. V. Bokotsey, and S. S. Yaremchuk, “Optoelectronic implementation of pulsed neurons and neural networks using bispin-devices,” Opt. Mem. Neural Networks, no. 19, pp. 154-165, 2010. https://doi.org/10.3103/S1060992X10020062 .