A Security Enhancement of the Message Queuing Telemetry Transport Protocol Using Transport Layer Security and Hash-Based Message Authentication Code
DOI:
https://doi.org/10.21460/jutei.2026.102.526Abstract
Message Queuing Telemetry Transport (MQTT) is a lightweight communication protocol widely used in Internet of Things (IoT) systems due to its efficiency and low resource consumption. However, MQTT does not provide built-in security mechanisms, making it vulnerable to attacks such as Man-in-the-Middle (MITM). This study aims to analyze the impact of implementing Transport Layer Security (TLS) and Hash-Based Message Authentication Code (HMAC-SHA256) on the security and performance of the MQTT protocol.
An experimental method was employed by comparing four communication scenarios: MQTT without security, MQTT with TLS, MQTT with HMAC-SHA256, and MQTT with the combination of TLS and HMAC-SHA256. The system was implemented using an ESP32, a DHT22 sensor, and a Mosquitto broker. Security was evaluated through MITM attack scenarios, while performance was assessed using Quality of Service (QoS) parameters, including latency, throughput, jitter, packet loss, memory usage, and CPU utilization.
The results indicate that TLS ensures communication confidentiality, while HMAC-SHA256 provides data integrity and message authentication. The combination of TLS and HMAC-SHA256 offers the highest level of security, although it introduces performance overhead. These findings demonstrate that the selection of security mechanisms should consider the trade-off between communication security and system performance in IoT applications.
Keywords: Internet of Things, MQTT, TLS, HMAC-SHA256, security, Quality of Service.
References
R. Chataut, A. Phoummalayvane, and R. Akl, “Unleashing the Power of IoT : A Comprehensive Review of IoT Applications and Future Prospects in Healthcare, Agriculture, Smart Homes, Smart Cities, and Industry 4.0,” Sensosrs, vol. 23, 2023.
[2] K. Küçük, A. Sevin, C. Bay, M. A. Ebleme, and Ü. Çavus, “A survey on communication protocols and performance evaluations for Internet of Things,” Digit. Commun. Networks, vol. 8, no. November 2019, pp. 1094–1104, 2022, doi: 10.1016/j.dcan.2022.03.013.
[3] M. Al-fayoumi and Q. A. Al-haija, “Capturing low-rate DDoS attack based on MQTT protocol in software Defined-IoT environment,” Array, vol. 19, no. June, p. 100316, 2023, doi: 10.1016/j.array.2023.100316.
[4] P. Verma, “MQTT Security Challenges and Solutions,” Int. J. Enhanc. Res. Sci. Technol. Eng., vol. 11, no. 4, 2022, [Online]. Available: https://www.researchgate.net/publication/360919708
[5] M. A. Khan et al., “A deep learning-based intrusion detection system for MQTT enabled IoT,” Sensors, vol. 21, no. 21, 2021, doi: 10.3390/s21217016.
[6] S. Chen, S. Jero, M. Jagielski, A. Boldyreva, and C. Nita-rotaru, “Secure Communication Channel Establishment : TLS 1 . 3 ( over TCP Fast Open ) versus QUIC ∗,” J. Cryptol., vol. 34, no. 3, pp. 1–41, 2021, doi: 10.1007/s00145-021-09389-w.
[7] A. R. Alkhafajee, A. M. A. Al-Muqarm, A. H. Alwan, and Z. R. Mohammed, “Security and Performance Analysis of MQTT Protocol with TLS in IoT Networks,” in Proceedings of the 4th International Iraqi Conference on Engineering Technology and Their Applications (IICETA 2021), 2021. doi: 10.1109/IICETA51758.2021.9717495.
[8] T. Prantl, L. Iffländer, S. Herrnleben, S. Engel, S. Kounev, and C. Krupitzer, “Performance Impact Analysis of Securing MQTT Using TLS,” in Proceedings of the ACM/SPEC International Conference on Performance Engineering (ICPE 2021), 2021. doi: 10.1145/3427921.3450253.
[9] J. Mades, G. Ebelt, B. Janjic, F. Lauer, C. C. Rheinlander, and N. Wehn, “TLS-Level Security for Low Power Industrial IoT Network Infrastructures,” in Proceedings of the 2020 Design, Automation and Test in Europe Conference and Exhibition (DATE 2020), 2020. doi: 10.23919/DATE48585.2020.9116285.
[10] M. Backendal, M. Bellare, F. Günther, and M. Scarlata, “When Messages are Keys: Is HMAC a dual-PRF?,” in 43rd International Cryptology Conference (CRYPTO 2023), 2023. doi: 10.1007/978-3-031-38548-3_22.
[11] M. S. Turan, “Keyed-Hash Message Authentication Code (HMAC),” 2024. doi: 10.6028/NIST.SP.800-224.ipd.
[12] B. Angkasa, Asriyanik, and A. Pambudi, “IMPLEMENTASI ALGORITMA HMAC-SHA-256 UNTUK KEAMANAN PRODUK,” Bit (Fakultas Teknol. Inf. Univ. Budi Luhur), vol. 20, no. 2, 2023.
[13] Ö. Şeker, G. Dalkılıç, and U. C. Çabuk, “MARAS: Mutual Authentication and Role-Based Authorization Scheme for Lightweight Internet of Things Applications,” Sensors, vol. 23, no. 12, 2023, doi: 10.3390/s23125674.
[14] T. Lawrence, I. Ali, T. Christopher, and F. Li, “A bandwidth efficient HMAC-based authentication scheme for network coding,” J. Inf. Secur. Appl., vol. 55, 2020, doi: 10.1016/j.jisa.2020.102658.
[15] L. A. Lizama-Pérez, “Digital signatures over HMAC entangled chains,” Eng. Sci. Technol. an Int. J., vol. 32, 2022, doi: 10.1016/j.jestch.2021.11.002.
[16] I. Petrescu, E. Niculae, V. Vulturescu, A. Dimitrescu, and L. M. Ungureanu, “Transport and Application Layer Protocols for IoT : Comprehensive Review,” Technologies, vol. 15, 2025, doi: 10.3390/technologies13120583.
[17] B. Mishra and A. Kertesz, “The Use of MQTT in M2M and IoT Systems : A Survey,” IEEE Access, vol. 8, pp. 201071–201086, 2020, doi: 10.1109/ACCESS.2020.3035849.
[18] F. Montori, L. Gigli, L. Sciullo, and M. D. I. Felice, “LA-MQTT : Location-aware Publish-subscribe Communications for the Internet of Things,” ACM Trans. Internet Things, vol. 3, no. 3, 2023.
[19] M. Kashyap, A. K. Dev, and V. Sharma, “Implementation and analysis of EMQX broker for MQTT protocol in the Internet of Things,” e-Prime - Adv. Electr. Eng. Electron. Energy, vol. 10, no. May, p. 100846, 2024, doi: 10.1016/j.prime.2024.100846.
[20] B. Dowling, M. Fischlin, F. Günther, and D. Stebila, “A Cryptographic Analysis of the TLS 1 . 3 Handshake Protocol,” J. Cryptol., vol. 34, no. 4, pp. 1–69, 2021, doi: 10.1007/s00145-021-09384-1.
[21] L. Sharma, M. Jangid, S. Roy, and A. Jain, “Blake2 Enhanced Cyclic Integrity and Authentication Protocol for SDN IoT Security,” Discov. Internet Things, 2026.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Huda Daulan, Firmansyah Maulana, Rohmat Gunawan
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors who publish articles in JUTEI agree on the following rules:
1. The author grants non exclusive royalty free rights, and is willing to publish articles online and complete (full access). With such rights JUTEI reserves the right to save, transfers, manages in various forms, maintains and publishes articles while keeping the author's name as the copyright owner.
2. Each author contained in the article has contributed fully to the substance and intellectual, and is accountable to the public. If in the future there is a copyright infringement notification then this will be responsibility of the author, not JUTEI.
