Distributed Machine Learning Algorithms for Resource-Constrained Heterogeneous Internet-of-Things Environments

Authors

Ahmed ImteajFollow

Document Type

Dissertation

Degree

Doctor of Philosophy (PhD)

Major/Program

Computer Science

First Advisor's Name

M. Hadi Amini

First Advisor's Committee Title

Committee Chair

Second Advisor's Name

Sundaraja Sitharama Iyengar

Second Advisor's Committee Title

Committee Member

Third Advisor's Name

Leonardo Bobadilla

Third Advisor's Committee Title

Committee Member

Fourth Advisor's Name

Ananda M. Mondal

Fourth Advisor's Committee Title

Committee Member

Fifth Advisor's Name

B. M. Golam Kibria

Fifth Advisor's Committee Title

Committee Member

Keywords

Distributed machine learning, Internet-of-Things, Federated learning, resource-constrained device, heterogeneity, prediction model, convergence.

Date of Defense

5-20-2022

Abstract

With the improvement of network infrastructures and advancement of IoT technologies, now it is desirable to perform computation at the edges, rather than sharing data with a central fusion center, which is privacy-intrusive. Both conventional (centralized) and distributed machine learning (ML) algorithms fail to address underlying challenges related to users’ privacy or capturing global knowledge of the whole network. To properly handle such challenges, a recently invented distributed ML technique, called Federated Learning was invented that shows us a pathway to construct a global model without exposing any user’s private data through on-device model training utilizing edge resources. However, FL may face various challenges due to the lack of a convenient mechanism to prepare a federated dataset, and also the heterogeneous nature of the resource-constrained agents such as systems, statistical and model heterogeneity. We developed a distributed sensing mechanism through which any federated agents can be triggered and activated for sensing the environment. That novel approach shows a pathway to carry out the FL process in a real-world environment. Following this, we developed an FL model, FedAR by monitoring agent activities and leveraging available local computing resources, particularly for resource-constrained IoT devices (e.g., mobile robots), to accelerate the learning process. Besides, we propose a tri-layer FL framework, FedPARL that helps resource-constrained FL agents consume less resources during training, and avoid untrustworthy and out-of-resource agents (e.g., low battery life) during agent selection for training and perform variable local epochs based on the agent’s resource availability. We perform model pruning to reduce the size of the agent model that is effective for an FL-IoT setting. Afterwards, we proposed a coupling of distillation and dynamic local task allocation technique through which we can effectively handle model and systems heterogeneity of FL-IoT environments. Further, we focused on applying our developed distributed ML algorithm to improve the resilience of critical infrastructures through knowledge exchange. We extended the work by proposing a novel technique, FedResilience, to handle weak critical infrastructure agents by enabling partial computational tasks from the resource-constrained agents and exchanging information without sharing any raw data. Besides, we implemented two other FL applications to recognize human activities and forecast customers’ financial distress in resource-constrained environments. From the experience we gathered over the past few years on FL, we conducted a comprehensive survey on FL, particularly for resource-constrained IoT environments that discussed the existing FL works, present challenges, and their potential solutions, applications, and future research directions in this domain.

Identifier

FIDC010711

ORCID

0000-0002-6975-3997

Previously Published In

1. Ahmed Imteaj, U. Thakker, S. Wang, J. Li and M. H. Amini, “A Survey on Federated Learning for Resource-Constrained IoT Devices,” in IEEE Internet of Things Journal, vol. 9, no. 1, pp. 1-24, 1 Jan.1, 2022, doi: 10.1109/JIOT.2021.3095077.
2. Ahmed Imteaj, and M. Hadi Amini, “FedAR: Activity and Resource-Aware Federated Learning Model for Distributed Mobile Robots”, in Proceedings of the 19th IEEE International Conference Machine Learning And Applications, 2020, Miami, USA.
3. Ahmed Imteaj and M. Hadi Amini. “FedPARL: Client Activity and Resource-Oriented Lightweight Federated Learning Model for Resource-Constrained Heterogeneous IoT Environment.” Frontiers in Communications and Networks 2 (2021): 10.
4. Ahmed Imteaj and M. Hadi Amini. “Leveraging Asynchronous Federated Learning to Predict Customers Financial Distress.” Intelligent Systems with Applications (2022): 200064.
5. Ahmed Imteaj, Irfan Khan, Javad Khazaei, M. Hadi Amini, “FedResilience: A Federated Learning Application to Improve Resilience of Resource-Constrained Critical Infrastructures,” Electronics 2021, 10(16):1917.
6. Ahmed Imteaj, M. Hadi Amini, Distributed Sensing Using Smart End-user Devices: Pathway to Federated Learning for Autonomous IoT, 2019 IEEE Conference on Computational Science & Computational Intelligence, 2019.
7. Ahmed Imteaj, M. Hadi Amini, and Javad Mohammadi. “Leveraging decentralized artificial intelligence to enhance resilience of energy networks.” In 2020 IEEE Power & Energy Society General Meeting (PESGM), pp. 1-5. IEEE, 2020.
8. *M. Hyman, *Calvin Mark, *Ahmed Imteaj, Hamed Ghiaie, Shabnam Rezapour, Arif M. Sadri, M. Hadi Amini, “Data Analytics to Evaluate the Impact of Infectious Disease on Economy: Case Study of COVID-19 Pandemic,” Patterns Journal (2021). [*Authors contributed equally]
9. M. Hadi Amini, Ahmed Imteaj, and Panos Pardalos, “Interdependent Networks: A Data Science Perspective”, Patterns Journal (2020).
10. Ahmed Imteaj, Raghad Alabagi and M. Hadi Amini, “Exploiting Federated Learning Technique to Recognize Human Activities in Resource-Constrained Environment”, in Proceedings of the 13th International Conference on Intelligent Human Computer Interaction (IHCI-2021), 2021, Ohio, USA.
11. Ahmed Imteaj, Distributed machine learning for collaborative mobile robots: PhD forum abstract, in Proceedings of the 18th Conference on Embedded Networked Sensor Systems (SenSys’20), Association for Computing Machinery, New York, NY, USA, 798–799, 2020.
12. Ahmed Imteaj, Urmish Thakker, Shiqiang Wang, Jian Li, and M. Hadi Amini. “Federated Learning for Resource-Constrained IoT Devices: Panoramas and State-of-the-art.” Accepted in Federated and Transfer Learning (2022).
13. M. Hadi Amini, Ahmed Imteaj, and Javad Mohammadi, “Distributed Machine Learning for Resilient Operation of Electric Systems”, in Proceedings of International Conference on Smart Energy Systems and Technologies (SEST)(2020).

Recommended Citation

Imteaj, Ahmed, "Distributed Machine Learning Algorithms for Resource-Constrained Heterogeneous Internet-of-Things Environments" (2022). FIU Electronic Theses and Dissertations. 5087.
https://digitalcommons.fiu.edu/etd/5087