Abstract

The growing complexity of modern healthcare demands intelligent systems capable of synthesizing heterogeneous data streams into actionable clinical insights. This paper presents AASMA (Adaptive Agent-based Smart Multimodal Assistant), a comprehensive AI-driven healthcare platform that integrates Electronic Health Records (EHR) with real-time wearable telemetry through a novel multimodal fusion engine. AASMA orchestrates twelve specialized AI agents—spanning exacerbation risk prediction, personalized anomaly detection, clinician burnout monitoring, drug repositioning, behavioral adherence nudging, microbiome forecasting, environmental alerting, algorithmic fairness auditing, federated learning, active learning, counterfactual simulation, and natural language interaction—to deliver proactive, explainable, and equitable care. The risk prediction subsystem employs XGBoost with SHAP (SHapley Additive exPlanations) for transparent assessment, while a hybrid Isolation Forest–Autoencoder architecture detects subtle baseline anomalies. Federated learning enables privacy-preserving collaborative model improvement across institutions. Experimental evaluation on a synthetic cohort of 2,000 patient profiles demonstrates that AASMA’s multimodal fusion achieves a 15.2% improvement in F1-score over single-source baselines, with the counterfactual simulator enabling clinicians to reduce adverse event rates by 18.7% through what-if scenario analysis. The platform is deployed as a containerized microservices architecture using Docker, Kubernetes, and Apache Kafka for real-time event streaming.

Authors

Govind Sharma, Shaurya Pal Singh, Disha Basu, B.V.A.N.S.S. Prabhakar Rao
Vellore Institute of Technology, Chennai, India

Keywords

Healthcare AI, Multimodal Fusion, Explainable AI (XAI), Federated Learning, Anomaly Detection, Behavioral Nudging, SHAP, XGBoost, Microservices Architecture