A prospective observational evaluation of the COMPOSER-LLM system was conducted across two distinct Emergency Departments within the University of California, San Diego Health system from February, 2023 through October, 2025. The system analyzes structured EHR data by integrating a large language model (Mixtral 8x7B), which is activated when the baseline COMPOSER model generates a prediction score within a pre-defined high-uncertainty range. Retrieval-Augmented Generation (RAG) was used to improve the LLM output, resulting in a significant reduction in false alarms per patient‑hour and improved specificity of the alert. The objective was to assess clinical acceptance of COMPOSER-LLM best practice advisory alerts by integrating analyses of BPA acknowledgment patterns with healthcare worker survey responses. The study sought to test the hypothesis that incorporating the LLM will significantly reduce the proportion of BPAs dismissed as “no infection suspected” and that nurses with less experience will perceive the BPA as more useful. The baseline COMPOSER system previously resulted in a 17% relative decrease in sepsis mortality at University of California, San Diego Health.